being able to parse HTML does not mean it can safely render HTML

This paper uses synthetic samples to demonstrate the processing method. The code has been verified using examples in an isolated environment; this does not guarantee effectiveness for all real files or products. Before processing important files, please retain the original copies and verify them against the boundaries and acceptance criteria outlined in this document. When you have successfully parsed a section containing <img src=x onerror=alert(1)> When the string is deserialized, this malicious code remains intact in the output. This is one of the most dangerous misconceptions in many document-processing pipelines: the belief that a “parser” is equivalent to a “sanitizer.” BeautifulSoup was designed to extract and modify data in a fault-tolerant manner, not to defend against XSS attacks. It faithfully preserves the original tag structure, including those hazardous attributes that browsers would otherwise interpret as executable instructions. Before embedding external HTML into an internal system or translation workflow, it is essential to establish a security-cleaning layer that operates independently of the parser. Misalignment of Parsing Capabilities and Security Responsibilities The official BeautifulSoup documentation explicitly defines it as “a library for extracting data from HTML and XML files,” emphasizing that, in conjunction with a parser, it provides idiomatic methods for navigating, searching, and modifying the tree structure. This positioning ensures that it remains neutral regarding the semantic safety of the content. When input <a href="javascript:void(0)">Click</a> At that time, BS4 will correctly construct an element with href The Tag object of the attribute, but it does not determine whether the protocol should be permitted. In document-processing scenarios, this neutrality is both an advantage and a risk. The advantage lies in its ability to handle various irregular legacy tags; the risk arises when developers mistakenly assume that “processed by BS4” equates to “sanitized,” leading them to render unverified HTML directly in the preview interface or write it into the translated deliverables. True secure sanitization requires an explicit whitelist strategy: only known-safe tag names, attribute names, and their value formats are retained after validation, while everything else is stripped out. This is not a built-in feature of BS4 but rather a business logic that users must layer on top of the parser. boundary input recognition in the synthesis example To illustrate the necessity of the cleansing logic, we construct a set of synthetic samples that cover typical attack vectors and legitimate content. Note that the following inputs are designed solely to demonstrate boundary conditions; they do not reflect real-world traffic statistics nor represent industry-standard thresholds. If you need to test actual documents, please first anonymize the business data by removing personal information and sensitive links before using it as input. correct input ： <p>See <a href="https://example.com/doc">reference</a>.</p> The expected output should fully preserve the paragraph and the link, because p and a is on the whitelist, and href uses the HTTPS protocol. error/boundary input 1 ： <div onclick="steal()">Safe text</div> Even if the text content is harmless, onclick It belongs to the event handler attribute; this attribute must be removed in its entirety, leaving only <div>Safe text</div> . If div If it is not in the tag whitelist, the entire tag should be replaced with its plain text content. error/boundary input 2 ： <a href="data:text/html,<script>alert(1)</script>">Link</a> data: URIs can carry executable content; even if the tag itself is valid, that href The value should also be set to null or deleted. The sanitizer cannot merely check whether the protocol prefix is http , it is also necessary to verify that the decoded content does not constitute a nested execution context. These cases demonstrate that security decisions are made during the attribute-level validation phase following parsing, rather than during the parsing process itself. BS4 is responsible for converting strings into iterable objects, while the sanitization logic determines which objects may remain intact. Whitelist-based memory sanitization implementation The following Python example demonstrates how to accomplish this process in memory. This code is for educational demonstration purposes only and does not handle multi-valued attributes, case normalization, entity encoding, or other such details; it must not be used directly in a production environment . Code depends on beautifulsoup4 (requires manual installation; version locking is recommended), does not initiate network requests and does not read from or write to the file system. All assertions are intended solely to express expected behavior; the actual execution results depend on your environment and the versions of dependencies. # 教学示例：演示BS4配合白名单清洗原理，非生产级实现 from bs4 import BeautifulSoup, Comment import re ALLOWED_TAGS = {'p', 'a', 'b', 'i', 'ul', 'li'} ALLOWED_ATTRS = {'href'} SAFE_URL_PATTERN = re.compile(r'^https?://', re.IGNORECASE) def sanitize_html(raw: str) -> str: soup = BeautifulSoup(raw, 'html.parser') # 移除注释 for comment in soup.find_all(string=lambda t: isinstance(t, Comment)): comment.extract() # 遍历所有标签 for tag in soup.find_all(True): if tag.name not in ALLOWED_TAGS: tag.unwrap() # 保留子节点，移除标签本身 continue # 清理属性 attrs_to_remove = [k for k in tag.attrs if k not in ALLOWED_ATTRS] for attr in attrs_to_remove: del tag[attr] # 校验剩余安全属性值 if 'href' in tag.attrs: href = tag['href'] if not isinstance(href, str) or not SAFE_URL_PATTERN.match(href.strip()): del tag['href'] return str(soup) # 合成测试用例（仅用于理解原理） safe = '<p>Hello <a href="https://ok.com">link</a></p>' unsafe_event = '<p onclick="x()">text</p>' unsafe_uri = '<a href="javascript:alert(1)">bad</a>' assert sanitize_html(safe) == '<p>Hello <a href="https://ok.com">link</a></p>' assert 'onclick' not in sanitize_html(unsafe_event) assert 'href' not in sanitize_html(unsafe_uri) The production environment should consider using a dedicated cleanroom (such as bleach ), and externalize the whitelist configuration. The key point is that the cleansing logic must explicitly specify “what is allowed,” rather than attempting to enumerate “what is prohibited.” Acceptance Criteria and Manual Review Trigger Points Code-level assertions can only verify whether predefined rules have been enforced; they cannot cover all real-world exceptions. The table below lists typical scenarios observed in actual document processing along with corresponding handling decisions, delineating the boundary between automated processing and human intervention. Observation of the phenomenon automatic processing decision Whether manual review is required Note tag name is not on the whitelist but contains important text unwrap retained text no ensures no semantic loss href is a relative path or mailto: removes the href attribute is may be a valid business link and requires manual confirmation JS with HTML entity encoding in attribute values Reject by string matching is needs to be verified whether it is a false positive nested too deeply, causing parsing to time out truncates or skips this fragment is prevents DoS, but may drop content The text length drops sharply after cleaning >30% is marked as suspicious is may excessively strip structural information Special note: “Code demonstration passed” does not equate to “end-to-end results are trustworthy.” In the table above, a “Yes” entry indicates that the automated process should be paused and handed over to human review; particularly in translation delivery scenarios, excessive cleaning may result in broken glossary references or invalidation of formatting conventions. Reference and Next Step Beautiful Soup Documentation Next steps: Run the aforementioned synthesis example in your locked-down dependency environment to verify that the basic cleansing logic meets expectations; then prepare a de-identified HTML sample containing real business tags, test it item by item against the acceptance checklist, and record the trigger ratios for automated processing versus manual review before deciding whether to integrate it into the formal translation workflow. uses this checklist for document translation The body text, links, code, and structure of the web document should be processed separately. Include the protected sections in the acceptance checklist, then evaluate the post-translation readability and page display. OmniDoc (product entry point) provides an entry point for translating multi-format documents, and the official website highlights its OCR, terminology database, and bilingual alignment features. For teams that wish to start processing directly from files, this is a path worth evaluating with real-world samples. Please use de-identified files that you are authorized to upload, and verify the results against the checklist provided in this document; we do not claim that OmniDoc utilizes the open-source libraries or internal algorithms referenced herein, nor do we substitute examples for product performance testing. - 1 -