Technical document translation is most afraid of translating the code along with it

This paper uses synthetic samples to demonstrate the processing method. The code has been verified using examples in an isolated environment; this does not guarantee effectiveness for all real files or products. Before processing important files, please retain the original copies and verify them against the boundaries and acceptance criteria specified in the text. puts user_id is translated as “user identifier,” or <div class="container"> Localizing the class attribute in [ ] is the most costly and hardest-to-diagnose error in technical documentation translation. Such issues typically do not cause the program to crash, but they can result in copied-and-pasted code that directly throws errors, or render API documentation examples completely inoperable. The key to solving this problem lies not in how intelligent the translation engine is, but in whether the preprocessing stage can accurately physically isolate “natural language” from “machine language” within the DOM tree. Why regular expressions cannot handle code protection Many teams have tried using regular expressions to replace code snippets before translation, but this approach is extremely fragile in HTML structures. HTML is not linear text; it is a nested tree structure. A <code> The tag may contain <span> highlighted, or enclosed by <a> Tags are wrapped to form links. Regular expressions cannot comprehend this nested semantics, making it very easy to cause tag-closing errors or overlook boundary cases. uses parsers such as Beautiful Soup, which can elevate code protection from “string matching” to “node traversal.” According to the official documentation of Beautiful Soup 4.15.0, we no longer search for specific character patterns; instead, we directly manipulate Tag objects in the DOM tree. By identifying pre 、 code 、 kbd 、 samp and other semantic tags, as well as those with specific classes (such as highlight 、 language-python ) elements, we can precisely target the regions that need to be masked, no matter how complex their internal structure may be. synthesis example: correct output and boundary trap To verify the protection logic, we need to construct a set of well-defined synthetic test cases. Please note that the following inputs are designed solely for demonstrating the parsing logic; they do not represent real production data and do not involve any external file reads. synthetic input HTML: <p>请运行 <code class="cmd">pip install -r requirements.txt</code> 来安装依赖。</p> <pre><code># 初始化配置 config = {"timeout": 30}</code></pre> <p>注意：<var>timeout</var> 参数单位为秒。</p> Expected correct extraction result: The translation system should only receive the following plain-text paragraphs for translation, with code sections either replaced by placeholders or completely stripped out: 1. "Please run [CODE_0] to install dependencies." 2. "Note: The unit of the [VAR_0] parameter is seconds." <pre> The content within the block should not appear in the text stream to be translated, or it should remain unchanged at the byte level upon export. Typical errors/boundary inputs and handling: * nested tag: <code><b>bold_code</b></code> . If only extracting code.string will return None, because the content is in the child node b In. Must use get_text() or recursively traverse descendants. * attribute value leakage: <a href="/api/v1/users">用户接口</a> . The translation target is solely “user interface”, href attribute values must be locked. If the entire tag is converted to a string and passed to the translator, the URL may be corrupted. * pseudocode tag: <div class="code-snippet"> . This is not a standard semantic tag; it must be additionally captured via a CSS class name whitelist, otherwise it will be translated as an ordinary paragraph. Implementation of code node identification based on the DOM tree The following Python example demonstrates how to use Beautiful Soup to traverse the document tree, distinguishing translatable text from protected code. This script is a purely in-memory instructional demonstration and does not involve any network requests, shell commands, or file writes; it is recommended to run it in beautifulsoup4>=4.15.0 Running in the environment. from bs4 import BeautifulSoup, NavigableString, Tag # 合成示例：包含内联代码、代码块及普通文本（仅用于演示，非真实数据） HTML_SAMPLE = """ <article> <p>使用 <code class="cli">kubectl get pods</code> 查看状态。</p> <pre><code>def hello(): print("Hello World")</code></pre> <p>返回值 <var>result</var> 可能为空。</p> </article> """ # 定义需要整体保护的标签集合 PROTECTED_TAGS = {'pre', 'code', 'kbd', 'samp', 'var'} def extract_translatable_segments(html_content: str) -> list[dict]: """ 提取HTML中的文本片段并标记是否可翻译。 注意：此为教学演示代码，生产环境需增加占位符生成与属性保护逻辑。 """ soup = BeautifulSoup(html_content, 'html.parser') segments = [] # 遍历所有文本节点，而非仅顶层标签 for element in soup.descendants: if isinstance(element, NavigableString): # 检查父链上是否存在受保护标签 parent_chain = list(element.parents) is_protected = any( p.name in PROTECTED_TAGS for p in parent_chain if isinstance(p, Tag) ) text = str(element).strip() if not text: continue segments.append({ 'text': text, 'translatable': not is_protected, 'context': element.parent.name if element.parent else 'root' }) return segments # 执行提取并断言预期行为 results = extract_translatable_segments(HTML_SAMPLE) translatable_texts = [s['text'] for s in results if s['translatable']] protected_texts = [s['text'] for s in results if not s['translatable']] # 验证：普通文本应被标记为可翻译 assert any('查看状态' in t for t in translatable_texts), "普通文本未被识别" # 验证：代码内容不应出现在可翻译列表中 assert not any('kubectl' in t for t in translatable_texts), "内联代码泄露至翻译区" assert not any('print' in t for t in translatable_texts), "代码块内容泄露至翻译区" # 验证：受保护内容被正确归类 assert any('kubectl' in t for t in protected_texts), "内联代码未被保护" print(f"可翻译片段数: {len(translatable_texts)}") print(f"受保护片段数: {len(protected_texts)}") The core of this code lies in element.parents chained check. It ensures that even if <code> is nested within <p> In this case, the text nodes inside it will also be correctly marked as non-translatable. Note that this is merely an external demonstration of extraction and validation, not a complete translation-replacement system; in actual production, placeholder backfilling and attribute protection must also be handled. Acceptance Criteria and Manual Review Boundary code protection cannot be judged solely by “running without errors.” The following table lists the acceptance actions that should be performed during actual delivery, as well as scenarios that automated scripts cannot cover and must therefore be subject to manual review. check item automated verification method Manual review trigger condition processing decision tag integrity compares the hash of the DOM tree structure before and after translation translation contains unclosed tags or improperly nested refuses to deliver, reverting to the original text for reprocessing attribute value retains performs a regular expression check to verify whether the key attribute (href/src/id) has been modified dynamic URL containing path parameters or a query string is flagged as high-risk and confirmed by the developer placeholder restores checks that the number of placeholders in the translation matches that of the original text placeholder position does not conform to the target language syntax reorders the sentence structure and splits it into separate sentences when necessary pseudocode recognition checks the class whitelist hit rate custom component or non-standard code container supplementary rules or manual annotation Comment translation checks whether comments within the code block have been skipped comments contain important explanations of business logic extract the annotation separately for translation Scope of Non-Applicability Statement: The aforementioned automated validation applies only to well-structured, semantically marked-up HTML. For plain text generated by Markdown rendering that lacks semantic tags, <div> + <span> code blocks, or single-line HTML that has been obfuscated and compressed, may not allow the parser to correctly identify code boundaries. Such documents must first undergo standardized formatting cleanup, or be handled directly by human reviewers. Furthermore, this validation logic applies only to ensuring that “code is not translated” and does not guarantee the quality of the translated natural language. References and Next Steps Beautiful Soup Official Documentation uses this checklist for document translation The body text, links, code, and structure of the web document should be processed separately. Include the protected sections in the acceptance checklist, then evaluate the readability and visual presentation of the translated content. OmniDoc (Product Entry) provides an entry point for translating multi-format documents, and the official website highlights its OCR, terminology database, and bilingual alignment features. For teams that wish to start processing directly from files, this is a path worth evaluating with real-world samples. Please use de-identified files that you are authorized to upload, and verify the results against the checklist provided in this document; we do not claim that OmniDoc utilizes the open-source libraries or internal algorithms referenced herein, nor do we substitute examples for product performance testing. - 1 -