PDF parsing

PDF can be viewed but not copied; it may not be a scanned document

lacks text, so it cannot be directly determined to have a “normal text layer,” nor can it be directly classified as a scanned document. Below, we separately rec

阅读文本 / Reading text
This paper uses synthetic samples to demonstrate the processing method. The code has been verified using examples in an isolated environment; this does not guarantee effectiveness for all real files or products. Before processing important files, please retain the original copies and verify them against the boundaries and acceptance criteria outlined in this document. Which problem does this paper address first? lacks text, so it cannot be directly determined to have a “normal text layer,” nor can it be directly classified as a scanned document. Below, we separately record the text, substitute characters, and page image references, outputting the status as “pending review”; moreover, image references do not prove that the image covers the entire page or that the text is recognizable. self-contained synthetic example dependencies are listed in the import list. The code constructs inputs in memory; runtime logs are provided separately and do not constitute a guarantee for all real-world files. from io import BytesIO from PIL import Image import pymupdf def inspect(page): text = page.get_text().strip() images = page.get_images() if "\ufffd" in text: state = "review_replacement_characters" elif text: state = "text_present_needs_semantic_review" elif images: state = "image_without_extracted_text" else: state = "empty_text_needs_review" return {"state": state, "characters": len(text), "image_refs": len(images)} with pymupdf.open() as doc: doc.new_page().insert_text((50, 50), "Text sample") doc.new_page() picture = BytesIO() Image.new("RGB", (16, 16), "red").save(picture, format="PNG") doc.new_page().insert_image(pymupdf.Rect(10, 10, 100, 100), stream=picture.getvalue()) assert inspect(doc[0])["state"] == "text_present_needs_semantic_review" assert inspect(doc[1])["state"] == "empty_text_needs_review" assert inspect(doc[2])["state"] == "image_without_extracted_text" assert inspect(doc[2])["characters"] == 0 PDF can be viewed but not copied; it may not be a scanned document Users can normally see the text in a browser or reader, but when copying it, they encounter garbled characters, blank spaces, or irrelevant symbols. This is usually neither file corruption nor a purely scanned document. The root cause of this phenomenon lies in the absence or error of the PDF’s ToUnicode CMap, which maps glyphs to Unicode. The rendering engine relies on glyph indices to render visual content, while clipboard and text-extraction tools depend on the mapping table to restore semantic characters. When these two mechanisms are disconnected, a “visually intact but machine-unreadable” bad-text layer emerges. Before proceeding with document translation or structured extraction, it is essential to first determine whether the issue stems from a mapping failure or genuine image-based content; otherwise, subsequent OCR or NLP processes will run needlessly on an incorrect premise. Glyph index and the fracture mechanism of Unicode mapping The PDF specification permits fonts to embed only glyph outlines and width information, without including character encoding mappings. If the software generating the PDF fails to correctly write the ToUnicode table, or if a custom encoding is used without providing decoding rules, the extractor will obtain only the raw glyph IDs (CIDs). These IDs are meaningful to rendering engines but entirely useless for text processing. Some tools attempt to infer mappings using heuristic algorithms, often producing outputs that appear reasonable yet are actually incorrect—for example, converting Chinese punctuation into Latin letters or replacing digits with control characters. Such “pseudo-text” is even more dangerous than plain whitespace, as it can pass null‑check validations while silently contaminating downstream processes. The key to detecting such issues lies not in whether text can be selected, but in verifying whether the extracted string is semantically consistent with the visual content. PyMuPDF Memory Composition Diagnostics and Assertion Verification In the code above, diagnose_text_layer function via get_text("text") obtains the original text and utilizes unicodedata.category filters control characters and whitespace, calculating the proportion of valid content. At the same time, through get_text("blocks") detects the presence of image blocks. Two assertions verify, respectively, the high printability rate (>0.9) for normal documents and a zero character count for scanned documents, thereby covering the core branching logic. Note: This diagnostic is based solely on statistical features; encrypted fonts or special encodings still require manual review. Acceptance Decision Table and Manual Review Boundary Reference link and next steps PyMuPDF Text Extraction Explained: https://pymupdf.readthedocs.io/en/latest/app1.html PyMuPDF Official Documentation Homepage: https://pymupdf.readthedocs.io/ Text and image signal splitting signal Next, check should not draw the conclusion has printable text verifies semantics, order, and source Being able to extract it means it’s correct. has a substitute character lookback encoding and font mapping All characters have been lost has an image but no text checks whether the image contains text must be a full-page scan Neither text nor images were detected for View page rendering content must be a blank page uses this checklist for document translation first ensures that the page order, source locations, and text integrity have been thoroughly checked, then submits the same de-identified sample to the translation process. After translation, it is still necessary to verify that the headings, body text, and figure captions correspond correctly; one should not rely solely on whether the text flows smoothly. OmniDoc (product entry point) provides an entry point for translating multi-format documents; the official website highlights its OCR, terminology database, and bilingual comparison features. For teams that prefer to start processing directly from files, this is a path worth evaluating with real-world samples. Please use de-identified files that you are authorized to upload, and verify the results against the checklist provided in this document. We do not claim that OmniDoc utilizes the open-source libraries or internal algorithms referenced herein, nor do we substitute examples for formal product performance testing. - 1 -
Updated 2026-10-08