阅读文本 / Reading text
This paper uses synthetic samples to demonstrate the processing method. The code has been verified using examples in an isolated environment; this does not guarantee effectiveness for all real files or products. Before processing important files, please retain the original copies and verify them against the boundaries and acceptance criteria outlined in this document.
Which problem does this paper address first?
lacks text, so it cannot be directly determined to have a “normal text layer,” nor can it be directly classified as a scanned document. Below, we separately record the text, substitute characters, and page image references, outputting the status as “pending review”; moreover, image references do not prove that the image covers the entire page or that the text is recognizable.
self-contained synthetic example
dependencies are listed in the import list. The code constructs inputs in memory; runtime logs are provided separately and do not constitute a guarantee for all real-world files.
from io import BytesIO
from PIL import Image
import pymupdf
def inspect(page):
text = page.get_text().strip()
images = page.get_images()
if "\ufffd" in text:
state = "review_replacement_characters"
elif text:
state = "text_present_needs_semantic_review"
elif images:
state = "image_without_extracted_text"
else:
state = "empty_text_needs_review"
return {"state": state, "characters": len(text), "image_refs": len(images)}
with pymupdf.open() as doc:
doc.new_page().insert_text((50, 50), "Text sample")
doc.new_page()
picture = BytesIO()
Image.new("RGB", (16, 16), "red").save(picture, format="PNG")
doc.new_page().insert_image(pymupdf.Rect(10, 10, 100, 100), stream=picture.getvalue())
assert inspect(doc[0])["state"] == "text_present_needs_semantic_review"
assert inspect(doc[1])["state"] == "empty_text_needs_review"
assert inspect(doc[2])["state"] == "image_without_extracted_text"
assert inspect(doc[2])["characters"] == 0
PDF can be viewed but not copied; it may not be a scanned document
Users can normally see the text in a browser or reader, but when copying it, they encounter garbled characters, blank spaces, or irrelevant symbols. This is usually neither file corruption nor a purely scanned document. The root cause of this phenomenon lies in the absence or error of the PDF’s ToUnicode CMap, which maps glyphs to Unicode. The rendering engine relies on glyph indices to render visual content, while clipboard and text-extraction tools depend on the mapping table to restore semantic characters. When these two mechanisms are disconnected, a “visually intact but machine-unreadable” bad-text layer emerges. Before proceeding with document translation or structured extraction, it is essential to first determine whether the issue stems from a mapping failure or genuine image-based content; otherwise, subsequent OCR or NLP processes will run needlessly on an incorrect premise.
Glyph index and the fracture mechanism of Unicode mapping
The PDF specification permits fonts to embed only glyph outlines and width information, without including character encoding mappings. If the software generating the PDF fails to correctly write the ToUnicode table, or if a custom encoding is used without providing decoding rules, the extractor will obtain only the raw glyph IDs (CIDs). These IDs are meaningful to rendering engines but entirely useless for text processing. Some tools attempt to infer mappings using heuristic algorithms, often producing outputs that appear reasonable yet are actually incorrect—for example, converting Chinese punctuation into Latin letters or replacing digits with control characters. Such “pseudo-text” is even more dangerous than plain whitespace, as it can pass null‑check validations while silently contaminating downstream processes. The key to detecting such issues lies not in whether text can be selected, but in verifying whether the extracted string is semantically consistent with the visual content.
PyMuPDF Memory Composition Diagnostics and Assertion Verification
In the code above,
diagnose_text_layer
function via
get_text("text")
obtains the original text and utilizes
unicodedata.category
filters control characters and whitespace, calculating the proportion of valid content. At the same time, through
get_text("blocks")
detects the presence of image blocks. Two assertions verify, respectively, the high printability rate (>0.9) for normal documents and a zero character count for scanned documents, thereby covering the core branching logic. Note: This diagnostic is based solely on statistical features; encrypted fonts or special encodings still require manual review.
Acceptance Decision Table and Manual Review Boundary
Reference link and next steps
PyMuPDF Text Extraction Explained: https://pymupdf.readthedocs.io/en/latest/app1.html
PyMuPDF Official Documentation Homepage: https://pymupdf.readthedocs.io/
Text and image signal splitting
signal
Next, check
should not draw the conclusion
has printable text
verifies semantics, order, and source
Being able to extract it means it’s correct.
has a substitute character
lookback encoding and font mapping
All characters have been lost
has an image but no text
checks whether the image contains text
must be a full-page scan
Neither text nor images were detected for
View page rendering content
must be a blank page
uses this checklist for document translation
first ensures that the page order, source locations, and text integrity have been thoroughly checked, then submits the same de-identified sample to the translation process. After translation, it is still necessary to verify that the headings, body text, and figure captions correspond correctly; one should not rely solely on whether the text flows smoothly.
OmniDoc (product entry point)
provides an entry point for translating multi-format documents; the official website highlights its OCR, terminology database, and bilingual comparison features. For teams that prefer to start processing directly from files, this is a path worth evaluating with real-world samples. Please use de-identified files that you are authorized to upload, and verify the results against the checklist provided in this document. We do not claim that OmniDoc utilizes the open-source libraries or internal algorithms referenced herein, nor do we substitute examples for formal product performance testing.
- 1 -
Updated 2026-10-08