dots.ocr is an open-source vision-language transformer mannequin developed for multilingual doc structure parsing and optical character recognition (OCR). It performs each structure detection and content material recognition inside a single structure, supporting over 100 languages and all kinds of structured and unstructured doc varieties.
Structure
- Unified Mannequin: dots.ocr combines structure detection and content material recognition right into a single transformer-based neural community. This eliminates the complexity of separate detection and OCR pipelines, permitting customers to change duties by adjusting enter prompts.
- Parameters: The mannequin incorporates 1.7 billion parameters, balancing computational effectivity with efficiency for many sensible situations.
- Enter Flexibility: Inputs may be picture recordsdata or PDF paperwork. The mannequin options preprocessing choices (comparable to fitz_preprocess) for optimizing high quality on low-resolution or dense multi-page recordsdata.
Capabilities
- Multilingual: dots.ocr is skilled on datasets spanning greater than 100 languages, together with main world languages and fewer widespread scripts, reflecting broad multilingual assist.
- Content material Extraction: The mannequin extracts plain textual content, tabular information, mathematical formulation (in LaTeX), and preserves studying order inside paperwork. Output codecs embody structured JSON, Markdown, and HTML, relying on the structure and content material sort.
- Preserves Construction: dots.ocr maintains doc construction, together with desk boundaries, method areas, and picture placements, guaranteeing extracted information stays trustworthy to the unique doc.
Benchmark Efficiency
dots.ocr has been evaluated in opposition to trendy doc AI methods, with outcomes summarized beneath:
| Benchmark | dots.ocr | Gemini2.5-Professional |
|---|---|---|
| Desk TEDS accuracy | 88.6% | 85.8% |
| Textual content edit distance | 0.032 | 0.055 |
- Tables: Outperforms Gemini2.5-Professional in desk parsing accuracy.
- Textual content: Demonstrates decrease textual content edit distance (indicating increased precision).
- Formulation and Structure: Matches or exceeds main fashions in method recognition and doc construction reconstruction.
Deployment and Integration
- Open-Supply: Launched underneath the MIT license, with supply, documentation, and pre-trained fashions accessible on GitHub. The repository offers set up directions for pip, Conda, and Docker-based deployments.
- API and Scripting: Helps versatile activity configuration through immediate templates. The mannequin can be utilized interactively or inside automated pipelines for batch doc processing.
- Output Codecs: Extracted outcomes are provided in structured JSON for programmatic use, with choices for Markdown and HTML the place applicable. Visualization scripts allow inspection of detected layouts.
Conclusion
dots.ocr offers a technical resolution for high-accuracy, multilingual doc parsing by unifying structure detection and content material recognition in a single, open-source mannequin. It’s significantly fitted to situations requiring sturdy, language-agnostic doc evaluation and structured data extraction in resource-constrained or manufacturing environments.
Try the GitHub Web page. Be at liberty to take a look at our GitHub Web page for Tutorials, Codes and Notebooks. Additionally, be happy to observe us on Twitter and don’t neglect to affix our 100k+ ML SubReddit and Subscribe to our E-newsletter.
Michal Sutter is an information science skilled with a Grasp of Science in Information Science from the College of Padova. With a strong basis in statistical evaluation, machine studying, and information engineering, Michal excels at reworking complicated datasets into actionable insights.
