Document parsing: what it does, where it fails and how to test it

The short version

• Document parsing converts PDFs, scans and forms into structured text and data that software can use.
• OCR reads characters, while parsing also recovers reading order, tables and the link between a label and its value.
• Parsing errors are quiet, because a shifted table still produces clean-looking text and raises no error.
• Basic parsing can list at under a cent a page, so checking and fixing the output is often the larger cost.
• The only reliable test of a parser is a field-by-field score on your own worst documents.

What does document parsing do?

Document parsing turns a file designed for human eyes into a structure a program can read. It recovers the text, the order it should be read in, and what each piece is: a heading, a table cell, a footnote or a form field. The usual output is Markdown, JSON or text tied to page coordinates.

The job exists because a document rarely stores what it shows. A PDF stores instructions for drawing characters at positions on a page. It does not reliably store paragraphs, columns or tables. A scan stores only pixels. A person sees a table at a glance. Software sees scattered fragments until something rebuilds the structure.

Take an invoice. The word 'Total' sits near a number, and a subtotal and a tax line sit near it too. Reading the characters gives you all of them. Parsing is what tells you which number belongs to which label.

How is document parsing different from OCR and data extraction?

OCR reads characters from an image. Parsing recovers the structure around those characters. Extraction pulls specific named fields out of that structure for a particular job. They are separate steps, and each one fails in its own way.

OCR is not always needed. A PDF exported from software usually has a text layer already, so the characters are available and the hard part is the layout. A scan or a phone photo has no text layer, so OCR comes first. Many products bundle these steps together, which is why the terms blur.

Which step does what?

What are the main ways to parse a document?

There are three main routes: read the file's own text layer, send pages to a hosted document service, or use an AI model that reads page images. The lines blur, because some of those models are sold as hosted services. A sensible system uses the cheapest route that works for each document type.

The choice changes how errors look. A classic OCR error often looks like an error: a garbled word or a stray symbol. A language model's error can read perfectly well, because the model is predicting likely text. That makes it harder to catch by eye.

The three routes at a glance

• •Text layer extraction: fast, cheap and repeatable, but it only works on files that have a text layer, and tables often come out flattened.
• •Hosted document services, such as Amazon Textract: little to run, but your documents leave your environment, so check how the vendor stores and uses them.
• •AI models that read page images, such as Mistral OCR 4 or olmOCR from the Allen Institute for AI: built for messy layouts, but they can produce fluent text that is not on the page.

Why does document parsing fail quietly?

Document parsing fails quietly because bad output still looks like valid text. No exception is raised when a two-column page is read straight across, or when a table shifts its values one column to the left. The pipeline finishes, and everything downstream treats the result as correct.

The common failures are structural. Multi-column pages get merged line by line. Tables that run across pages lose their header row, so the second page has numbers with no column names. Merged cells push values under the wrong heading. Footnotes land in the middle of a sentence. Stamps, signatures and handwritten notes disappear, even when they are the part that matters.

This is why parsing errors get blamed on the model. A language model given a shifted table will answer confidently from it, and the answer will point to a real document. Someone sees a wrong figure and concludes the model made it up. The model read what it was given.

Why does document parsing matter for AI agents?

Document parsing matters for AI agents because an agent acts on what it reads. In a search tool, a parsing error produces a wrong answer that a person might catch. In an agent, the same error can become a payment, a record update or an email to a customer.

So an agent needs more than text. Keep the page number and position of every extracted value, so a person can check the source in seconds. Validate values against rules the agent cannot talk its way around: line items should sum to the total, dates should fall in a sensible range, a supplier should already exist. Send anything that fails to a person.

Treat parsed text as untrusted input. A document can contain text written to instruct the model, and a parser will pass it along like any other text. Parsed content is data to read, never instructions to follow.

How much does document parsing cost?

Basic parsing can list at under a cent a page, but the per-page fee is rarely the biggest cost. Amazon's Textract pricing page works through plain text detection at $0.0015 per page. Mistral's OCR 4 launch announcement lists $2 per 1,000 pages with the batch discount and $5 per 1,000 pages for Document AI.

These figures are not like for like. They are the vendors' own list prices for different products. As plain arithmetic, 100,000 pages at those three rates comes to $150, $200 and $500. Check the rate for the feature you actually need, because plain text detection is not the same work as reading tables or forms. Prices also change, and the AWS page is undated.

Self-hosting changes the shape of the bill. The olmOCR developers publish a cost estimate for their own tool, but it rests on February 2025 GPU rental rates, is largely scaled up from a 1,288-page test, and covers plain text output only. Treat an estimate like that as a starting point, then add the cost of whoever keeps the system running.

The larger cost is usually people. Someone has to review exceptions, fix errors and maintain the pipeline as document formats change. Set a reviewer's hourly rate beside the prices above and the per-page fee stops being the interesting number. If you have no internal figure, the US Bureau of Labor Statistics publishes national median pay for bookkeeping clerks and for software developers.

How do you test a document parser?

Test a parser on your own worst documents and score it field by field. A demo on a clean sample tells you little, because it uses the easy pages. What you need to know is how the parser behaves on the faxed, rotated and multi-page material your business really receives.

Field-level scoring matters because a document-level pass hides the cost of errors. A document can be almost entirely right and still carry a wrong total. A missed address line and a wrong payment amount are not the same failure, so weight them by what each one would cost you.

A simple parser test

1. Pull a few dozen real documents, and include the ugly ones: poor scans, phone photos, rotated pages and tables that run across pages.
2. Write down the correct values by hand for the fields that matter. This is your answer key.
3. Run each candidate parser and compare every field to the key, rather than reading the output by eye.
4. Weight the errors by consequence, then check whether your validation rules catch them.
5. Keep the set. Run it again whenever a vendor updates a model or your document mix changes.

When is document parsing not worth building?

Document parsing is not worth building when the data already exists somewhere in structured form. If a supplier portal has an export, or a partner can send a feed, take that instead. Parsing a PDF to recover data that started in a database is a last resort.

It is also a poor fit when volume is low enough for a person to handle comfortably, or when nobody will own the exceptions. Every parsing system meets documents it cannot handle. If no one is assigned to those, they pile up and people stop trusting the output.

Simple cases may not need an AI model at all. If every document comes from one system in one template, a small rules-based script is cheaper and easier to debug. Use the heavier tools when the layouts vary and that variety is the real problem.