What is data extraction, and when is it worth automating?

The short version

• Data extraction pulls specific values out of documents, web pages or databases and writes them as named fields that software can use.
• OCR turns an image into text, and web scraping fetches pages, but neither decides which value means what.
• A wrong extraction looks the same as a right one, so errors pass silently unless values are checked.
• Accuracy figures do not travel between document types, so measure on a labelled sample of your own documents.
• An extraction API's list price is only part of the cost, because review, integration and upkeep sit outside it.

What is data extraction?

Data extraction is the process of pulling specific values out of a source and writing them into a structure that software can use. The source might be a PDF, a scanned form, an email, a web page or another database. The output is named fields, such as supplier, invoice date and total, usually as JSON or rows in a table.

The point is that a person or a system downstream can now act on the value without reading the original. A payment run can use the total. A report can sum it. A rule can flag it. Until the value is extracted, it is text on a page, and the only thing that can use it is somebody reading it.

In data engineering, the term also means the first step of a pipeline, where records are copied out of a source system before being transformed and loaded. That version is largely routine when the source has a documented interface. This guide spends most of its time on the harder version, where the source was written for humans.

What are the main methods of data extraction?

There are five main methods: querying a structured source, scraping web pages, template or rules-based extraction, document AI services and language model extraction. The right one depends on how much the layout varies and how much a wrong value costs.

Most real systems combine them. A template handles the few senders that make up most of the volume, and a language model takes the long tail, with the same checks applied to both. Starting with the simplest method that works on your documents gives you something you can test and reason about.

Which extraction method fits which source?

How does data extraction work, step by step?

A working system does seven things in order: it takes in the file, reads it, finds the fields, normalises them, checks them, routes anything doubtful to a person and writes the result with a pointer back to the source. Reading and finding get the attention. Checking and routing decide whether the output can be trusted.

1. Ingest. Take the file from the inbox, upload or source system, and keep the original unchanged.
2. Read. Use the embedded text where a file has it, and OCR or a vision model where it does not.
3. Locate. Match values to a fixed schema that names every field, its type and whether it can be empty.
4. Normalise. Put dates, currencies and units into one format, so that '03/04/2026' has only one meaning.
5. Validate. Check what can be checked: line items sum to the total, the supplier exists, the date is plausible.
6. Route. Send failures to a person, with the source page beside the value.
7. Write. Store each value with a pointer to the document and page it came from.

Is data extraction the same as OCR or web scraping?

No. OCR turns an image of text into text. Web scraping fetches content from web pages. Data extraction decides which value means what. OCR and scraping are often steps inside an extraction system, but neither one tells you that the second number on the page is the total.

The distinction matters when you buy. An OCR product with high character accuracy can still give you the wrong total, because every character was read correctly and the wrong number was picked. Ask what is being measured: characters or fields.

How accurate is AI data extraction?

There is no single figure that travels. Accuracy depends on the documents, the fields and how errors are counted, so a number from another document type tells you little about yours. The only figure worth planning around is one measured on a labelled sample of your own documents.

A published study shows the right way to ask the question. Gartlehner and colleagues compared a team using Claude with human verification against a human-only team, across 6 systematic reviews and 9,341 data elements from 63 studies, measured against a reference standard. It tests a model plus a person, which is how a real system would run. But the documents were full-text research PDFs, not invoices or contracts, so its results cannot simply be carried across to business documents.

To measure your own, take a sample of real documents, including the ugly ones. Have a person record the correct value for every field, then score the system field by field. Report fields separately, because a system that is perfect on dates and unreliable on totals does not have an average problem. It has a totals problem.

We do not have accuracy figures from our own work to publish. Be careful with anyone who quotes one without saying which documents it came from.

Why do data extraction systems fail silently?

Because the wrong output looks the same as the right output. A misread total is still a valid number in the right field, so nothing errors and the pipeline carries on. The failure surfaces later, in a report that does not reconcile or a payment that should not have gone out.

The patterns repeat. A net figure is returned where the gross was wanted. A day and month are swapped. A table row shifts by one when a line wraps. A field missing from the document gets filled with a plausible value, because a language model asked for a value tends to produce one. A supplier redesigns an invoice, and a template that worked yesterday starts reading the wrong box.

The defences are mechanical. Let the schema say a field is empty. Cross-check values against each other and against systems you already hold. Keep the page reference beside each value so a reviewer can check it quickly. Sample accepted documents as well as flagged ones, since a review queue only shows you the errors the system noticed.

What does data extraction cost?

The API is only one part of the cost. The rest is integration, human review of flagged fields, exception handling and upkeep when formats change. List prices are easy to compare. Those other lines are not, and they are where budgets tend to go wrong.

Published prices show how hard a straight comparison is. AWS's worked example for Amazon Textract, for US West (Oregon) and the first one million pages, covers API usage only. Nanonets prices by credits spent across workflow blocks, so its pricing card is not a fixed page allowance. It is also one vendor's price, not a market average. One charges for API usage by the page, and the other depends on how your workflow is built.

For the human side, the U.S. Bureau of Labor Statistics publishes national medians for data entry keyers and for software developers. Use your own rates if you are outside the US. The comparison matters because automation swaps some keying hours for some developer hours, and a developer's hour costs more. The build pays back only if the keying hours saved are large and keep recurring.

Be careful with headline savings. A Parseur survey with QuestionPro put the cost of manual data entry at $28,500 per employee per year. It is self-reported, from 500 US-based professionals in July 2025. Parseur sells data extraction software, and the page states no sampling method and gives no company-size breakdown. Treat it as a prompt to count your own hours, not a figure for a business case.

Should you buy a data extraction tool or build your own?

Buy first if your documents are common types such as invoices, receipts or standard forms. Build when the fields are specific to your business, the documents are unusual, or the result has to feed your own rules and systems. Either way, test on your own documents before signing anything.

A bought tool does not remove the surrounding work. It still needs integration, a review route and someone who owns it when a format changes. The purchase decides who maintains the model step. It does not decide who is responsible for a wrong value reaching your finance system.

Does web data extraction need a different approach?

Yes, because you do not control the source. A web page can change its layout, block automated access or change its terms without telling you. Web extraction therefore needs monitoring for empty or shifted results, someone responsible for repairs and a check that you are allowed to collect the data.

It is also often small-team work, which makes that upkeep harder to staff. In a December 2025 survey of their communities, Apify and The Web Scraping Club reported that 35.8% of respondents were freelancers and 49.1% worked in startups and small or medium-sized businesses. Apify sells scraping tools, the sample is described only as 'hundreds of web scraping professionals', and it reflects who is in those communities, not the whole market. The plain question still applies: who fixes this when the site changes?

Before scraping, check whether the site offers an API or a data export. A documented interface is easier to maintain, and asking costs nothing.

Do you need an AI agent for data extraction?

Usually not. If the fields and the steps are known in advance, a fixed pipeline with one model step is easier to test, cheaper to run and simpler to debug. An agent earns its place only when the next step depends on what the document says.

The case for one is real but narrow. An invoice refers to a purchase order, so the system has to find that order before it can check the total. A research task has to decide which pages to open next. In these cases the path cannot be written down in advance. Even then, keep the extraction itself as a fixed, tested function that the agent calls, so the part you measure stays the same.

Each extra decision a model makes is another place a wrong value can enter, and another thing to evaluate.

When is data extraction not worth automating?

When the volume is low, the job does not repeat, or a person must check every value anyway. A few documents a week rarely repay a build. A one-off migration may be cheaper done by hand. And if every field needs full review, you may have moved the work without removing it.

The checks below take less time than a vendor demo, and they often settle the question before any money is spent.

What should you check before automating data extraction?

• •Count the documents per month and the minutes each one takes today.
• •Collect real samples, including the worst ones, not only the clean examples.
• •Write down what a wrong value costs, field by field.
• •Ask whether the sender can supply the data directly through an API or export.
• •Name the person who reviews exceptions and the person who owns repairs when formats change.