What an AI agent can actually do with a PDF

The short version

• PDF is a print format, so an agent either reads a text layer that somebody else's software wrote or looks at a picture of the page.
• An accuracy claim about document extraction means nothing without a gold set, because accuracy varies by field type and by document.
• Field accuracy compounds, so a per-field score in the nineties still leaves most multi-field documents with an error somewhere.
• A PDF is untrusted input, and an agent that can act on a document can be instructed by it.
• If the document type is fixed and the next step never varies, build a pipeline, not an agent.

Does 'ai agent pdf' mean a guide to read or an agent to build?

Two different people search this. One wants a PDF to read, usually a vendor or open-source guide to agent design, and that is a download rather than a decision. The other wants to build something that takes a PDF in and does work on the other side: read the invoice, find the clause, update the record. This page is about the second, because that is where the effort and the money go.

The two have one thing in common. The guides all describe an agent as a model with tools in a loop. A PDF is the most common first tool anybody reaches for, and it is also the one that makes the loop misbehave, because the input is not structured and the model will not tell you when it guessed.

What actually happens when an agent opens a PDF?

One of two things, and the difference decides your whole design. Either the file carries a text layer, written by some other piece of software when the document was created, and the agent reads that. Or it does not, because the page is a scan or a photograph, and the only option is to render each page as an image and have a vision model read it the way a person would.

Neither route gives you fields. A PDF stores glyphs at coordinates. There is no concept of a table row, a column header, or a label attached to a value. Reading order is inferred. A two-column page can interleave. A table is lines and floating text that happen to line up. Every tool that hands you a tidy table from a PDF is guessing, well or badly, and the guesses fail on exactly the documents you care about: the ones with merged cells, footnotes and a layout nobody checked.

There are also hard ceilings on the direct route. Anthropic's PDF support documentation sets a limit of 600 pages per request, dropping to 100 when the context window is under 1M tokens, and 128 MB per request. It also tells you to estimate cost from page count, which is the right instinct: pages, not documents, are the unit you pay for. A 900 page credit agreement is a chunking and retrieval problem before it is an extraction problem.

How accurate is PDF extraction?

Nobody can tell you, for your documents, without measuring them. Anyone quoting a single accuracy figure for 'document extraction' is quoting an average over somebody else's corpus, and your corpus is different in the ways that matter.

You can see how much work an honest answer takes by looking at what a benchmark costs to build. ExtractBench, from Contextual AI, covers 35 real PDF documents, 2,076 pages and 12,867 human-annotated gold fields across five schema domains including SEC filings, credit agreements and research papers, run as 210 extraction attempts against six frontier models. That is the price of being able to say anything precise about extraction quality: a person sitting with the documents, field by field.

The second thing an average hides is compounding. This next part is arithmetic rather than a measurement, but it is the arithmetic that decides whether you can run unattended. If a model gets 95% of fields right and the errors are independent, a 40 field form comes out completely clean about one time in eight. Errors are not really independent, and one bad scan tends to ruin several fields at once, but the direction holds: per-field accuracy that sounds excellent can still mean most documents need a human to look.

So measure per field, and measure by field type. A date in a labelled box is a different problem from a governing law clause 180 pages into an agreement. Report them separately, or you will average a solved problem together with an unsolved one and learn nothing from the result.

What does it cost to run PDFs through a model?

Count pages rather than documents, then choose between paying per token and paying per GPU hour.

The olmOCR paper from Ai2 is a useful reference point for the shape of that decision. Ai2 measured its own 7B vision-language model on a single NVIDIA L40S at $0.79 per hour with a 12% retry rate, and put GPT-4o batch at over $6,240 per million pages at February 2025 list prices. Treat the absolute numbers as history, because model pricing has moved repeatedly since. The structure is what lasts: a hosted frontier model bills per token and scales linearly with pages, while a self-hosted small model bills per GPU hour and gets cheaper the more you feed it.

Retries belong in the budget. A measured 12% retry rate means roughly one page in eight goes round twice, which also means your throughput estimate is wrong by about that much if you plan as though every page passes first time.

The cost that actually surprises people is not inference. It is the review queue. If a fifth of documents need a human to confirm a field, you have hired for that, and the rate at which the model abstains sensibly matters more to the business case than the price per page.

Can a PDF attack an agent?

Yes, and this is the failure mode specific to agents rather than to extraction. A PDF arrives from outside. An agent reads it and then acts with tools. If the document contains text aimed at the model rather than at the reader, a line in white, a footer, a filename, the agent may follow it. This is indirect prompt injection, and a document is a perfectly good delivery vehicle for it.

AgentDojo, from ETH Zurich and Invariant Labs, built 97 realistic agent tasks, 74 tools and 629 security test cases to measure this. Its Slack suite reported a 92% attack success rate, falling to 8% once a secondary attack detector was in place. Those figures are for the frontier models of late 2024, so read them as evidence that the class of attack works rather than as today's risk level.

The design response does not depend on the numbers. Treat everything extracted from a document as data, never as instructions. Keep the list of tools the agent may call fixed in code, so a document cannot widen it. Put writes, payments and outbound email behind an explicit allowlist, with a human confirmation for anything irreversible. An agent that can only read and queue cannot be talked into doing damage.

Which approach fits which document?

When is an agent the wrong tool for a PDF?

When the document type is fixed and the next step never varies. What you want then is a pipeline: parse, extract into a schema, validate every field, write the result, raise an exception when validation fails. It is deterministic, cheap to test, and when it breaks you can point at the step that broke. Wrapping that in an agent loop buys you nothing and costs you reproducibility.

An agent earns its place when the path cannot be fixed in advance. The inbox holds six document types and you do not know which arrived. A missing figure has to be looked up elsewhere before the record can be written. The work needs three tools in an order that depends on what the first one found. That is control flow a model is genuinely better at choosing than a flowchart is.

Said plainly: a lot of what gets called a 'PDF agent' is a pipeline with one model call in the middle, and it should be built as one. That is not a lesser project. It is the version that still works in a year, and it is also the thing an agent would call if you added one later.

How do you know it works before letting it act unattended?

Build the gold set before you build the agent. Take 50 to 200 real documents, including the bad scans and the odd layouts everybody wants to exclude, and have somebody who knows the domain write down the correct value for every field. That artefact is what turns 'it looks good' into a number you can defend, and it outlives every model you swap underneath it.

Then decide what the system does when it is unsure. A model will produce a fluent, plausible value for a field that is not in the document at all, so 'not found' has to be a valid and tested output rather than an absence. Abstention is a feature. An agent that sends 15% of documents to a person and is right about the rest is far more useful than one that answers everything and is quietly wrong on a twentieth of it.

If scepticism inside your own team feels like the obstacle, it is not an unusual one. Stack Overflow's 2025 developer survey asked developers specifically about how far they trust AI output, and 33,244 of them answered that question. However the responses split, a question that size sits on the survey because the subject is contested among the people who would have to maintain this. The way through it is the gold set and the error analysis, not another demo.

Is a document agent realistic for a small team?

Yes, and the constraint is not model access. Adoption still skews to large organisations: the U.S. Census Bureau's Business Trends and Outlook Survey, covering data collected between December 2025 and May 2026, found 37% of firms with at least 250 employees reported using AI in the previous two weeks. Smaller firms report less, and it is not because the API is harder for them to call.

What large firms have is people to annotate documents and a team to absorb a review queue. That is the real barrier. It is also why a small team should pick one document type, one schema and one downstream action, and finish it. One narrow document agent that runs every day beats a general one nobody trusts enough to leave alone.