Reading a page instead of its characters

Traditional OCR does one job well. It looks at a region of an image, decides which character is there, and moves on. Decades of engineering have made that extremely good.

The trouble is that most documents worth extracting are not sequences of characters. They are forms, where the meaning of a value comes from the label beside it. They are tables, where a number means nothing without its row and its column. They are invoices, contracts and reports, where layout carries as much information as the text does.

OCR reads all of that and hands back a string with the structure removed. Recovering it afterwards is a second engineering problem, usually solved with positional heuristics that work until somebody changes the template.

What a VLM does differently

It reads the page.

The layout is available to it in the same way it is available to a person. It can tell that a value sits inside a cell, that a cell belongs to a column, and that the column has a heading three rows up. It can handle a form where somebody wrote outside the box. It can read handwriting, which conventional OCR has always been weakest at, because it is not matching character shapes against a reference set.

Ask for the total from an invoice and it finds the total, rather than returning every number on the page and leaving you to work out which one that was.

What it costs

Throughput, mostly. A vision-language model is doing considerably more work per page than a character recogniser, and the gap shows up immediately at volume.

On one build we measured 83 tokens per second on a local deployment, handling layout, tables and handwriting together. That number is specific to that model, that configuration and that hardware, and it is the sort of thing worth measuring rather than assuming. It is the difference between quoting a document pipeline honestly and discovering the throughput after committing to a deadline.

It also has failure modes OCR does not. It will occasionally produce a plausible value that is not on the page, which is a worse failure than an unreadable character because it does not announce itself. Anything going into a system of record needs a confidence signal, a validation rule, or a person, depending on what the value is for.

The validation is usually simpler than it sounds. Totals that should equal the sum of their lines. Dates that have to fall inside a known range. Reference numbers with a checkable format. Most documents carry enough internal structure to catch an invented value without anyone reading the page, and the ones that do not are exactly the ones worth routing to a person.

Why run it locally

Two reasons, and they are not the same reason.

The first is that some documents cannot go to a cloud API at all. Contracts under privilege, records with identifiable data, anything covered by an undertaking you have given your own customers. For those, the location of the model is the entire requirement and the rest of this is detail.

The second is volume. Per-page pricing is comfortable at pilot scale and stops being comfortable at production scale. The crossover is a calculation rather than an opinion, and it is worth running before anyone commits either way.

When OCR is still the right answer

Often.

If the documents are clean, typed, and consistently laid out, and all you need is the text, character recognition is faster, cheaper and entirely sufficient. Reaching for a vision-language model there is paying for capability you are not using.

The case for the heavier approach is structure. Forms, tables, handwriting, mixed layouts, documents that vary. If your documents are a single column of printed prose, use the tool built for that.

How to decide

Take fifty documents that represent the messy end of your set rather than the tidy end. Run both approaches. Compare what comes out against what a person would have extracted.

The answer is usually obvious within an afternoon, and it is far cheaper than discovering it after the pipeline is built.