Turning unstructured documents into structured, reliable data: methods and limits

Jan 27

Markets run on structured information. Relationships open doors and trust enables transactions, but beneath them, commerce depends on information in forms people can act on: a price, a contract, an invoice, a shipping manifest, a quality certificate.

Friedrich Hayek made the point in 1945. The price system, he argued, works because it compresses knowledge scattered across millions of people into a signal anyone can use. George Akerlof showed the reverse in his study of the market for lemons: when buyers cannot verify what they are buying, markets shrink or fail. Structured, trustworthy information is what lets parties who never meet do business with each other.

Inside organisations, most of that information does not start out structured. It arrives as emails, PDFs, scanned forms, spreadsheets with free-text columns, photos of nameplates, handwritten site notes and voice messages. Industry analysts commonly estimate that the large majority of enterprise data is unstructured. Before it can drive a quote, a filing or a decision, someone has to turn it into fields, values and records. Usually, that person is a skilled data-entry specialist.

What changed

Until recently, automating this work meant rigid templates. Optical character recognition could read printed text, and rules could pull values from fixed positions on standard forms. Anything that varied, which is most real-world input, went to a person.

Modern AI changes the economics. Multimodal models can read a varied invoice, a handwritten note or a photograph of a label, understand what each element means in context, and map it to a defined structure. Voice notes in several languages can be transcribed and turned into the same fields. The range of inputs that can be structured reliably has widened dramatically.

But "can read" is not the same as "can be trusted". The value lies in the method around the model.

A reliable method has five steps

Define the target structure. Before extracting anything, decide exactly which fields you need, in what format, and which values are allowed. A clear schema underpins everything that follows.
Extract, with context. The system reads the input and fills the schema, using what it knows about the domain: product codes, supplier names, units, typical ranges.
Validate against rules. Check every extracted value: does the total equal the sum of the lines, is the certificate date before the expiry date, does the part number exist in the catalogue, is the unit plausible?
Route by confidence. Values that pass every check and carry high confidence go straight through. Anything uncertain, inconsistent or missing goes to a person, with the source highlighted so the check takes seconds, not minutes.
Learn from corrections. Each correction a person makes is captured and added to the knowledge the system draws on, so the same mistake becomes less likely next time.

Where extraction fails

Honesty about limits is what makes the method trustworthy. Common failure points include:

Poor-quality inputs: low-resolution scans, glare on photographs, faint handwriting.
Complex tables: merged cells, tables split across pages, nested line items.
Ambiguity: a figure that could be a weight or a price, a date in a format that could be read two ways.
Missing information: the system cannot extract what is not there, and must say so rather than guess.
Plausible errors: the most dangerous failure is a value that looks right but is wrong. Unconstrained generative models can fill gaps with confident inventions.

Each of these is manageable with the right design: validation rules catch inconsistencies, confidence routing sends doubtful cases to people, and the system is told explicitly to leave a field empty rather than invent a value.

How to measure extraction quality

Overall accuracy figures hide what matters. Better measures work field by field:

Precision: of the values the system extracted, how many were right?
Recall: of the values present in the document, how many did it find?
Straight-through rate: what share of documents needed no human touch?
Review time: how long it takes a person to check a flagged case.

Wang and Strong's influential framework for data quality adds a useful reminder: quality is not only accuracy. Data must also be relevant to the task, complete, timely, consistently represented and accessible to those who need it. A perfectly extracted value that arrives too late, or in a format the next system cannot read, has failed too.

Structure creates capability

Once information is structured and trusted, new possibilities emerge. A manufacturer can quote from a customer's drawing on the day it arrives. A sustainability team can see the state of supplier evidence at any moment, not only at audit time. A financial services firm can answer a client from verified documents. An agricultural buyer can trace a consignment to the fields it came from.

The capability is not the extraction itself. It is what the organisation can do once its information is in a form people and systems can act on.

Questions to ask before automating document work

Which document types consume the most skilled time today, and how varied are they?
What exact structure does the next step need?
Which validation rules can automatically check extracted values?
At what confidence level should a case be sent to a person, and who reviews it?
How will precision, recall and review time be measured from the start?

How Mitochondria's agentic AI systems structure information

The approach. Each unit is configured with the target structure its workflow needs, the validation rules that check it and the confidence levels that send a case to a person. Systems read documents, photos, spreadsheets and voice notes as they arrive, in the languages people use.

What it enables. Quotes from drawings and photos, compliance evidence from certificates and registers, disclosures from utility bills and site records, and client answers from verified documents. Every extracted value is traceable to its source, and every document processed successfully adds to the knowledge the systems draw on.

The impact. Skilled people stop retyping information and start using it. Data arrives structured, validated and on time, and the organisation can act on what it knows.