Essay

Documents are evidence, not just data

Essay · October 2026 · about 8 minutes

A receipt, an insurance claim, a news report about an outbreak, a revision of a Wikipedia article. We feed them to models as text, rows or pixels. But each of them exists to prove something to someone, and an AI system that forgets where a claim came from has kept the data and thrown away the evidence.

A document is a claim with a witness

A receipt says: this shop, on this day, sold these things at these prices, for this total. Nobody reads a receipt for pleasure. It is there to be checked, by an accountant, an employer, an insurer, a tax office.

That is also why it gets forged. With colleagues in La Rochelle we built one of the few public datasets for it: 988 scanned receipts with their transcriptions, of which 163 were altered by hand, in realistic ways, and annotated (Tornés et al., 2023a). The forgeries are small. A total, a quantity, a date, a product line. Nothing in the image needs to look wrong.

Read as a bag of words, a forged receipt is almost identical to the real one. Read as a set of claims that have to agree with each other, it is not. In a second paper we gave a language model the receipt three ways: as raw text, as extracted entities, and as triples built from a small ontology of shops, products, prices and totals. How well forgeries were caught depended on which entities and relations the model could see (Tornés et al., 2023b). The evidence is not in any single field. It is in whether the fields still hold together.

Car insurance claims make the same point from the other direction. Most of what an insurer holds is a table: dates, amounts, vehicle, policy. To use a language model at all, we had to serialise those tables into text, and the way we flattened them changed what the model could learn (Telnoff et al., 2023). Turning a record into a sentence is already an interpretation.

A page is not a photo

Even the explanation of a decision has to respect where the evidence sits. LIME, a popular explanation method, hides regions of an image and watches how the classifier's answer changes. The regions usually come from superpixel algorithms designed for photographs, where pixels of similar colour belong together. On a document page, that logic cuts words in half and joins lines that have nothing to do with each other.

The standard benchmark for this, RVL-CDIP, has its own history. Its pages come from the Legacy Tobacco Documents Library, industry papers made public through the 1998 tobacco settlement: documents that were, quite literally, evidence before they became training data.

So we measured it, varying the segmentation, the sampling and the surrogate model one at a time. The segmentation mattered most. On the same invoice, the agreement between repeated explanations went from 0.43 with Quickshift to 0.93 with regions built from OCR text boxes and 0.98 with a plain grid (Telnoff et al., 2026). The segmentation is not a preprocessing detail. It decides what an explanation is allowed to talk about. If it cannot point to “this line, this total, this signature”, it is not pointing to evidence.

Who said it, where, and when

Epidemic surveillance runs on documents too. Long before official figures, outbreaks show up in local news, in many languages, often in places with little annotated data. Several of our papers worked on finding those reports and pulling out the disease and the location they mention, across languages and at the level of individual words (Mutuvi et al., 2020; 2021).

An alert that says “cholera, somewhere, recently” is not useful. What an analyst needs is the sentence, the newspaper, the date and the place, so that the signal can be checked against other sources before anyone acts on it. Even the way a model splits words into tokens changes what it can find in low-resource languages (Mutuvi et al., 2023): the same report can produce an alert or nothing, depending on choices that nobody downstream will ever see.

Edits are evidence too

Some documents are never finished. A Wikipedia article is the sum of every revision ever made to it, and that history is part of what it says.

With Hamest Tamrazyan and M. Garambois we followed Ukrainian cultural heritage on English Wikipedia: from 25 seed concepts such as the Kyiv Pechersk Lavra, the Holodomor and Crimea, to 40 articles and their complete histories, 69,908 revisions by 19,769 editors. A language model restated and labelled every edit, and two experts checked thousands of them by hand. About one revision in five reshaped who is seen to have made, inherited or lost a heritage (Garambois et al., 2026; Tamrazyan et al., to appear).

Almost none of these edits are false. A word changes, a clause adds blame, a properly sourced quotation disappears while the sentence it supported stays. Fact-checking cannot catch an edit in which nothing is false, and the metadata the platform keeps, quality grades and page protection, did not separate these edits from the rest. The harm is only visible if you read the text as a sequence of claims, each with a source and an author, at a date.

Data came from somewhere too

The same logic applies to the documents we train models on. In the Apertus project, a fully open multilingual model built in Switzerland, part of the work was simply respecting where the data came from: openly available text, with website owners' opt-outs honoured, and a pipeline that can be inspected (Hernández-Cano et al., 2026). A training corpus is also a collection of documents, each with an origin, an owner and conditions. Treating it as an anonymous pile of tokens is a choice, not a fact.

Keeping the trace

None of this needs exotic methods. It needs systems that refuse to drop the link between a claim and where it came from:

• Point back to the source. Every extracted value should carry its page, region, sentence or revision, so a person can check it in one click.
• Model relations, not only fields. A total that does not match its items is evidence; neither number alone is.
• Explain in the document's own units. Lines, cells, blocks and signatures, not colour patches.
• Keep time and authorship. When something was written, edited or reported is part of what it means.
• Treat flattening as interpretation. Turning a table or a page into text decides what a model can see; record how it was done.
• Evaluate verifiability. Ask not only whether the answer is right, but whether someone could tell if it were wrong.

Documents were made to be checked. The least our systems can do is leave them checkable.