TPDL 2025 · 2025 · Historical AI

A Study of Temporal Fusion Strategies for Named Entity Recognition in Historical Texts

This paper asks how time can be added to named entity recognition models without turning the whole architecture into a monster. It compares lightweight temporal fusion strategies for historical NER, using publication years as explicit signals and testing whether models become more robust across decades, languages, noisy periods, and drifting entity forms.

Publication link →

Research question

Historical language does not stay politely still. Names change, places drift, organizations appear and vanish, spellings move, and old documents arrive with OCR noise already attached.

The paper asks whether publication-year metadata can help Transformer-based NER models deal with this diachronic mess, and which temporal fusion strategy works best: early fusion, late fusion, absolute year encoding, or distance-from-present encoding.

Main result

Late fusion strategies are the most robust. They usually perform better than early fusion and the baseline, especially in early or noisy periods where historical variation and OCR degradation make entity recognition harder.

The gains are modest, not fireworks, but they are consistent enough to be useful. Late cross-attention is the strongest candidate, with the clearest significant improvement over the baseline.

Method / experiment

The model is treated as a standard token classifier, but each document also carries a publication year. That year is embedded and fused with token representations using several strategies.

The study compares a baseline with no temporal fusion, early cross-attention, late adapter fusion, concatenation, relative temporal fusion, and late cross-attention. Each strategy is tested with absolute year embeddings and time-distance embeddings, where the year is represented as distance from 2025.

Experiments use the French and German subsets of a historical NER benchmark, covering long time spans and noisy OCR conditions.

Why it matters

Most NER models behave as if every document lives in the same flat present. That is convenient, and historically ridiculous.

This paper shows that even simple metadata — the year — can be structurally useful when it is injected in the right place. It matters for historical NLP because temporal drift is not a decorative problem; it changes what entities look like, how they are mentioned, and whether they are recognized at all.

Five figures tell the story: temporal fusion over decades, absolute versus distance-based time, entity length, entity type gains, and whether models really encode time internally.

F1 scores over time for French and German historical NER under absolute and time-distance temporal modes. — **Figure 1.** F1 scores across publication years. The useful pattern is not a giant jump, but a stability story: late fusion holds up better across time, especially in earlier and noisier decades.

Average F1 score difference between time-distance and absolute temporal modes for French and German. — **Figure 2.** Absolute year versus time-distance encoding. German benefits more clearly from time-distance for several strategies; French is more mixed, which is annoying but honest.

Difference in F1 score between long and short entities across decades and temporal fusion strategies. — **Figure 3.** Entity length sensitivity. Late fusion tends to behave more steadily for longer entity mentions, especially where older documents make surfaces more fragile.

Distribution of gain over baseline by entity type for French and German. — **Figure 4.** Entity-type gains. Locations show the largest variability and occasional gains, which makes sense: places drift, names change, and historical geography refuses to be tidy.

Probing accuracy grouped by fusion type and fusion strategy. — **Figure 5.** Probing for time. Late-fusion models encode temporal information more strongly than the baseline or early fusion, even when the gold year is not directly available at probing time.

Reuse / links

Useful for historical NER, temporal modelling, metadata-aware NLP, diachronic evaluation, and experiments where publication dates are available but should not be treated as decorative metadata.

Reuse ideas: add year embeddings, compare early and late fusion, test absolute versus relative time, evaluate by decade, and probe whether temporal information is actually internalised.

One-line takeaway

Time helps historical NER most when it is added after the model has understood the sentence — not before the model has even started reading it.