A 200,000-page Punjab assembly archive stops stalling on OCR noise: an agentic retrieval loop lifts answer F1 from about 0.18 to about 0.35
Synopsis
India's Punjab Vidhan Sabha records from 1950 to 2024 survive as roughly 200,000 scanned pages mixing Hindi, Punjabi and English, where broken layouts and OCR errors propagate into retrieval and question answering. The system couples layout segmentation and multi-script OCR with an agentic retrieval-augmented generation pipeline, in which an evaluator model judges whether retrieved evidence suffices and rewrites the query when it does not. On a stratified benchmark of 1,000 pages and 200 gold-annotated question-answer pairs, the optimized pipeline reaches MRR@1 of 0.810 and Hit@3 of 0.925; raising the retrieval-rewrite iteration cap from 1 to 3 lifts answer F1 from about 0.18 to about 0.35.
Interpretation
Retrieval over historical legislative archives no longer treats OCR as a one-time preprocessing step, because the retrieval stage can go back and rewrite the query when evidence is insufficient. Earlier pipelines treated layout analysis and OCR as separate front-end steps, and recognition errors then passed straight into retrieval and question answering. On a 1,000-page, 200-query gold-annotated benchmark balanced across decade, script and document type, the optimized pipeline scores MRR@1 0.810, Hit@3 0.925 and Hit@5 0.955, against 0.775, 0.900 and 0.940 for standard BGE-M3.
Before generating an answer, the system judges whether the retrieved context is sufficient and, if not, rewrites the query and searches again. The agentic loop uses an information-gain threshold to test context sufficiency; sufficient context goes straight to generation, insufficient context triggers query rewriting. On a fixed 200-query evaluation set, raising the retrieval-rewrite iteration cap from 1 to 3 moves MRR@1 from about 0.89 to about 0.97, Precision@3 from about 0.33 to about 0.37, and answer F1 from about 0.18 to about 0.35.
Layout-model quality directly sets the quality of the extracted text, and swapping the layout model under the same recognizer changes the outcome sharply. CRAFT layout with the same CRNN recognizer yields 89.00% character recognition rate and 60.35% word recognition rate. On a 20-page hand-transcribed subset spanning degradation levels, fine-tuned YOLO with CRNN reaches 95.67% character and 80.03% word recognition rate; Google OCR and Surya OCR score higher, while this pipeline runs fully locally.
Perspective
The demonstrated scope is cross-lingual question answering over roughly 200,000 pages of Punjab Vidhan Sabha records from 1950 to 2024, evaluated on a stratified benchmark of 1,000 pages and 200 gold-annotated queries, with OCR comparison on a separate 20-page hand-transcribed subset; the authors' stated next steps are improving OCR robustness, reducing inference latency and preserving complex table structures.
Severe OCR errors on degraded pages can still affect dense retrieval and entity matching; the iterative agentic loop and LLM-based sufficiency checks add inference latency over standard retrieval; complex financial tables and unstructured tabular data may lose hierarchical structure during text-to-chunk conversion.