Skip to main content
Back to timeline
researchsquare.comSource publication:

A 2B model turns table screenshots into queryable databases, hitting 0.86 exact match and 0.99 foreign-key recall on rendered tables

Synopsis

Instead of markup that keeps a table's layout, the output is an executable SQLite database written straight from one document image. PIX2DB fine-tunes Qwen3-VL-2B with LoRA on 36,589 pairs with exact gold databases and adds a deterministic verify-and-repair layer, reaching 0.86 table exact match and 0.99 foreign-key recall on rendered tables from unseen databases, against 0.75/0.75 for the same-family 8B model prompted for SQL and 0.47/0.00 for an 8B HTML-to-SQLite pipeline. On TabRecSet, unseen in training, it is the best system that writes a database directly (0.52) but trails the 8B HTML pipeline (0.58), mostly in reading text from photographs.

AI-generated editorial illustration: PIX2DB: Reconstructing Executable Relational Databases from Document Images with a 2B Vision-Language Model

Interpretation

The target output for a table image becomes a SQLite database that loads and can be queried as it is, with schema, column types, keys and rows inside it. Earlier table-understanding systems recover structure as cell grids, HTML or Markdown, leaving merged cells, two-level headers and the relation between an invoice header and its line items for downstream code to decide. In a pilot on 150 rendered images, the same-family 8B model asked for SQL produced executable output 93% of the time but a fully correct table only 61% of the time; asked for HTML loaded into SQLite by a script, it reached 46%.

A 2B model plus a deterministic verify-and-repair layer outperforms larger models of its family on most evaluation sets. The same-family 8B model prompted for SQL reaches 0.75 table exact match on the synthetic test, the 8B HTML-to-SQLite pipeline 0.47, and the untuned 2B base model 0.53. On 900 synthetic test images from unseen databases, PIX2DB-2b reaches 0.86 table exact match and 0.99 foreign-key recall; on FinTabNet validation it reaches cell F1 0.96 and on CORD test receipts recall 0.93.

Training data yields exact gold databases by two routes: rendering samples of Spider databases at six layout levels, and converting PubTabNet, FinTabNet and CORD annotations under the same normalization rules. Real annotations describe only how a table looks, so conversion expands spans, flattens headers into column names, strips thousand separators and currency signs from numbers, and rejects tables whose body contains colspans. Every gold dump is executed and scored against itself, and any score below 1.0 counts as a data bug; the check found and fixed composite keys declared as primary keys, numbers stored as text, and identifiers that are SQL keywords.

Synthetic data teaches the normalization conventions, while real data teaches what real cell contents look like. A version trained on 24k synthetic images only reaches 0.85 on the synthetic test, comparable to 0.86 with real data added, but only 0.43 cell F1 on PubTabNet, where 82 of 150 outputs did not execute at all. Adding 18.8k converted real tables and receipts lifts PubTabNet cell F1 to 0.81, FinTabNet to 0.96, CORD recall to 0.93 and TabRecSet from 0.44 to 0.52, without costing synthetic accuracy.

Perspective

The demonstrated setting is single-page tables, receipts and charts in English document images, with output as a dump that executes in an unmodified SQLite engine; handwriting, multi-page tables and tables longer than 30 rows are outside the training distribution, and tables whose body contains section rows were excluded from both training and evaluation.

The real held-out sets share sources with training data and therefore overstate performance on unfamiliar documents, which is why TabRecSet is reported separately; TabRecSet lacks header annotations, forcing a first-row-as-header assumption; all numbers come from a single training run, so the intervals reflect sampling of test items rather than training variance; charts without value labels are read only approximately.