Skip to main content

Arijit Sehanobish

2 related articles

Subscribe to updates

arXiv Oct 12, 2026
MLP linearization moves to low-degree rational fits: Qwen3-8B drops 14% of parameters and decodes faster
Instead of relying only on polynomial random features, MLP linearization now approximates activations with low-degree rational functions and splits each layer, via partial fractions, into separable weight-side and input-side inner products. Across synthetic fitting, vision, 3D and language models this route gives lower error at the same feature budget; compressing Qwen3-8B by replacing its last eight feed-forward layers removes 1.17B parameters (14% of the model) and raises decoding throughput from 763.3 to 809.6 tokens/s, with reasoning and knowledge benchmarks roughly at parity with the uncompressed model.
Instead of relying only on polynomial random features, MLP linearization now approximates activations with low-degree rational functions and splits each layer, via partial fractions, into separable weight-side and input-side inner products. Across synthetic fitting, vision, 3D and language models this route gives lower error at the same feature budget; compressing Qwen3-8B by replacing its last eight feed-forward layers removes 1.17B parameters (14% of the model) and raises decoding throughput from 763.3 to 809.6 tokens/s, with reasoning and knowledge benchmarks roughly at parity with the uncompressed model.
Columbia University Seoul National University
Arijit Sehanobish（First author）
Rationally Linearized Multilayer Perceptrons (RaLiMPs)MLP Linearization Random Features Large language models On-Policy Distillation
arXiv Oct 5, 2026
Opaque integer keys push general Text-to-SQL below 50% on financial data; FLINT adds a lookup agent to close the gap
FLINT, a Text-to-SQL system for production financial databases, grounds natural-language questions in question-specific table constraints instead of relying on column-name similarity. General systems do well on benchmarks like Spider and BIRD, where schemas are shallow and column values are readable, but financial databases store concepts as opaque integer keys, so even simple queries need multiple joins and those methods fall below 50%. FLINT resolves concepts with a lookup agent, retrieves structurally similar query templates, and prunes large schemas along foreign-key chains; on two datasets totaling 359 questions over production financial schemas it outperforms baselines using the same LLM, and it is deployed in a financial data retrieval service.
FLINT, a Text-to-SQL system for production financial databases, grounds natural-language questions in question-specific table constraints instead of relying on column-name similarity. General systems do well on benchmarks like Spider and BIRD, where schemas are shallow and column values are readable, but financial databases store concepts as opaque integer keys, so even simple queries need multiple joins and those methods fall below 50%. FLINT resolves concepts with a lookup agent, retrieves structurally similar query templates, and prunes large schemas along foreign-key chains; on two datasets totaling 359 questions over production financial schemas it outperforms baselines using the same LLM, and it is deployed in a financial data retrieval service.
Arijit Sehanobish（First author）
FLINT Text-to-SQL Schema linking Financial data retrieval

You’re all caught up