Skip to main content
Search

# Long-context reasoning

4 related articles

Subscribe to updates
  1. arXiv

    Long-context QA stops re-searching every query: a trained builder agent lifts accuracy from 0.42 to 0.75

    Long-document question answering usually starts searching only when a query arrives, so the same corpus is reread for every request. Librarian Agents instead trains a builder agent ahead of queries to reorganize raw documents into a multi-level synthetic filesystem with reference links, which a frozen answering agent then navigates. On BrowseComp-Plus with a Qwen3.5-4B answerer, accuracy reaches 0.75 against 0.42 for embedding retrieval and 0.44 for reranking, while an untrained builder reaches 0.28.

    Stanford UniversityCarnegie Mellon University
    Anikait Singh（First author）
    Librarian AgentsAnticipatory Workspace ConstructionGRPORetrieval-Augmented GenerationLong-context reasoning
  2. Transactions of the Association for Computational Linguistics

    Self-consistency backfires on long context: seven of eight samples agree on the same wrong answer

    On long-context tasks, self-consistency, the practice of sampling several answers and aggregating them, no longer improves performance and often lowers it, because positional bias makes the samples repeat the same mistake and the vote turns that mistake into a consensus.

    Johns Hopkins University
    Adam Byerly（First author）
    Self-ConsistencyPosition BiasLong-context reasoningCorrelated Errors
  3. arXiv

    π² builds reasoning data from Wikipedia tables, lifting gpt-oss-20b and Qwen3-4B long-context reasoning by +6.25% and +3.37% average absolute accuracy

    The work introduces π², a QA curation pipeline that extracts and expands tables from Wikipedia, generates complex reasoning questions from those tables plus relevant metadata with answers automatically determined and validated through dual-path code execution, and back-translates chain-of-thought solutions grounded in realistic context; supervised fine-tuning with gpt-oss-20b and Qwen3-4B-Instruct-2507 on π² yields consistent improvements across four long-context reasoning benchmarks and the alike π²-Bench, with average absolute accuracy gains of +6.25% and +3.37% respectively, and analysis indicates reasoning style contributes little while faithful reasoning patterns from back translation and grounded realistic long context are crucial.

    Quyet V. Do（First author）
    QA curation pipelineLong-context reasoningSupervised Fine-tuningLarge language models
  4. arXiv

    Rewriting public documents into about 10k synthetic samples lifts a 35B student from 13.7% to 24.6% on CL-bench, near a trillion-parameter frontier model

    The work proposes a human-annotation-free synthetic supervision pipeline: it deeply rewrites 3,515 public documents via entity renaming and numeric perturbation, has LLMs generate questions and rubrics that require reasoning over the document, and admits only samples that genuinely depend on the document via a with-document versus without-document rubric gap check, yielding about 9,625 training samples; SFT raises Qwen3.6-35B-A3B on CL-bench from 13.7% to 22.8%, a subsequent rubric-reward RL stage reaches 24.6%, comparable to Qwen3.8-2.4T at 23.9%, with broad transfer to long-context understanding, instruction following, and reasoning, while code generation and knowledge stay mostly flat.

    Haoyi Wu（First author）
    Synthetic Data GenerationLong-context reasoningSupervised Fine-tuningReinforcement Learning

You’re all caught up