Skip to main content
Search

Changdae Oh

2 related articles

Subscribe to updates
  1. arXiv

    Pre-trained models with a light inference harness cover more solutions on agentic tasks, while post-training imposes a measurable sharpening tax

    The work finds that pre-trained LLMs with a light inference harness, despite far lower single-shot accuracy (pass@1), often surpass their post-trained counterparts in solution coverage (pass@K) given sufficient test-time budget; it proposes the Sharpening Tax metric to quantify post-training's loss in test-time scalability, shows across 14 base/post-trained model pairs from four families and three agentic benchmarks (42 cases) that the tax is prevalent, estimable from a few rollouts, and correlates with other metrics, and presents PTGS, a difficulty-adaptive Bayesian sampler that reduces the tax.

    Stanford UniversityNew York UniversityUniversity of Wisconsin–Madison
    Changdae Oh（First author）
    Sharpening TaxPosterior-Tempered Group SamplingPost-trainingLarge language modelsagentic benchmarks
  2. arXiv

    Post-training pushes tasks to always-solved or never-solved: a "sharpening tax" appears across 14 base/post-trained pairs on three agentic benchmarks, and PTGS lowers it

    The work compares 14 open base/post-trained checkpoint pairs (Gemma-4, Ministral-3, Qwen2.5, Qwen3.5, 3B to 35B) on three multi-turn tool-calling benchmarks (BFCL v4 multi-turn, WebShop, ACEBench) and finds that harness-equipped base models, despite far lower pass@1, often surpass their post-trained counterparts in pass@K given enough test-time budget; it names this coverage loss the Sharpening Tax, shows post-training bimodalizes per-task success toward always-solved or never-solved, and proposes PTGS, a per-prompt difficulty-adaptive temperature sampler that improves single-shot accuracy and coverage while paying a smaller tax in two agentic environments.

    Stanford UniversityNew York UniversityUniversity of Wisconsin–Madison
    Changdae Oh（First author）
    Sharpening TaxPosterior-Tempered Group SamplingPost-training

You’re all caught up