University of Maryland, College Park

A new benchmark turns senior researchers' corrections of scientific AI agents into 248 two-way judgment questions: given only the project record from before the correction, four of five frontier models more often chose the original plan that a human had overruled, topping out at 63.5% against 73.4% for fresh researchers. The cases come from 26 real projects across six fields run over roughly half a year, and each asks a model to choose between the original plan and the human-corrected alternative.

A new benchmark turns senior researchers' corrections of scientific AI agents into 248 two-way judgment questions: given only the project record from before the correction, four of five frontier models more often chose the original plan that a human had overruled, topping out at 63.5% against 73.4% for fresh researchers. The cases come from 26 real projects across six fields run over roughly half a year, and each asks a model to choose between the original plan and the human-corrected alternative.

The work presents a model-agnostic framework that lets frozen LLMs and VLMs keep learning during deployment through three forms of external expertise: a Skill guiding reasoning and tool use, a Knowledge Memory storing reliable facts, and a Multimodal Knowledge Base that keeps visual cases and guides the model to compare each retrieved case with the current image; an update is kept only if it helps on new cases without degrading earlier ones. Across six benchmarks and four base models, the framework improves online medical performance by up to 34.2% over the base model, generalizes to unseen cases, and transfers to other models without further optimization.

The work presents a model-agnostic framework that lets frozen LLMs and VLMs keep learning during deployment through three forms of external expertise: a Skill guiding reasoning and tool use, a Knowledge Memory storing reliable facts, and a Multimodal Knowledge Base that keeps visual cases and guides the model to compare each retrieved case with the current image; an update is kept only if it helps on new cases without degrading earlier ones. Across six benchmarks and four base models, the framework improves online medical performance by up to 34.2% over the base model, generalizes to unseen cases, and transfers to other models without further optimization.

The authors formulate Rhetorical Robustness as a joint requirement (stable judgments under content-preserving rewrites plus discrimination across papers), build RobustReview from 60 anonymized ICLR 2026 submissions under 10 rhetorical conditions for 1,260 full manuscripts, and evaluate 30 reviewer configurations, finding that low rewrite sensitivity often coincides with score collapse across papers ('false robustness') and that human alignment ranks reviewers differently; they then introduce SciCore, a dual-branch reviewer that averages a full-manuscript judgment with a judgment based on an extracted structured science core, achieving a leading joint stability-discrimination profile in the primary GPT-5.5 comparison (ICC 0.775, SPR 0.652, discriminability 0.

The authors formulate Rhetorical Robustness as a joint requirement (stable judgments under content-preserving rewrites plus discrimination across papers), build RobustReview from 60 anonymized ICLR 2026 submissions under 10 rhetorical conditions for 1,260 full manuscripts, and evaluate 30 reviewer configurations, finding that low rewrite sensitivity often coincides with score collapse across papers ('false robustness') and that human alignment ranks reviewers differently; they then introduce SciCore, a dual-branch reviewer that averages a full-manuscript judgment with a judgment based on an extracted structured science core, achieving a leading joint stability-discrimination profile in the primary GPT-5.5 comparison (ICC 0.775, SPR 0.652, discriminability 0.

The work introduces Align Then Reason (ATR), in which a monotonic CTC alignment scorer between candidate-text phonetic units and frame-level lip representations supplies per-unit soft tokens and a calibrated global alignment score to an LLM reasoner that outputs a Yes/No judgment with a short explanation, raising Qwen3.5-9B mean AUC from 0.610 to 0.920 on a seven-language benchmark and improving on three unseen MuAViC languages and two real dubbing downstream tasks.

The work introduces Align Then Reason (ATR), in which a monotonic CTC alignment scorer between candidate-text phonetic units and frame-level lip representations supplies per-unit soft tokens and a calibrated global alignment score to an LLM reasoner that outputs a Yes/No judgment with a short explanation, raising Qwen3.5-9B mean AUC from 0.610 to 0.920 on a seven-language benchmark and improving on three unseen MuAViC languages and two real dubbing downstream tasks.

Using EditLens and Pangram to label Common Crawl web data from 2021 to 2026, the authors find that 27.5% of tokens passing FineWeb quality filtering in June 2026 are AI-generated, rising to 31.1% in August, and by pretraining 800 models from 19.9M to 973M parameters while varying the ratio of added AI to human tokens they fit a new scaling law with separate benefit and harm terms: AI tokens lower loss for data-starved models before saturating and reversing into harm, raise loss almost immediately for Chinchilla-optimal models, and the law predicts held-out sizes with lower error than eleven existing laws, implying that training at August 2026's AI share takes 1.6x the compute.

Using EditLens and Pangram to label Common Crawl web data from 2021 to 2026, the authors find that 27.5% of tokens passing FineWeb quality filtering in June 2026 are AI-generated, rising to 31.1% in August, and by pretraining 800 models from 19.9M to 973M parameters while varying the ratio of added AI to human tokens they fit a new scaling law with separate benefit and harm terms: AI tokens lower loss for data-starved models before saturating and reversing into harm, raise loss almost immediately for Chinchilla-optimal models, and the law predicts held-out sizes with lower error than eleven existing laws, implying that training at August 2026's AI share takes 1.6x the compute.