arXiv ROSS introduces a selective-supervision relearning procedure that takes historical self-generated rollouts saved from domain-specific RL, multi-teacher on-policy distillation (MOPD), or agentic RL, uses an outcome verifier to keep successful trajectories and an LLM reviewer to mark which model-generated spans are worth imitating, then applies loss only to those selected tokens while keeping the full trajectory as context, improving upstream checkpoints through an offline SFT stage without new policy rollouts and raising Qwen3.6-35B-A3B's six-benchmark MOPD average from 58.40% to 62.20% and SWE-bench Verified from 64.20% to 68.40%.
ROSS introduces a selective-supervision relearning procedure that takes historical self-generated rollouts saved from domain-specific RL, multi-teacher on-policy distillation (MOPD), or agentic RL, uses an outcome verifier to keep successful trajectories and an LLM reviewer to mark which model-generated spans are worth imitating, then applies loss only to those selected tokens while keeping the full trajectory as context, improving upstream checkpoints through an offline SFT stage without new policy rollouts and raising Qwen3.6-35B-A3B's six-benchmark MOPD average from 58.40% to 62.20% and SWE-bench Verified from 64.20% to 68.40%.
ROSS introduces a selective-supervision relearning procedure that takes historical self-generated rollouts saved from domain-specific RL, multi-teacher on-policy distillation (MOPD), or agentic RL, uses an outcome verifier to keep successful trajectories and an LLM reviewer to mark which model-generated spans are worth imitating, then applies loss only to those selected tokens while keeping the full trajectory as context, improving upstream checkpoints through an offline SFT stage without new policy rollouts and raising Qwen3.6-35B-A3B's six-benchmark MOPD average from 58.40% to 62.20% and SWE-bench Verified from 64.20% to 68.40%.
ROSS introduces a selective-supervision relearning procedure that takes historical self-generated rollouts saved from domain-specific RL, multi-teacher on-policy distillation (MOPD), or agentic RL, uses an outcome verifier to keep successful trajectories and an LLM reviewer to mark which model-generated spans are worth imitating, then applies loss only to those selected tokens while keeping the full trajectory as context, improving upstream checkpoints through an offline SFT stage without new policy rollouts and raising Qwen3.6-35B-A3B's six-benchmark MOPD average from 58.40% to 62.20% and SWE-bench Verified from 64.20% to 68.40%.
arXiv TabFM-Auto pairs the frozen tabular foundation model TabFM with a language-model coding agent that iteratively rewrites four pipeline stages—data cleaning, feature engineering, context selection, and post-processing—guided by dataset metadata and validation feedback; across all 51 TabArena datasets five configurations take the top five overall positions, the best (Codex with Opus 5) raises TabFM from 1785.3 to 2013.0 Elo, the discovered pipelines transfer without further search to TabPFN-3, TabICLv2, and EXAONE-Tabular (+69 to +143 Elo), and TabFM-Auto ranks first overall among MLE agents on the 8 tabular competitions of MLE-Bench.
TabFM-Auto pairs the frozen tabular foundation model TabFM with a language-model coding agent that iteratively rewrites four pipeline stages—data cleaning, feature engineering, context selection, and post-processing—guided by dataset metadata and validation feedback; across all 51 TabArena datasets five configurations take the top five overall positions, the best (Codex with Opus 5) raises TabFM from 1785.3 to 2013.0 Elo, the discovered pipelines transfer without further search to TabPFN-3, TabICLv2, and EXAONE-Tabular (+69 to +143 Elo), and TabFM-Auto ranks first overall among MLE agents on the 8 tabular competitions of MLE-Bench.
TabFM-Auto pairs the frozen tabular foundation model TabFM with a language-model coding agent that iteratively rewrites four pipeline stages—data cleaning, feature engineering, context selection, and post-processing—guided by dataset metadata and validation feedback; across all 51 TabArena datasets five configurations take the top five overall positions, the best (Codex with Opus 5) raises TabFM from 1785.3 to 2013.0 Elo, the discovered pipelines transfer without further search to TabPFN-3, TabICLv2, and EXAONE-Tabular (+69 to +143 Elo), and TabFM-Auto ranks first overall among MLE agents on the 8 tabular competitions of MLE-Bench.
TabFM-Auto pairs the frozen tabular foundation model TabFM with a language-model coding agent that iteratively rewrites four pipeline stages—data cleaning, feature engineering, context selection, and post-processing—guided by dataset metadata and validation feedback; across all 51 TabArena datasets five configurations take the top five overall positions, the best (Codex with Opus 5) raises TabFM from 1785.3 to 2013.0 Elo, the discovered pipelines transfer without further search to TabPFN-3, TabICLv2, and EXAONE-Tabular (+69 to +143 Elo), and TabFM-Auto ranks first overall among MLE agents on the 8 tabular competitions of MLE-Bench.
arXiv The work shows that the zero-variance sequential importance sampling (SIS) proposal for binary matrices with fixed margins is exactly the policy of a unit-reward GFlowNet, and proposes MarginFlow, a set transformer that reads the remaining margins and, trained on 1,904 margins, runs zero-shot on 1,190 held-out margins, matching or beating the post-hoc best of 31 analytically designed configurations on 1,187 of them with a median effective sample fraction of 99.8%.
The work shows that the zero-variance sequential importance sampling (SIS) proposal for binary matrices with fixed margins is exactly the policy of a unit-reward GFlowNet, and proposes MarginFlow, a set transformer that reads the remaining margins and, trained on 1,904 margins, runs zero-shot on 1,190 held-out margins, matching or beating the post-hoc best of 31 analytically designed configurations on 1,187 of them with a median effective sample fraction of 99.8%.
The work shows that the zero-variance sequential importance sampling (SIS) proposal for binary matrices with fixed margins is exactly the policy of a unit-reward GFlowNet, and proposes MarginFlow, a set transformer that reads the remaining margins and, trained on 1,904 margins, runs zero-shot on 1,190 held-out margins, matching or beating the post-hoc best of 31 analytically designed configurations on 1,187 of them with a median effective sample fraction of 99.8%.
The work shows that the zero-variance sequential importance sampling (SIS) proposal for binary matrices with fixed margins is exactly the policy of a unit-reward GFlowNet, and proposes MarginFlow, a set transformer that reads the remaining margins and, trained on 1,904 margins, runs zero-shot on 1,190 held-out margins, matching or beating the post-hoc best of 31 analytically designed configurations on 1,187 of them with a median effective sample fraction of 99.8%.
arXiv The work introduces Hindsight-Divergence Localization (HDL), which picks branch points in a trajectory by how much token log-likelihoods change once verifier feedback and a reflection are added, then builds each training group from a few complete root trajectories plus continuations that reuse the root prefix; across math, code, and agent tasks with three models it cuts generated tokens by 35–61% and rollout wall-clock time by 18–45% relative to GRPO while improving task performance, with gains up to 12.46 percentage points on agent tasks.
The work introduces Hindsight-Divergence Localization (HDL), which picks branch points in a trajectory by how much token log-likelihoods change once verifier feedback and a reflection are added, then builds each training group from a few complete root trajectories plus continuations that reuse the root prefix; across math, code, and agent tasks with three models it cuts generated tokens by 35–61% and rollout wall-clock time by 18–45% relative to GRPO while improving task performance, with gains up to 12.46 percentage points on agent tasks.
The work introduces Hindsight-Divergence Localization (HDL), which picks branch points in a trajectory by how much token log-likelihoods change once verifier feedback and a reflection are added, then builds each training group from a few complete root trajectories plus continuations that reuse the root prefix; across math, code, and agent tasks with three models it cuts generated tokens by 35–61% and rollout wall-clock time by 18–45% relative to GRPO while improving task performance, with gains up to 12.46 percentage points on agent tasks.
The work introduces Hindsight-Divergence Localization (HDL), which picks branch points in a trajectory by how much token log-likelihoods change once verifier feedback and a reflection are added, then builds each training group from a few complete root trajectories plus continuations that reuse the root prefix; across math, code, and agent tasks with three models it cuts generated tokens by 35–61% and rollout wall-clock time by 18–45% relative to GRPO while improving task performance, with gains up to 12.46 percentage points on agent tasks.
arXiv The study introduces Hyperspherical Semantic Trajectory Analysis (HSTA), an unsupervised pipeline that encodes 20,000 arXiv preprints and 10,000 USPTO patent abstracts with Sentence-BERT, projects them onto a unit hypersphere, clusters them into eight sub-topics with Spherical K-Means alongside UMAP reduction, and defines two metrics, Semantic Centroid Vector Drift and Commercialization Offset, then links them to Epoch AI compute data through Vector Autoregressive Granger tests, finding that Large Language Models (drift 0.332) and Artificial Intelligence Systems (0.234) evolve fastest semantically while quarterly paper volume velocity alone does not Granger-cause frontier training compute surges at conventional significance levels.
The study introduces Hyperspherical Semantic Trajectory Analysis (HSTA), an unsupervised pipeline that encodes 20,000 arXiv preprints and 10,000 USPTO patent abstracts with Sentence-BERT, projects them onto a unit hypersphere, clusters them into eight sub-topics with Spherical K-Means alongside UMAP reduction, and defines two metrics, Semantic Centroid Vector Drift and Commercialization Offset, then links them to Epoch AI compute data through Vector Autoregressive Granger tests, finding that Large Language Models (drift 0.332) and Artificial Intelligence Systems (0.234) evolve fastest semantically while quarterly paper volume velocity alone does not Granger-cause frontier training compute surges at conventional significance levels.
The study introduces Hyperspherical Semantic Trajectory Analysis (HSTA), an unsupervised pipeline that encodes 20,000 arXiv preprints and 10,000 USPTO patent abstracts with Sentence-BERT, projects them onto a unit hypersphere, clusters them into eight sub-topics with Spherical K-Means alongside UMAP reduction, and defines two metrics, Semantic Centroid Vector Drift and Commercialization Offset, then links them to Epoch AI compute data through Vector Autoregressive Granger tests, finding that Large Language Models (drift 0.332) and Artificial Intelligence Systems (0.234) evolve fastest semantically while quarterly paper volume velocity alone does not Granger-cause frontier training compute surges at conventional significance levels.
The study introduces Hyperspherical Semantic Trajectory Analysis (HSTA), an unsupervised pipeline that encodes 20,000 arXiv preprints and 10,000 USPTO patent abstracts with Sentence-BERT, projects them onto a unit hypersphere, clusters them into eight sub-topics with Spherical K-Means alongside UMAP reduction, and defines two metrics, Semantic Centroid Vector Drift and Commercialization Offset, then links them to Epoch AI compute data through Vector Autoregressive Granger tests, finding that Large Language Models (drift 0.332) and Artificial Intelligence Systems (0.234) evolve fastest semantically while quarterly paper volume velocity alone does not Granger-cause frontier training compute surges at conventional significance levels.
arXiv The work introduces the "Think Before You Score" paradigm and the Thinking Reward Model (TRM), which first generates a case-adaptive rubric for each task condition and candidate output, then inspects the candidate criterion by criterion and aggregates the evidence into a fine-grained pointwise reward; it also proposes PD-GRPO to use pairwise preference supervision for better discrimination while mitigating score polarization, and reports state-of-the-art results among open-source reward models on image generation and editing reward-modeling benchmarks, highly competitive with proprietary alternatives, with TRM-guided reinforcement learning consistently improving diverse visual generation models.
The work introduces the "Think Before You Score" paradigm and the Thinking Reward Model (TRM), which first generates a case-adaptive rubric for each task condition and candidate output, then inspects the candidate criterion by criterion and aggregates the evidence into a fine-grained pointwise reward; it also proposes PD-GRPO to use pairwise preference supervision for better discrimination while mitigating score polarization, and reports state-of-the-art results among open-source reward models on image generation and editing reward-modeling benchmarks, highly competitive with proprietary alternatives, with TRM-guided reinforcement learning consistently improving diverse visual generation models.
The work introduces the "Think Before You Score" paradigm and the Thinking Reward Model (TRM), which first generates a case-adaptive rubric for each task condition and candidate output, then inspects the candidate criterion by criterion and aggregates the evidence into a fine-grained pointwise reward; it also proposes PD-GRPO to use pairwise preference supervision for better discrimination while mitigating score polarization, and reports state-of-the-art results among open-source reward models on image generation and editing reward-modeling benchmarks, highly competitive with proprietary alternatives, with TRM-guided reinforcement learning consistently improving diverse visual generation models.
The work introduces the "Think Before You Score" paradigm and the Thinking Reward Model (TRM), which first generates a case-adaptive rubric for each task condition and candidate output, then inspects the candidate criterion by criterion and aggregates the evidence into a fine-grained pointwise reward; it also proposes PD-GRPO to use pairwise preference supervision for better discrimination while mitigating score polarization, and reports state-of-the-art results among open-source reward models on image generation and editing reward-modeling benchmarks, highly competitive with proprietary alternatives, with TRM-guided reinforcement learning consistently improving diverse visual generation models.
arXiv The work proposes PMOPD, which builds per-matrix low-dimensional subspace memories from the cumulative parameter displacements of completed task blocks and projects both gradients and Adafactor optimizer updates of later tasks to remove components conflicting with protected task directions, while a lightweight conflict probe sets task order and a subspace-consistency criterion sets the cycle count; across Code, Reason, and Math, PMOPD improves every evaluated capability over MOPD, raising the three-task average by 2.54 points on Qwen2.5-7B and 2.09 points on Llama-3.1-8B.
The work proposes PMOPD, which builds per-matrix low-dimensional subspace memories from the cumulative parameter displacements of completed task blocks and projects both gradients and Adafactor optimizer updates of later tasks to remove components conflicting with protected task directions, while a lightweight conflict probe sets task order and a subspace-consistency criterion sets the cycle count; across Code, Reason, and Math, PMOPD improves every evaluated capability over MOPD, raising the three-task average by 2.54 points on Qwen2.5-7B and 2.09 points on Llama-3.1-8B.
The work proposes PMOPD, which builds per-matrix low-dimensional subspace memories from the cumulative parameter displacements of completed task blocks and projects both gradients and Adafactor optimizer updates of later tasks to remove components conflicting with protected task directions, while a lightweight conflict probe sets task order and a subspace-consistency criterion sets the cycle count; across Code, Reason, and Math, PMOPD improves every evaluated capability over MOPD, raising the three-task average by 2.54 points on Qwen2.5-7B and 2.09 points on Llama-3.1-8B.
The work proposes PMOPD, which builds per-matrix low-dimensional subspace memories from the cumulative parameter displacements of completed task blocks and projects both gradients and Adafactor optimizer updates of later tasks to remove components conflicting with protected task directions, while a lightweight conflict probe sets task order and a subspace-consistency criterion sets the cycle count; across Code, Reason, and Math, PMOPD improves every evaluated capability over MOPD, raising the three-task average by 2.54 points on Qwen2.5-7B and 2.09 points on Llama-3.1-8B.
Nature News Mineral physicist Alisha Clark used the Z machine at Sandia National Laboratories to send shockwaves through water-infused glass samples, recreating pressures near Earth's core during its formation; earlier experiments showed the wet glass became easier to compress as pressure rose, leading her to propose that molten rock could have locked water inside Earth during its magma-ocean phase rather than losing it all or receiving it later from comets and asteroids.
Mineral physicist Alisha Clark used the Z machine at Sandia National Laboratories to send shockwaves through water-infused glass samples, recreating pressures near Earth's core during its formation; earlier experiments showed the wet glass became easier to compress as pressure rose, leading her to propose that molten rock could have locked water inside Earth during its magma-ocean phase rather than losing it all or receiving it later from comets and asteroids.
Mineral physicist Alisha Clark used the Z machine at Sandia National Laboratories to send shockwaves through water-infused glass samples, recreating pressures near Earth's core during its formation; earlier experiments showed the wet glass became easier to compress as pressure rose, leading her to propose that molten rock could have locked water inside Earth during its magma-ocean phase rather than losing it all or receiving it later from comets and asteroids.
Mineral physicist Alisha Clark used the Z machine at Sandia National Laboratories to send shockwaves through water-infused glass samples, recreating pressures near Earth's core during its formation; earlier experiments showed the wet glass became easier to compress as pressure rose, leading her to propose that molten rock could have locked water inside Earth during its magma-ocean phase rather than losing it all or receiving it later from comets and asteroids.
arXiv The work introduces PrismQuant, a quantizer-aware rotation framework that aligns the leading activation eigenspace with the group-constant subspace of asymmetric grouped INT4, formulates rotation design as a Ky Fan trace maximization with a provably optimal closed-form solution, realizes it with compact Householder (compact-WY) transforms without gradient training, and evaluates W4A4KV4 post-training quantization on Llama, Qwen, and Mistral (including dense models up to 70B and a 30B mixture-of-experts model): on Llama-3.2-3B it sets the state of the art among compared methods in perplexity and accuracy, on Llama-3.1-70B it attains 3.85 perplexity and 72.46% average zero-shot accuracy (0.22 percentage points below full precision), and in a Llama-3.
The work introduces PrismQuant, a quantizer-aware rotation framework that aligns the leading activation eigenspace with the group-constant subspace of asymmetric grouped INT4, formulates rotation design as a Ky Fan trace maximization with a provably optimal closed-form solution, realizes it with compact Householder (compact-WY) transforms without gradient training, and evaluates W4A4KV4 post-training quantization on Llama, Qwen, and Mistral (including dense models up to 70B and a 30B mixture-of-experts model): on Llama-3.2-3B it sets the state of the art among compared methods in perplexity and accuracy, on Llama-3.1-70B it attains 3.85 perplexity and 72.46% average zero-shot accuracy (0.22 percentage points below full precision), and in a Llama-3.
The work introduces PrismQuant, a quantizer-aware rotation framework that aligns the leading activation eigenspace with the group-constant subspace of asymmetric grouped INT4, formulates rotation design as a Ky Fan trace maximization with a provably optimal closed-form solution, realizes it with compact Householder (compact-WY) transforms without gradient training, and evaluates W4A4KV4 post-training quantization on Llama, Qwen, and Mistral (including dense models up to 70B and a 30B mixture-of-experts model): on Llama-3.2-3B it sets the state of the art among compared methods in perplexity and accuracy, on Llama-3.1-70B it attains 3.85 perplexity and 72.46% average zero-shot accuracy (0.22 percentage points below full precision), and in a Llama-3.
The work introduces PrismQuant, a quantizer-aware rotation framework that aligns the leading activation eigenspace with the group-constant subspace of asymmetric grouped INT4, formulates rotation design as a Ky Fan trace maximization with a provably optimal closed-form solution, realizes it with compact Householder (compact-WY) transforms without gradient training, and evaluates W4A4KV4 post-training quantization on Llama, Qwen, and Mistral (including dense models up to 70B and a 30B mixture-of-experts model): on Llama-3.2-3B it sets the state of the art among compared methods in perplexity and accuracy, on Llama-3.1-70B it attains 3.85 perplexity and 72.46% average zero-shot accuracy (0.22 percentage points below full precision), and in a Llama-3.
arXiv The work recasts extreme visual-token compression as a token-parameterization problem, separating basis transformation and structured truncation (which fix the retained subspace and compressibility) from coordinate organization (which affects optimization and cross-modal alignment, i.e. learnability), and designs Braco, a lightweight four-step coder combining transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling; experiments show Braco forms the favorable empirical accuracy–efficiency frontier under 23x–64x compression and remains competitive at 144x, reaching 95.2% accuracy while reducing prefill FLOPs by 84.2%–86.
The work recasts extreme visual-token compression as a token-parameterization problem, separating basis transformation and structured truncation (which fix the retained subspace and compressibility) from coordinate organization (which affects optimization and cross-modal alignment, i.e. learnability), and designs Braco, a lightweight four-step coder combining transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling; experiments show Braco forms the favorable empirical accuracy–efficiency frontier under 23x–64x compression and remains competitive at 144x, reaching 95.2% accuracy while reducing prefill FLOPs by 84.2%–86.
The work recasts extreme visual-token compression as a token-parameterization problem, separating basis transformation and structured truncation (which fix the retained subspace and compressibility) from coordinate organization (which affects optimization and cross-modal alignment, i.e. learnability), and designs Braco, a lightweight four-step coder combining transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling; experiments show Braco forms the favorable empirical accuracy–efficiency frontier under 23x–64x compression and remains competitive at 144x, reaching 95.2% accuracy while reducing prefill FLOPs by 84.2%–86.
The work recasts extreme visual-token compression as a token-parameterization problem, separating basis transformation and structured truncation (which fix the retained subspace and compressibility) from coordinate organization (which affects optimization and cross-modal alignment, i.e. learnability), and designs Braco, a lightweight four-step coder combining transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling; experiments show Braco forms the favorable empirical accuracy–efficiency frontier under 23x–64x compression and remains competitive at 144x, reaching 95.2% accuracy while reducing prefill FLOPs by 84.2%–86.
arXiv The work introduces budgeted active test-time adaptation (ATTA), in which labels are available for only a fraction of batches in a long test stream, and proposes WISE-ATTA: lightweight online signals (the fraction of low-entropy predictions) plus a sliding-history quantile threshold and a budget-debt controller decide when to request supervision, while prediction drift relative to an EMA anchor selects a single sample to label; on ImageNet-C under CTTA/FTTA and on ImageNet-R/K/A it matches or improves on recent ATTA methods at one label per batch and reports up to 50% fewer labels.
The work introduces budgeted active test-time adaptation (ATTA), in which labels are available for only a fraction of batches in a long test stream, and proposes WISE-ATTA: lightweight online signals (the fraction of low-entropy predictions) plus a sliding-history quantile threshold and a budget-debt controller decide when to request supervision, while prediction drift relative to an EMA anchor selects a single sample to label; on ImageNet-C under CTTA/FTTA and on ImageNet-R/K/A it matches or improves on recent ATTA methods at one label per batch and reports up to 50% fewer labels.
The work introduces budgeted active test-time adaptation (ATTA), in which labels are available for only a fraction of batches in a long test stream, and proposes WISE-ATTA: lightweight online signals (the fraction of low-entropy predictions) plus a sliding-history quantile threshold and a budget-debt controller decide when to request supervision, while prediction drift relative to an EMA anchor selects a single sample to label; on ImageNet-C under CTTA/FTTA and on ImageNet-R/K/A it matches or improves on recent ATTA methods at one label per batch and reports up to 50% fewer labels.
The work introduces budgeted active test-time adaptation (ATTA), in which labels are available for only a fraction of batches in a long test stream, and proposes WISE-ATTA: lightweight online signals (the fraction of low-entropy predictions) plus a sliding-history quantile threshold and a budget-debt controller decide when to request supervision, while prediction drift relative to an EMA anchor selects a single sample to label; on ImageNet-C under CTTA/FTTA and on ImageNet-R/K/A it matches or improves on recent ATTA methods at one label per batch and reports up to 50% fewer labels.
arXiv The work introduces FRAC, a selective state space model architecture derived from fractional dynamics that approximates a heavy-tailed fractional kernel with a finite-state, log-spaced sum of exponential modes, replacing exponential forgetting with power-law long memory and improving long-context performance over SSM baselines such as Mamba2, GDN, and Mamba3 on synthetic long-tail and recall tasks, 1.3B-parameter language modeling, and DNA modeling, while staying competitive on short-context tasks.
The work introduces FRAC, a selective state space model architecture derived from fractional dynamics that approximates a heavy-tailed fractional kernel with a finite-state, log-spaced sum of exponential modes, replacing exponential forgetting with power-law long memory and improving long-context performance over SSM baselines such as Mamba2, GDN, and Mamba3 on synthetic long-tail and recall tasks, 1.3B-parameter language modeling, and DNA modeling, while staying competitive on short-context tasks.
The work introduces FRAC, a selective state space model architecture derived from fractional dynamics that approximates a heavy-tailed fractional kernel with a finite-state, log-spaced sum of exponential modes, replacing exponential forgetting with power-law long memory and improving long-context performance over SSM baselines such as Mamba2, GDN, and Mamba3 on synthetic long-tail and recall tasks, 1.3B-parameter language modeling, and DNA modeling, while staying competitive on short-context tasks.
The work introduces FRAC, a selective state space model architecture derived from fractional dynamics that approximates a heavy-tailed fractional kernel with a finite-state, log-spaced sum of exponential modes, replacing exponential forgetting with power-law long memory and improving long-context performance over SSM baselines such as Mamba2, GDN, and Mamba3 on synthetic long-tail and recall tasks, 1.3B-parameter language modeling, and DNA modeling, while staying competitive on short-context tasks.
arXiv The work proposes SaveRouter, a sparse-supervision routing framework that selectively acquires informative query-model feedback and shares capability information across related queries, using only about 33-41% of available training feedback across four routing benchmarks while maintaining competitive or better routing quality and reducing the break-even deployment volume by roughly 1.9-9.5x relative to the fastest conventional fully supervised router.
The work proposes SaveRouter, a sparse-supervision routing framework that selectively acquires informative query-model feedback and shares capability information across related queries, using only about 33-41% of available training feedback across four routing benchmarks while maintaining competitive or better routing quality and reducing the break-even deployment volume by roughly 1.9-9.5x relative to the fastest conventional fully supervised router.
The work proposes SaveRouter, a sparse-supervision routing framework that selectively acquires informative query-model feedback and shares capability information across related queries, using only about 33-41% of available training feedback across four routing benchmarks while maintaining competitive or better routing quality and reducing the break-even deployment volume by roughly 1.9-9.5x relative to the fastest conventional fully supervised router.
The work proposes SaveRouter, a sparse-supervision routing framework that selectively acquires informative query-model feedback and shares capability information across related queries, using only about 33-41% of available training feedback across four routing benchmarks while maintaining competitive or better routing quality and reducing the break-even deployment volume by roughly 1.9-9.5x relative to the fastest conventional fully supervised router.
arXiv To address two obstacles in scaling Distributional Diffusion Models (DDMs)—multi-particle training overhead that grows with the number of particles, and globally fixed scoring-rule hyperparameters that force a single trade-off across sampling budgets—the work defers particle expansion to late transformer layers and introduces time-dependent scoring-rule schedules informed by the dynamical regimes of Biroli2024; combined with a DiT-based latent setup, this makes DDM training practical on class-conditional ImageNet-256², reaching 4.48 FID at 4 steps and 2.38 at 50 steps with DiT-XL/2 from a single model trained from scratch in one stage, without a teacher, self-distillation or JVPs, and the same recipe transfers to text-to-image generation.
To address two obstacles in scaling Distributional Diffusion Models (DDMs)—multi-particle training overhead that grows with the number of particles, and globally fixed scoring-rule hyperparameters that force a single trade-off across sampling budgets—the work defers particle expansion to late transformer layers and introduces time-dependent scoring-rule schedules informed by the dynamical regimes of Biroli2024; combined with a DiT-based latent setup, this makes DDM training practical on class-conditional ImageNet-256², reaching 4.48 FID at 4 steps and 2.38 at 50 steps with DiT-XL/2 from a single model trained from scratch in one stage, without a teacher, self-distillation or JVPs, and the same recipe transfers to text-to-image generation.
To address two obstacles in scaling Distributional Diffusion Models (DDMs)—multi-particle training overhead that grows with the number of particles, and globally fixed scoring-rule hyperparameters that force a single trade-off across sampling budgets—the work defers particle expansion to late transformer layers and introduces time-dependent scoring-rule schedules informed by the dynamical regimes of Biroli2024; combined with a DiT-based latent setup, this makes DDM training practical on class-conditional ImageNet-256², reaching 4.48 FID at 4 steps and 2.38 at 50 steps with DiT-XL/2 from a single model trained from scratch in one stage, without a teacher, self-distillation or JVPs, and the same recipe transfers to text-to-image generation.
To address two obstacles in scaling Distributional Diffusion Models (DDMs)—multi-particle training overhead that grows with the number of particles, and globally fixed scoring-rule hyperparameters that force a single trade-off across sampling budgets—the work defers particle expansion to late transformer layers and introduces time-dependent scoring-rule schedules informed by the dynamical regimes of Biroli2024; combined with a DiT-based latent setup, this makes DDM training practical on class-conditional ImageNet-256², reaching 4.48 FID at 4 steps and 2.38 at 50 steps with DiT-XL/2 from a single model trained from scratch in one stage, without a teacher, self-distillation or JVPs, and the same recipe transfers to text-to-image generation.
arXiv Meituan's LongCat team presents LongCat-DeepResearch, which shifts early research iteration from the full report to an executable ResearchSpec: multiple planning agents first search external sources and consolidate a research plan, researchers then investigate and draft citation-bearing sections in parallel independent contexts, and a Global Editor assigns cross-section ownership while Local Editors make targeted revisions, yielding 55.25 on DeepResearchBench, 51.35 on DeepResearchBench II, and 79.83 on ResearchRubrics, and 76.04 on an in-house benchmark, second among four compared systems.
Meituan's LongCat team presents LongCat-DeepResearch, which shifts early research iteration from the full report to an executable ResearchSpec: multiple planning agents first search external sources and consolidate a research plan, researchers then investigate and draft citation-bearing sections in parallel independent contexts, and a Global Editor assigns cross-section ownership while Local Editors make targeted revisions, yielding 55.25 on DeepResearchBench, 51.35 on DeepResearchBench II, and 79.83 on ResearchRubrics, and 76.04 on an in-house benchmark, second among four compared systems.
Meituan's LongCat team presents LongCat-DeepResearch, which shifts early research iteration from the full report to an executable ResearchSpec: multiple planning agents first search external sources and consolidate a research plan, researchers then investigate and draft citation-bearing sections in parallel independent contexts, and a Global Editor assigns cross-section ownership while Local Editors make targeted revisions, yielding 55.25 on DeepResearchBench, 51.35 on DeepResearchBench II, and 79.83 on ResearchRubrics, and 76.04 on an in-house benchmark, second among four compared systems.
Meituan's LongCat team presents LongCat-DeepResearch, which shifts early research iteration from the full report to an executable ResearchSpec: multiple planning agents first search external sources and consolidate a research plan, researchers then investigate and draft citation-bearing sections in parallel independent contexts, and a Global Editor assigns cross-section ownership while Local Editors make targeted revisions, yielding 55.25 on DeepResearchBench, 51.35 on DeepResearchBench II, and 79.83 on ResearchRubrics, and 76.04 on an in-house benchmark, second among four compared systems.
arXiv The work introduces APM-Bench, which reformulates egocentric streaming interaction as multi-session life trajectories (549 sessions, 104 trajectories, 2,719 candidates, averaging 69 minutes of video per trajectory) and evaluates general video models and eight specialized memory systems on cross-session understanding, real-time perception, and adaptive response, plus an evidence-availability-aware test; results reveal a clear utility-latency-storage trade-off: raw video memory gives the strongest cross-session performance but needs GiB-scale storage and high latency, text summaries cut storage to the KiB scale at the cost of cross-session performance, event-structured memory shows the strongest utility among specialized systems, and adaptive response remains difficult even with rich history
The work introduces APM-Bench, which reformulates egocentric streaming interaction as multi-session life trajectories (549 sessions, 104 trajectories, 2,719 candidates, averaging 69 minutes of video per trajectory) and evaluates general video models and eight specialized memory systems on cross-session understanding, real-time perception, and adaptive response, plus an evidence-availability-aware test; results reveal a clear utility-latency-storage trade-off: raw video memory gives the strongest cross-session performance but needs GiB-scale storage and high latency, text summaries cut storage to the KiB scale at the cost of cross-session performance, event-structured memory shows the strongest utility among specialized systems, and adaptive response remains difficult even with rich history
The work introduces APM-Bench, which reformulates egocentric streaming interaction as multi-session life trajectories (549 sessions, 104 trajectories, 2,719 candidates, averaging 69 minutes of video per trajectory) and evaluates general video models and eight specialized memory systems on cross-session understanding, real-time perception, and adaptive response, plus an evidence-availability-aware test; results reveal a clear utility-latency-storage trade-off: raw video memory gives the strongest cross-session performance but needs GiB-scale storage and high latency, text summaries cut storage to the KiB scale at the cost of cross-session performance, event-structured memory shows the strongest utility among specialized systems, and adaptive response remains difficult even with rich history
The work introduces APM-Bench, which reformulates egocentric streaming interaction as multi-session life trajectories (549 sessions, 104 trajectories, 2,719 candidates, averaging 69 minutes of video per trajectory) and evaluates general video models and eight specialized memory systems on cross-session understanding, real-time perception, and adaptive response, plus an evidence-availability-aware test; results reveal a clear utility-latency-storage trade-off: raw video memory gives the strongest cross-session performance but needs GiB-scale storage and high latency, text summaries cut storage to the KiB scale at the cost of cross-session performance, event-structured memory shows the strongest utility among specialized systems, and adaptive response remains difficult even with rich history
arXiv The authors propose a two-axis taxonomy of spoken conversational memory — acoustic evidence type (speech semantics, speaker identity, paralinguistic cues, environmental sound) crossed with memory operation (information extraction, multi-session reasoning, temporal evolution tracking, answer refusal) — and build VoxMem on it: 3,196 evaluation instances over 34,743 spoken sessions (177 hours) at four context budgets from 8K to 64K tokens; evaluating 15 large audio language models, no model exceeds 40% overall accuracy at 32K (best 38.5%), models remember what was said far better than who said it, how it was said, or what was audible, and replacing audio with exact transcripts drops speaker accuracy from 69.8% to 10.3% while speech-semantics accuracy barely moves (75.9% to 71.0%).
The authors propose a two-axis taxonomy of spoken conversational memory — acoustic evidence type (speech semantics, speaker identity, paralinguistic cues, environmental sound) crossed with memory operation (information extraction, multi-session reasoning, temporal evolution tracking, answer refusal) — and build VoxMem on it: 3,196 evaluation instances over 34,743 spoken sessions (177 hours) at four context budgets from 8K to 64K tokens; evaluating 15 large audio language models, no model exceeds 40% overall accuracy at 32K (best 38.5%), models remember what was said far better than who said it, how it was said, or what was audible, and replacing audio with exact transcripts drops speaker accuracy from 69.8% to 10.3% while speech-semantics accuracy barely moves (75.9% to 71.0%).
The authors propose a two-axis taxonomy of spoken conversational memory — acoustic evidence type (speech semantics, speaker identity, paralinguistic cues, environmental sound) crossed with memory operation (information extraction, multi-session reasoning, temporal evolution tracking, answer refusal) — and build VoxMem on it: 3,196 evaluation instances over 34,743 spoken sessions (177 hours) at four context budgets from 8K to 64K tokens; evaluating 15 large audio language models, no model exceeds 40% overall accuracy at 32K (best 38.5%), models remember what was said far better than who said it, how it was said, or what was audible, and replacing audio with exact transcripts drops speaker accuracy from 69.8% to 10.3% while speech-semantics accuracy barely moves (75.9% to 71.0%).
The authors propose a two-axis taxonomy of spoken conversational memory — acoustic evidence type (speech semantics, speaker identity, paralinguistic cues, environmental sound) crossed with memory operation (information extraction, multi-session reasoning, temporal evolution tracking, answer refusal) — and build VoxMem on it: 3,196 evaluation instances over 34,743 spoken sessions (177 hours) at four context budgets from 8K to 64K tokens; evaluating 15 large audio language models, no model exceeds 40% overall accuracy at 32K (best 38.5%), models remember what was said far better than who said it, how it was said, or what was audible, and replacing audio with exact transcripts drops speaker accuracy from 69.8% to 10.3% while speech-semantics accuracy barely moves (75.9% to 71.0%).
arXiv The study systematically characterizes the scaling properties of same-family on-policy distillation (OPD) on Qwen2.5 (0.5B–14B) math reasoning, finding a regular early useful-transfer regime in which held-out accuracy rises approximately linearly with the square root of token-level reverse KL from the student initialization, and fitting power laws in student scale, teacher scale, and teacher gold score that predict peak accuracy and transfer rate, with every weak-to-strong student peaking above its own teacher.
The study systematically characterizes the scaling properties of same-family on-policy distillation (OPD) on Qwen2.5 (0.5B–14B) math reasoning, finding a regular early useful-transfer regime in which held-out accuracy rises approximately linearly with the square root of token-level reverse KL from the student initialization, and fitting power laws in student scale, teacher scale, and teacher gold score that predict peak accuracy and transfer rate, with every weak-to-strong student peaking above its own teacher.
The study systematically characterizes the scaling properties of same-family on-policy distillation (OPD) on Qwen2.5 (0.5B–14B) math reasoning, finding a regular early useful-transfer regime in which held-out accuracy rises approximately linearly with the square root of token-level reverse KL from the student initialization, and fitting power laws in student scale, teacher scale, and teacher gold score that predict peak accuracy and transfer rate, with every weak-to-strong student peaking above its own teacher.
The study systematically characterizes the scaling properties of same-family on-policy distillation (OPD) on Qwen2.5 (0.5B–14B) math reasoning, finding a regular early useful-transfer regime in which held-out accuracy rises approximately linearly with the square root of token-level reverse KL from the student initialization, and fitting power laws in student scale, teacher scale, and teacher gold score that predict peak accuracy and transfer rate, with every weak-to-strong student peaking above its own teacher.
arXiv The work presents SoL-Refiner, a one-step video refiner that turns low-resolution generator outputs into 4K video through a three-stage recipe of high-resolution continual training, reinforcement-learning post-training with frame-based reward models, and final one-step distillation, and introduces Refiner-Bench, a video refinement benchmark using a shared-input protocol to compare refiners at roughly 2K output resolution; at 2K the one-step model outperforms all evaluated external refiners on VBench and UniPercept averages, at 3840×2176 it improves both metrics over the three-step LTX-2.3 Refiner, and with the complete acceleration stack it achieves an 8.91× speedup in refinement latency over the same baseline in the 2K latency setting.
The work presents SoL-Refiner, a one-step video refiner that turns low-resolution generator outputs into 4K video through a three-stage recipe of high-resolution continual training, reinforcement-learning post-training with frame-based reward models, and final one-step distillation, and introduces Refiner-Bench, a video refinement benchmark using a shared-input protocol to compare refiners at roughly 2K output resolution; at 2K the one-step model outperforms all evaluated external refiners on VBench and UniPercept averages, at 3840×2176 it improves both metrics over the three-step LTX-2.3 Refiner, and with the complete acceleration stack it achieves an 8.91× speedup in refinement latency over the same baseline in the 2K latency setting.
The work presents SoL-Refiner, a one-step video refiner that turns low-resolution generator outputs into 4K video through a three-stage recipe of high-resolution continual training, reinforcement-learning post-training with frame-based reward models, and final one-step distillation, and introduces Refiner-Bench, a video refinement benchmark using a shared-input protocol to compare refiners at roughly 2K output resolution; at 2K the one-step model outperforms all evaluated external refiners on VBench and UniPercept averages, at 3840×2176 it improves both metrics over the three-step LTX-2.3 Refiner, and with the complete acceleration stack it achieves an 8.91× speedup in refinement latency over the same baseline in the 2K latency setting.
The work presents SoL-Refiner, a one-step video refiner that turns low-resolution generator outputs into 4K video through a three-stage recipe of high-resolution continual training, reinforcement-learning post-training with frame-based reward models, and final one-step distillation, and introduces Refiner-Bench, a video refinement benchmark using a shared-input protocol to compare refiners at roughly 2K output resolution; at 2K the one-step model outperforms all evaluated external refiners on VBench and UniPercept averages, at 3840×2176 it improves both metrics over the three-step LTX-2.3 Refiner, and with the complete acceleration stack it achieves an 8.91× speedup in refinement latency over the same baseline in the 2K latency setting.
arXiv PanoVLN performs vision-and-language navigation in continuous environments with a 4B vision-language model and RGB-only panoramic input, combining longer action-sequence supervision, confidence-guided execution, decision-centric training data, and fused panoramic geometric features to reach 77.3% and 78.0% success rate on R2R-CE and RxR-CE Val-Unseen, exceeding the previous best by 11.9 and 8.7 percentage points, and enabling real-world quadruped navigation with fewer pauses.
PanoVLN performs vision-and-language navigation in continuous environments with a 4B vision-language model and RGB-only panoramic input, combining longer action-sequence supervision, confidence-guided execution, decision-centric training data, and fused panoramic geometric features to reach 77.3% and 78.0% success rate on R2R-CE and RxR-CE Val-Unseen, exceeding the previous best by 11.9 and 8.7 percentage points, and enabling real-world quadruped navigation with fewer pauses.
PanoVLN performs vision-and-language navigation in continuous environments with a 4B vision-language model and RGB-only panoramic input, combining longer action-sequence supervision, confidence-guided execution, decision-centric training data, and fused panoramic geometric features to reach 77.3% and 78.0% success rate on R2R-CE and RxR-CE Val-Unseen, exceeding the previous best by 11.9 and 8.7 percentage points, and enabling real-world quadruped navigation with fewer pauses.
PanoVLN performs vision-and-language navigation in continuous environments with a 4B vision-language model and RGB-only panoramic input, combining longer action-sequence supervision, confidence-guided execution, decision-centric training data, and fused panoramic geometric features to reach 77.3% and 78.0% success rate on R2R-CE and RxR-CE Val-Unseen, exceeding the previous best by 11.9 and 8.7 percentage points, and enabling real-world quadruped navigation with fewer pauses.