# Rubric-Based Evaluation

FrontierPhysics replaces fixed physics questions and paper reproduction with given solutions by 56 end-to-end tasks that 42 physicists built from their own research, spanning nine areas of theoretical, computational and experimental physics. An agent receives only a high-level prompt like a principal investigator's brief, then must review the literature, set a plan, run the research, and deliver both deterministically checkable numerical outputs and a manuscript. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts, and manual review of trajectories traces the main weaknesses to insufficiently rigorous reasoning and limited expert intuition rather than gaps in physics knowledge.

FrontierPhysics replaces fixed physics questions and paper reproduction with given solutions by 56 end-to-end tasks that 42 physicists built from their own research, spanning nine areas of theoretical, computational and experimental physics. An agent receives only a high-level prompt like a principal investigator's brief, then must review the literature, set a plan, run the research, and deliver both deterministically checkable numerical outputs and a manuscript. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts, and manual review of trajectories traces the main weaknesses to insufficiently rigorous reasoning and limited expert intuition rather than gaps in physics knowledge.

The work introduces LexReward, a taxonomy built with legal experts that decomposes legal response quality into Style (lexical and syntactic), Element (subjects, facts, statutes, decisions) and Chain (order, completeness, correctness, non-redundancy), designs rubrics scored by rules or an LLM judge, uses the resulting rewards to build preference data and train LexRM, a family of Chinese legal reward models; experiments show the rubrics distinguish responses of different quality, DPO improves generation across all three dimensions, LexRM-guided selection beats random selection in test-time scaling, and LexRM used for GRPO yields gains in every dimension and outperforms rule-based outcome rewards.

The work introduces LexReward, a taxonomy built with legal experts that decomposes legal response quality into Style (lexical and syntactic), Element (subjects, facts, statutes, decisions) and Chain (order, completeness, correctness, non-redundancy), designs rubrics scored by rules or an LLM judge, uses the resulting rewards to build preference data and train LexRM, a family of Chinese legal reward models; experiments show the rubrics distinguish responses of different quality, DPO improves generation across all three dimensions, LexRM-guided selection beats random selection in test-time scaling, and LexRM used for GRPO yields gains in every dimension and outperforms rule-based outcome rewards.

The work builds SciGen-Verify, a benchmark for explainable verification of scientific image generation (1,350 samples: 406 instruction following, 287 multidisciplinary reasoning, 657 world knowledge, with a three-tier cascading protocol over binary judgement, explanation, and editing instruction), and develops SciGen-Verifier on Qwen3-VL-8B-Instruct trained by cold-start supervised fine-tuning (48,311 instances) followed by a curriculum-based two-stage reinforcement learning pipeline (rubric-guided process reward, then outcome reward), reaching 86.28% overall judgement accuracy, 81.79% explanation accuracy, and 57.42% editing-instruction accuracy on the benchmark while also serving as an online critic for iterative image rectification.

The work builds SciGen-Verify, a benchmark for explainable verification of scientific image generation (1,350 samples: 406 instruction following, 287 multidisciplinary reasoning, 657 world knowledge, with a three-tier cascading protocol over binary judgement, explanation, and editing instruction), and develops SciGen-Verifier on Qwen3-VL-8B-Instruct trained by cold-start supervised fine-tuning (48,311 instances) followed by a curriculum-based two-stage reinforcement learning pipeline (rubric-guided process reward, then outcome reward), reaching 86.28% overall judgement accuracy, 81.79% explanation accuracy, and 57.42% editing-instruction accuracy on the benchmark while also serving as an online critic for iterative image rectification.

ShieldVLA is a safety-aligned fine-tuning framework for vision-language-action (VLA) models that learns a model-free Hamilton-Jacobi reachability value function from visual observations as a safety critic, uses that critic to gate policy optimization into reward maximization inside feasible regions and recovery near unsafe states, and replaces manual per-step cost labels with rubric-based VLM safety scores; across five navigation and manipulation benchmarks (Dubins-VL, TurtleBot-Nav, Safety-CHORES Nav/Fetch, Franka-Reach) and multiple VLA backbones it reduces cumulative safety cost by 57% on average and improves task success rate by 0.13 over SafeVLA.

ShieldVLA is a safety-aligned fine-tuning framework for vision-language-action (VLA) models that learns a model-free Hamilton-Jacobi reachability value function from visual observations as a safety critic, uses that critic to gate policy optimization into reward maximization inside feasible regions and recovery near unsafe states, and replaces manual per-step cost labels with rubric-based VLM safety scores; across five navigation and manipulation benchmarks (Dubins-VL, TurtleBot-Nav, Safety-CHORES Nav/Fetch, Franka-Reach) and multiple VLA backbones it reduces cumulative safety cost by 57% on average and improves task success rate by 0.13 over SafeVLA.

The work first shows, on 900 human-rated SVG samples generated by Claude-Opus-4.6, Qwen3-32B, and Qwen3-8B, that prompting a vision-language judge with a multi-axis rubric correlates with human judgment far better than scalar metrics such as CLIP and aesthetic scores (Spearman 0.7929, Goodman-Kruskal Gamma 0.7574), then introduces RULER: it derives a six-item instance-aware rubric from the text instruction alone, has a judge VLM score each rendered rollout item-by-item, and optimizes the weighted reward with GRPO, raising the rubric score on MMSVG-Illustration and MMSVG-Icon from the Qwen3-8B backbone's 0.432 and 0.395 to 0.693 and 0.683, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3 without paired SVG ground truth or human preference labels.

The work first shows, on 900 human-rated SVG samples generated by Claude-Opus-4.6, Qwen3-32B, and Qwen3-8B, that prompting a vision-language judge with a multi-axis rubric correlates with human judgment far better than scalar metrics such as CLIP and aesthetic scores (Spearman 0.7929, Goodman-Kruskal Gamma 0.7574), then introduces RULER: it derives a six-item instance-aware rubric from the text instruction alone, has a judge VLM score each rendered rollout item-by-item, and optimizes the weighted reward with GRPO, raising the rubric score on MMSVG-Illustration and MMSVG-Icon from the Qwen3-8B backbone's 0.432 and 0.395 to 0.693 and 0.683, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3 without paired SVG ground truth or human preference labels.