# Benchmark Evaluation

RSI Arena hands language-model post-training to ten frontier models: each agent starts from the same 30-billion-parameter base, picks its own data, writes its own training code and nominates checkpoints within six days and 1,000 GPU-hours. All ten produced working assistants, and the model trained by GPT-6 Astra closed a median 76% of the base-to-Instruct gap across 12 capability benchmarks; with answer length and formatting controlled, its human-preference rating is level with NVIDIA Instruct, 1,095 against 1,094. A second stage returned each model's blind battle records to its agent, and eight of the ten models improved their style-controlled rating relative to NVIDIA Instruct, while benchmark gains stayed modest and showed no clear link to preference gains.

RSI Arena hands language-model post-training to ten frontier models: each agent starts from the same 30-billion-parameter base, picks its own data, writes its own training code and nominates checkpoints within six days and 1,000 GPU-hours. All ten produced working assistants, and the model trained by GPT-6 Astra closed a median 76% of the base-to-Instruct gap across 12 capability benchmarks; with answer length and formatting controlled, its human-preference rating is level with NVIDIA Instruct, 1,095 against 1,094. A second stage returned each model's blind battle records to its agent, and eight of the ten models improved their style-controlled rating relative to NVIDIA Instruct, while benchmark gains stayed modest and showed no clear link to preference gains.

This study had five current large language models (GPT 5.3, Claude Sonnet 4.6, Gemini 2.5 Flash, Perplexity, and Grok) reproduce three statistical tasks from a published clinical workflow under identical datasets and standardised prompts—descriptive table generation, Kaplan–Meier survival analysis, and Cox proportional hazards modelling—and compared their outputs with analyses by two experts trained in mathematical statistics on grouping correctness, numerical accuracy, missing-value handling, and quality of generated R code, finding that all five models produced correct descriptive statistics once dataset variables were specified explicitly, that two models failed the initial descriptive benchmark because of variable-name ambiguity but recovered after prompt clarification, that all models

This study had five current large language models (GPT 5.3, Claude Sonnet 4.6, Gemini 2.5 Flash, Perplexity, and Grok) reproduce three statistical tasks from a published clinical workflow under identical datasets and standardised prompts—descriptive table generation, Kaplan–Meier survival analysis, and Cox proportional hazards modelling—and compared their outputs with analyses by two experts trained in mathematical statistics on grouping correctness, numerical accuracy, missing-value handling, and quality of generated R code, finding that all five models produced correct descriptive statistics once dataset variables were specified explicitly, that two models failed the initial descriptive benchmark because of variable-name ambiguity but recovered after prompt clarification, that all models

Using 20 standardised geriatric pharmacotherapy vignettes (fictional older adults aged 72–88 across four clinical domains, each containing three potentially inappropriate medications and one START-type omission anchored to the AGS Beers Criteria and STOPP/START version 3), this study had GPT-5.2, Claude Sonnet 4.5, and Gemini 3 Pro respond under an identical master prompt and default end-user settings, with two geriatricians blinded to model identity independently rating anonymised outputs on a 100-point rubric (output quality 0–80, critical safety-risk prioritisation 0–20), finding that Stage 1 item-level answer-key concordance was uniformly high with limited between-model discrimination while expert-rated total scores differed significantly across models (p < 0.001; Kendall's W = 0.

Using 20 standardised geriatric pharmacotherapy vignettes (fictional older adults aged 72–88 across four clinical domains, each containing three potentially inappropriate medications and one START-type omission anchored to the AGS Beers Criteria and STOPP/START version 3), this study had GPT-5.2, Claude Sonnet 4.5, and Gemini 3 Pro respond under an identical master prompt and default end-user settings, with two geriatricians blinded to model identity independently rating anonymised outputs on a 100-point rubric (output quality 0–80, critical safety-risk prioritisation 0–20), finding that Stage 1 item-level answer-key concordance was uniformly high with limited between-model discrimination while expert-rated total scores differed significantly across models (p < 0.001; Kendall's W = 0.