Brian Ko

AI research agents must train and evaluate a model for every candidate program, so execution cost caps how many candidates they can compare per step. MLE-World, finetuned from Qwen3.5-4B, reads a program's code and data description and predicts its terminal output, exception type and validation score, plus a confidence score. On held-out programs from 17 MLE-bench Lite tasks its predicted terminal output matches the real run on 77.9% of programs versus 41.1% for the strongest prompted baseline, and search guided by it reaches up to 22.1% higher observed mean performance than the agent without a world model while using less real execution.

AI research agents must train and evaluate a model for every candidate program, so execution cost caps how many candidates they can compare per step. MLE-World, finetuned from Qwen3.5-4B, reads a program's code and data description and predicts its terminal output, exception type and validation score, plus a confidence score. On held-out programs from 17 MLE-bench Lite tasks its predicted terminal output matches the real run on 77.9% of programs versus 41.1% for the strongest prompted baseline, and search guided by it reaches up to 22.1% higher observed mean performance than the agent without a world model while using less real execution.