CLEAR turns papers into checkable execution specs first: 29.87 on PaperBench at $7.91 per paper
Synopsis
CLEAR inserts a structured execution-specification layer between a research paper and runnable code, converting research documents into human-checkable implementation units, dependencies, interfaces and evaluation procedures before generating and locally repairing code. On 20 PaperBench tasks it averages a score of 29.87 at an average cost of $7.91 per paper; against AiScientist, cost falls 35.16% while the score gives up 3.86 points. On MLE-Bench Lite, all 22 tasks produce valid submissions and 77.27% win any medal.
Interpretation
CLEAR splits machine-learning research engineering into two questions: what should be implemented, and how it is implemented. Research documents stop serving directly as prompts for code generation and are first converted into a human-inspectable execution specification that later planning, coding and repair all reference. Earlier agent systems coupled research interpretation directly with code generation and execution in one loop, so implicit assumptions in a document were filled in during generation, early misinterpretations were written into downstream code, and failures were often recovered by regenerating large portions of the workflow. On 20 PaperBench tasks CLEAR averages 29.87 at $7.91 per paper; AiScientist averages 33.73 at $12.20 per paper, so CLEAR costs 35.16% less and scores 3.86 points lower. The two systems use different backbone models: CLEAR uses Claude-Opus 4.6 and Claude-Sonnet 4.6, while AiScientist uses Gemini-3-Flash and GLM-5.
The workflow runs in three stages: parse the document into a specification, generate code in specification-bounded units, then locate failures during execution and repair only the affected parts. The specification has a machine-oriented structured view and a human-readable method summary; the structured view records algorithms, inputs and outputs, dependencies, parameters, interfaces, evaluation metrics and submission constraints, while the summary serves as the inspection interface. A planning model decomposes the work into specification-bounded subtasks and sets the number of low-cost coding workers by task complexity; a lifecycle harness tracks specification, generation, execution, failure and repair states, and on PaperBench CLEAR consumes about 564K tokens per task versus about 842K without the specification layer.
The specification layer's contribution shows up directly in ablation: removing document-to-specification parsing degrades both effectiveness and cost. The no-specification variant lets the planning model read raw documents directly; on PaperBench its average score drops 19.18% and average cost rises 27.43%, and on MLE-Bench Lite valid submission, above median and any medal fall by 9.09%, 10.53% and 17.64% respectively. A homogeneous configuration that keeps the specification workflow but uses GLM-5 for both planning and coding scores 11.59% below full CLEAR on PaperBench at 51.07% higher average cost, indicating the Opus and Sonnet division of labor trades off performance and cost more favorably.
Perspective
Evaluation covers 20 paper-replication tasks on PaperBench and 22 competition tasks on MLE-Bench Lite, with one Nvidia A800 80GB GPU and a 24-hour budget per task, and GPT-5.4 as the PaperBench judge. The experiments state that no manual review or modification of system-generated documents was used, so the reported behavior is without human intervention.
CLEAR varies noticeably across PaperBench tasks, with larger gaps remaining on tasks such as adaptive-pruning and robust-clip, so effectiveness is task-dependent. Cross-model token counts use different model families and tokenizers and should be read as a system-level usage comparison rather than a model-independent measure of computational cost.