École Supérieure en Informatique de Sidi Bel Abbes

Quantum programs are judged by probabilistic outputs and domain-specific checkers, so a syntactically valid test can still disagree with the benchmark verdict. QTestBench has models write syntactic, semantic, and behavioral pytest modules per task, injects candidate solutions, and compares each test verdict against the benchmark checker rather than the tests themselves across three validation cohorts. On eight PennyLane QHack challenges and ten Qiskit HumanEval tasks, generated suites almost always execute, yet discrimination lags: the best test quality is 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality.

Quantum programs are judged by probabilistic outputs and domain-specific checkers, so a syntactically valid test can still disagree with the benchmark verdict. QTestBench has models write syntactic, semantic, and behavioral pytest modules per task, injects candidate solutions, and compares each test verdict against the benchmark checker rather than the tests themselves across three validation cohorts. On eight PennyLane QHack challenges and ten Qiskit HumanEval tasks, generated suites almost always execute, yet discrimination lags: the best test quality is 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality.