Mingyu Chen

For repository-level software development, researchers built ReqClarifyBench, pairing 341 tasks with vague user intents, human-validated requirement documents, and code evaluation artifacts, and ran multi-turn clarification with simulated users carrying three real-world personas. Across six LLMs, asking effectively did not mean writing better documents: the strongest questioner scored lower on requirement alignment, yet adding interactive clarification raised average alignment from 0.212 to 0.494, a 133.1% relative gain, with code-generation scores rising too.

For repository-level software development, researchers built ReqClarifyBench, pairing 341 tasks with vague user intents, human-validated requirement documents, and code evaluation artifacts, and ran multi-turn clarification with simulated users carrying three real-world personas. Across six LLMs, asking effectively did not mean writing better documents: the strongest questioner scored lower on requirement alignment, yet adding interactive clarification raised average alignment from 0.212 to 0.494, a 133.1% relative gain, with code-generation scores rising too.