# LLM search agent evaluation

Japanese search agents previously lacked a repeatable quantitative evaluation, and most Japanese benchmarks needing retrieval asked for only a single lookup. Preferred Networks built and released Snapshot BrowseComp-ja, which poses 250 Japanese questions against a frozen web corpus of about 8.24 million Japanese and 45.52 million English documents. All three public models scored low without a retrieval tool, and Qwen3.8-27B reached 89.2% on all 250 questions with one.

Japanese search agents previously lacked a repeatable quantitative evaluation, and most Japanese benchmarks needing retrieval asked for only a single lookup. Preferred Networks built and released Snapshot BrowseComp-ja, which poses 250 Japanese questions against a frozen web corpus of about 8.24 million Japanese and 45.52 million English documents. All three public models scored low without a retrieval tool, and Qwen3.8-27B reached 89.2% on all 250 questions with one.