Guanqun Yang

PatchHolmes is a two-phase patch retrieval system whose first stage fuses BM25 with time decay and Qwen3-Embedding-8B dense retrieval via RRF into a top-100 candidate set, and whose second stage runs a frozen open-weight LLM agent that views all 100 candidates listwise through four tools (list_candidates, read_commit, read_file_diff, submit_answer) and submits a single best commit; on 809 CVEs from GitHubAD it reaches 59.95% Recall@1, 25.34% above the pointwise classifier Favia and 31.40% above IRCoT, adds 27.32% Recall@1 over the retriever's top pick on identical candidates, and transfers unchanged to PatchFinder_top10 to lift Recall@1 from 24.28% to 39.86%.

PatchHolmes is a two-phase patch retrieval system whose first stage fuses BM25 with time decay and Qwen3-Embedding-8B dense retrieval via RRF into a top-100 candidate set, and whose second stage runs a frozen open-weight LLM agent that views all 100 candidates listwise through four tools (list_candidates, read_commit, read_file_diff, submit_answer) and submits a single best commit; on 809 CVEs from GitHubAD it reaches 59.95% Recall@1, 25.34% above the pointwise classifier Favia and 31.40% above IRCoT, adds 27.32% Recall@1 over the retriever's top pick on identical candidates, and transfers unchanged to PatchFinder_top10 to lift Recall@1 from 24.28% to 39.86%.

The work presents SkillSeek, an open-source two-stage skill retriever (a BGE-base bi-encoder feeding a small cross-encoder, exposed over MCP), and across a grid of two skill pools and two backbones on the 89-task SkillsBench benchmark observes that plain bm25 alone records a pass rate at or above Liu et al.'s LLM-mediated loop on three of four settings, with a small cross-encoder covering the remaining difference on the fourth (34K pool with Qwen3.5) at the same 0.442, while total per-trial spend drops from USD 51.30 to USD 27.54 (within fifty cents of the USD 27.41 no-skill baseline), a pattern the authors attribute to a first-stage recall ceiling.

The work presents SkillSeek, an open-source two-stage skill retriever (a BGE-base bi-encoder feeding a small cross-encoder, exposed over MCP), and across a grid of two skill pools and two backbones on the 89-task SkillsBench benchmark observes that plain bm25 alone records a pass rate at or above Liu et al.'s LLM-mediated loop on three of four settings, with a small cross-encoder covering the remaining difference on the fourth (34K pool with Qwen3.5) at the same 0.442, while total per-trial spend drops from USD 51.30 to USD 27.54 (within fifty cents of the USD 27.41 no-skill baseline), a pattern the authors attribute to a first-stage recall ceiling.