Language models keep giving the same answer: search embedding gaps to cut collisions by up to 12.8%
Synopsis
Language models often return semantically similar answers to open-ended questions even when many plausible alternatives exist. Gap Detection searches a shared text embedding space for relevant but unoccupied directions, scoring each by the product of query relevance (Demand) and dissimilarity to the nearest existing item (Defensibility), then mapping directions to concept words and text. On five curated topics and 90 INFINITY-CHAT queries, reported mean relative reductions in output Collision are 12.8% and 10.8%; in iterative synthetic-data augmentation with 100 training examples per method, Collision falls from 0.324 to 0.298 and the Vendi Score rises from 27.16 to 29.20, while accuracy differs by one example on a 40-example test set.
Interpretation
Answers to open-ended questions are no longer picked only from what already exists; the system first looks for relevant directions in embedding space that no existing response occupies. Prior practice selected or rewrote among existing responses rather than explicitly searching for uncovered directions. On five curated topics and 90 INFINITY-CHAT queries, reported mean relative reductions in output Collision are 12.8% and 10.8%.
A direction's quality is the product of two quantities: relevance to the query and dissimilarity from the nearest existing item. GapScore combines Demand and Defensibility into one product objective optimized on the unit sphere. Candidate directions are then mapped to concept words by a language model and turned into text through task-specific filtering and constrained prompting; component ablations expose a trade-off between semantic separation and relevance.
The same search also serves generative engine optimization and iterative synthetic-data augmentation. In iterative synthetic-data augmentation with 100 training examples per method, Collision falls from 0.324 to 0.298 relative to undirected accumulation. In the same setting the Vendi Score rises from 27.16 to 29.20, while the accuracy difference is one example on a 40-example test set.
Perspective
The report covers five curated topics, 90 INFINITY-CHAT queries, a generative engine optimization instantiation over four product markets, and an iterative synthetic-data augmentation setting with 100 training examples per method. The intended reader is a researcher or engineer dealing with homogenization in open-ended generation, using it to understand the explicit gap-search mechanism and its reported scope.
Component ablations expose a trade-off between semantic separation and relevance, and better direction scores do not uniformly yield better downstream metrics. The report stresses that task validity must be checked separately from geometric novelty, and in the synthetic-data augmentation setting the accuracy difference is one example.