Tool outputs no longer have to be text: Coca-Cola compresses search results into latent tokens, cutting a web-search agent's token use by more than 70%
Synopsis
Each time a web-search agent calls a tool, the long returned text is appended verbatim to its context, so after several steps the context balloons and dilutes attention. Coca-Cola instead compresses those observations into a few continuous latent tokens and embeds them directly in the agent's input embedding space, with action-preserving training aligning the next-action distributions under full and compressed observations. Across four QA benchmarks and two model families, total token consumption falls by 70.01%–89.24% and end-to-end runtime by 24.45%–55.97%, while EM and F1 exceed the uncompressed agent in every setting.
Interpretation
Tool-returned search results no longer enter the agent's context as text tokens; they are compressed into a few continuous latent tokens inserted directly into the input embedding sequence. Earlier token pruning and text summarization kept the compressed result as a discrete text sequence, and the need for semantic coherence inherently limited the achievable compression ratio. On NQ with the Qwen models, average input tokens drop from 14,362.12 to 1,766.53, an 87.70% reduction; on 2Wiki with the GLM models they drop from 31,693.47 to 3,003.38, a 90.52% reduction.
The compressor pairs a context-aware encoder, which reads the task instruction, the current-step reasoning trajectory and the tool observation together, with a latent adapter that maps the encoding into a fixed number of latent tokens at the target agent's embedding dimension. Latent tokens and the target agent live in different representation spaces and cannot be used directly, so a continuous interface is needed to connect them. At inference each tool observation is compressed into 128 continuous latent tokens inserted between special boundary markers; training updates only the encoder's LoRA parameters and the adapter parameters while the target agent stays frozen.
Action-preserving training makes the latent tokens retain decision-critical information rather than reconstruct the original observation text. Compared with ICAE, which shares the same architecture but uses a text-reconstruction objective, Coca-Cola reaches a higher compression rate in all eight model–dataset settings. On 2Wiki with the GLM models the compression rate is 89.24% versus 74.12% for ICAE; on HotpotQA with the Qwen models F1 is 33.75% versus 31.21% for ICAE.
Perspective
The demonstrated scope is multi-step question answering by a web-search agent on four open-domain QA benchmarks, NQ, TriviaQA, 2Wiki and HotpotQA, with two model pairs, Qwen3.5-35B-A3B with Qwen3.5-4B and GLM-4.7-Flash with GLM-Z1-9B, retrieving up to 5 results per tool call and allowing at most 5 tool calls per task.
The information-capacity analysis of latent tokens rests on assumptions of finite precision, a fixed codebook and realizable coordinate combinations, and usable information still depends on the compressor's encoding ability and the agent's ability to use the representations. The latent-token ablation shows F1 rising with token count, and the default of 128 is a trade-off between efficiency and performance; removing the reasoning state or the task goal lowers EM, indicating decision-critical information cannot be determined from the observation alone.