Agent traces stop needing tags: describe a behavior in one sentence, and a decision model reads each trajectory and streams back the matches

Agent traces can now be searched by behavior rather than by tags, metadata, or full-text keywords. Such questions did not map cleanly to a single trace attribute and could not be answered by full-text search, so teams either set up an evaluator ahead of time or went through traces manually. In LangSmith, a user describes the behavior in plain language and the system returns the traces where it happened, and the search can run alongside existing filters such as errored traces or a specific tag.

Retrieval is done by a fast decision model that reads the content of each trace, rather than by compressing a whole trace into one vector. Vector search packs a whole trace into one embedding, and that compression is lossy, so a trace where the agent did the work successfully and one where it only said it did can end up with nearly the same embedding. A search first narrows candidates with structured filters, then the decision model checks each remaining trajectory against the description, many in parallel, and matches are streamed back as they are found.

Traces are normalized at ingest into a session-level trajectory that lists each message, tool call, and tool result once. Traces arrive from the OpenAI and Anthropic SDKs, LangChain, LangGraph, the Vercel AI SDK, OpenTelemetry, and more, each recording messages a little differently, and agents often re-send the conversation history with every LLM call. SmithDB builds the trajectory as traces arrive and puts every supported format into the same shape, so one search works across agents built on different SDKs and repeat messages no longer bloat the context.

The feature is aimed at teams that use LangSmith to trace agent runs, for debugging, mining eval datasets, filtering annotation queues, and product analysis. AI Search is in public beta on LangSmith Cloud in the GCP US, EU, and APAC regions at no additional cost; org admins can turn it on in settings, after which it shows up on the tracing page.

The hardest conversations to classify are those with subtle differences, such as telling whether the user gave up or got what they needed and left, which can come down to a couple of turns in a long trajectory; the team wants to see whether a model tuned on traces tells them apart more reliably. The fixed token budget works well when the answer is found in what the user and agent said, but other questions need different evidence, such as an agent ignoring a tool error that only shows up in the tool results, and the team is exploring question-based context selection and signals that show when the model is unsure so it can take a second pass with more context.