NVIDIA VSS Blueprint 3.3 builds a visual AI agent from one prompt in under 30 minutes and cuts VLM input tokens by 80%

The Build Vision Agent skill composes multiple workflows such as alerting, search, and summarization into one deployment plan and reuses shared infrastructure including VIOS, Kafka, Redis, Elasticsearch, HAProxy ingress, and MCP services. Earlier VSS skills handled individual operations such as deployment, camera setup, summarization, search, alerts, and analytics; 3.3 organizes them as deployment skills, operation skills, tools, and benchmarks, with vss-build-vision-ai composing the rest. The text lists four validated developer profiles (base, alerts, lvs, search) with their capabilities and states that the skill starts from the closest profile, adds or removes only exact service keys, and converges shared roles onto one instance; on a two-GPU RTX PRO 6000 Blackwell host, combining search and alerts added only an alert bridge and real-time VLM, FP8 Cosmos 3 Nano shared the detector's GPU, and a recorded alerts build reached a live, previewable deployment in under 30 minutes.

Adaptive EVS lowers runtime VLM cost through dynamic pruning and event-aware batching: each patch is compared with the prior frame using cosine similarity and unchanged patches are dropped before reaching the language model; clips above about 70% token retention are batched as events, those below about 30% are dropped or flushed, and the rest run normally. EVS already ships in vLLM and the Cosmos NIM microservices at a fixed pruning rate; the adaptive version in 3.3 is integrated into the real-time VLM microservice and decides which tokens to keep per patch and per frame. Measurements on an RTX PRO 6000 Blackwell running Cosmos 3 Super FP8 show alert contextualization latency down 17% (1,021 ms to 844 ms), concurrent real-time VLM streams up 46% (13 to 19), and a 60-minute video summarized in about half the time with 80% fewer VLM input tokens; the text also notes results vary with scene motion, chunk length, and similarity threshold, and recommends benchmarking representative footage first.

The deployment output is a self-contained build: _builds/<name>/override.env (the Foundation, the effective Compose profiles, and only the settings changed), compose.yml, and resolved.yml, one flattened Compose file produced with docker compose config that deploys on its own, without modifying the repository's deploy/docker/ tree. The skill shows an architecture diagram for review before anything is written or deployed, then runs validation, deployment, and readiness checks; it also asks whether to deploy an agent harness, defaulting to NemoClaw, a host-side sandbox with the VSS skills installed, while answering no produces a headless stack driven by the VSS CLI. The text lists these artifacts and steps, and states that extending a running deployment uses a smaller delta that reuses existing services.

The approach targets three recurring cost drivers of multi-workflow production deployments: development cost (connecting microservices, shared services, model endpoints, environment variables, and APIs without duplicating infrastructure), operating cost (every stream, frame window, prompt, and visual token can increase GPU usage, queueing delay, and end-to-end summarization latency), and change cost (moving from proof of concept to production, adding capabilities, and keeping configuration, documentation, and operations aligned). 3.3 reduces cost on both the development side and the runtime side rather than optimizing only one. The text illustrates the value of workflow composition with smart city and warehouse scenarios and details the three cost drivers.

The approach targets development teams deploying real-time and recorded video agents on their own or isolated networks, for scenarios that combine alerting, search, summarization, reporting, and Q&A, such as bottling lines, warehouses, and smart cities. Adaptive EVS is most useful when VLMs read many frames and produce short responses, as in dense captioning, long-video summarization, and alert verification; it runs inside the RT-VLM container rather than against remote endpoints, is optional, and is enabled and tuned in override.env via VIA_EVS_SESSION, VLM_VIDEO_PRUNING_RATE, and VLLM_EVS_SIMILARITY_THRESHOLD. The text recommends deploying on a trusted, isolated network with authentication, TLS, rate limiting, and external controls.

The text explicitly states that Adaptive EVS results vary with scene motion, chunk length, and similarity threshold and recommends benchmarking representative footage before choosing production defaults, so the 80% token reduction, 46% concurrency gain, and 17% latency drop should be read as measurements under a specific hardware and model configuration rather than general guarantees. The roughly 70% and roughly 30% thresholds in event-aware batching are stated as approximate and are scene-dependent. In addition, this is a blueprint release post without independent third-party reproduction, cross-hardware comparison, or a full evaluation of accuracy loss, which remain open questions a reader would still need to verify before adoption.