Traffic lights skip the perception pipeline: Traffic-VL reads raw camera views and cuts average queues 10.4% over three weeks at ten intersections
Synopsis
Traffic signal control has relied on structured traffic states from dedicated perception pipelines, or on converting simulator images into intermediate representations before deciding. Traffic-VL is an eight-billion-parameter vision-language model that maps synchronized multi-view images and natural-language phase descriptions directly to phase choices, trained with SynTra-V, which turns physics-based simulation into semantically audited photorealistic surveillance imagery. It matches the structured-state Traffic-R1 on five closed-loop benchmarks, reaches 82.1% phase-selection accuracy on real surveillance images, and over three weeks at ten intersections reduced average queue length by 10.4% relative to historical manual operation.
Interpretation
A signal controller can now pick the green phase directly from multi-view intersection camera images plus phase descriptions written in natural language, without a dedicated perception pipeline first collapsing the picture into lane-level traffic states. Earlier visual methods were mostly trained on simulator-native renderings and converted images into intermediate traffic representations or handed them to a separate controller, splitting visual understanding from control optimization into two stages. On five closed-loop Jinan and Hangzhou benchmarks, Traffic-VL performs comparably to the structured-state Traffic-R1, with paired equivalence tests supporting equivalence in average travel time and average waiting time; on Jinan1 its average travel time is 269.71 s, against 333.06 s for Gemini 3 Pro and 300.53 s for adapted VLMLight.
The model encodes each camera view into visual tokens and feeds them, together with natural-language text describing which turning movements each candidate phase releases, into one eight-billion-parameter model that locates those lanes in the images and compares their observed demand. Training runs in two stages: offline reinforcement learning on semantically audited SynTra-V photorealistic images with expert phase labels, then online STPO-V optimizing the policy against long-horizon traffic outcomes in closed-loop simulation, with the vision encoder frozen online. The offline stage uses roughly 5,000 photorealistic multi-view samples spanning diverse city styles and traffic densities; the online stage interacts in a CityFlow closed loop, freezing the vision encoder while updating all remaining parameters.
Perspective
The demonstrated scope covers five closed-loop Jinan and Hangzhou simulation benchmarks, twelve unseen OmniTraffic intersections, the 1,296-sample real surveillance benchmark Real-1296, and a three-week field run on ten connected arterial intersections of one traffic-management platform covering fifteen weekday evening peaks; the model issues phase-priority recommendations that the existing signal controller executes under minimum and maximum green times, pedestrian service, and yellow and all-red clearance constraints.
Each decision uses synchronized multi-view still images rather than video, so the model receives no explicit motion history at inference; the authors also note that it inherits residual perceptual errors and hallucinations from its vision-language backbone, may in rare safety-critical scenes miss an emergency vehicle or wrongly prioritize an ordinary one, and that an erroneous flow-direction message can propagate to neighboring controllers in multi-intersection coordination.