# Trajectory evaluation

The work proposes a dual-process-inspired hybrid framework in which a TransFuser++ neural planner generates human-like trajectories in routine driving while a meta-cognitive component, using risk fields shaped by knowledge-graph reasoning, decides when to switch to a model predictive control planner; tested in CARLA on out-of-distribution scenarios of vehicles (including an ambulance) running a red light, it reduced vehicle-to-vehicle collisions from 48 to 5 out of 160 simulations, an 89% reduction.

The work proposes a dual-process-inspired hybrid framework in which a TransFuser++ neural planner generates human-like trajectories in routine driving while a meta-cognitive component, using risk fields shaped by knowledge-graph reasoning, decides when to switch to a model predictive control planner; tested in CARLA on out-of-distribution scenarios of vehicles (including an ambulance) running a red light, it reduced vehicle-to-vehicle collisions from 48 to 5 out of 160 simulations, an 89% reduction.

The work presents LiteTrajEval, a lightweight architecture for budget-bounded trajectory evaluation that derives compact domain-specific rule profiles offline, then preprocesses each trajectory online, marks heuristic failure signals, serializes it under a fixed global budget, and invokes a single rubric-guided LLM judge to produce structured diagnostic reports; on public Magentic-One-style and tau-bench-style trajectory datasets it improves failure-localization alignment with human annotations by roughly 20–35 percentage points on Magentic-One and up to 23 percentage points on tau-retail compared with AgentRx, while reducing cost by about 6x and evaluation time by more than 8x, and it has also been deployed in the authors' enterprise agentic platform.

The work presents LiteTrajEval, a lightweight architecture for budget-bounded trajectory evaluation that derives compact domain-specific rule profiles offline, then preprocesses each trajectory online, marks heuristic failure signals, serializes it under a fixed global budget, and invokes a single rubric-guided LLM judge to produce structured diagnostic reports; on public Magentic-One-style and tau-bench-style trajectory datasets it improves failure-localization alignment with human annotations by roughly 20–35 percentage points on Magentic-One and up to 23 percentage points on tau-retail compared with AgentRx, while reducing cost by about 6x and evaluation time by more than 8x, and it has also been deployed in the authors' enterprise agentic platform.

The work presents a continuous evaluation framework for enterprise agent skills, applied to two Business Value Determination skills in a real enterprise VAR system, using independently computed live-API ground truth and template test cases with runtime-resolved placeholders to check tool selection, arguments, execution ordering, and database integrity via programmatic checks plus a narrowly scoped LLM judge; across 240 automatic trials, 175 passed all applicable final numerical checks, yet 162 of them (92.6%; Wilson 95% CI 87.7–95.6%) still had at least one evaluator-detected deviation, and under a broader seven-check final-state definition 151/164 (92.1%; 95% CI 86.9–95.3%) still violated a trajectory check, with dependency attribution compressing a mean 6.34 failed checks to 2.65 roots.

The work presents a continuous evaluation framework for enterprise agent skills, applied to two Business Value Determination skills in a real enterprise VAR system, using independently computed live-API ground truth and template test cases with runtime-resolved placeholders to check tool selection, arguments, execution ordering, and database integrity via programmatic checks plus a narrowly scoped LLM judge; across 240 automatic trials, 175 passed all applicable final numerical checks, yet 162 of them (92.6%; Wilson 95% CI 87.7–95.6%) still had at least one evaluator-detected deviation, and under a broader seven-check final-state definition 151/164 (92.1%; 95% CI 86.9–95.3%) still violated a trajectory check, with dependency attribution compressing a mean 6.34 failed checks to 2.65 roots.