Ngoc Phuoc An Vo

The work presents a continuous evaluation framework for enterprise agent skills, applied to two Business Value Determination skills in a real enterprise VAR system, using independently computed live-API ground truth and template test cases with runtime-resolved placeholders to check tool selection, arguments, execution ordering, and database integrity via programmatic checks plus a narrowly scoped LLM judge; across 240 automatic trials, 175 passed all applicable final numerical checks, yet 162 of them (92.6%; Wilson 95% CI 87.7–95.6%) still had at least one evaluator-detected deviation, and under a broader seven-check final-state definition 151/164 (92.1%; 95% CI 86.9–95.3%) still violated a trajectory check, with dependency attribution compressing a mean 6.34 failed checks to 2.65 roots.

The work presents a continuous evaluation framework for enterprise agent skills, applied to two Business Value Determination skills in a real enterprise VAR system, using independently computed live-API ground truth and template test cases with runtime-resolved placeholders to check tool selection, arguments, execution ordering, and database integrity via programmatic checks plus a narrowly scoped LLM judge; across 240 automatic trials, 175 passed all applicable final numerical checks, yet 162 of them (92.6%; Wilson 95% CI 87.7–95.6%) still had at least one evaluator-detected deviation, and under a broader seven-check final-state definition 151/164 (92.1%; 95% CI 86.9–95.3%) still violated a trajectory check, with dependency attribution compressing a mean 6.34 failed checks to 2.65 roots.