Researchers propose a five-criterion trajectory-level framework for agentic cognitive depth, arguing profiles should accompany end-to-end scores

It turns the application-focused four components of agentic LLM systems (Planning, Memory, Tools, Control Flow) into a trajectory-level measurable profile with five criteria: context sensitivity, temporal continuity, multimodal coordination, adaptive interaction, and metacognitive monitoring. Single-call benchmarks (MMLU, HELM, BIG-bench, GPQA) aggregate task scores and agentic benchmarks (GAIA, SWE-bench, AgentBench, WebArena) measure only end-to-end completion, leaving the failing part of the agentic loop unclear; this work makes the full trajectory the object of measurement and supplies per-criterion diagnostics. A conceptual and measurement-framework paper reporting no new experiments; the argument rests on comparison with existing frameworks and benchmarks and on published results such as SWE-bench Pro scoring below SWE-bench Verified and TRIP-Bench reporting at most fifty percent success on its easy split and below ten percent on hard subsets.

It adds metacognitive monitoring as a separate fifth criterion, a layer above the object-level loop that monitors and regulates the whole run, and notes current agentic frameworks usually lack it. The first four criteria measure how well existing components are used; the fifth measures whether the system monitors and regulates the full trajectory. Self-reflection and retry loops come closest but usually revise a local answer and leave the full run unregulated. It cites MIRROR, which evaluates four levels of metacognitive behaviour across sixteen models from eight labs and finds compositional self-prediction fails across models and self-knowledge often fails to transfer or update from feedback, alongside work showing self-reported confidence correlates with correctness and models can partly monitor internal activations, indicating monitoring signals exist while control remains weak.

It links the five criteria to the agent's world model, the maintained representation of task, environment, tools, and agent state, and lists seven components with externalised forms. Current agentic systems spread world-model components across prompts, weights, vector stores, tools, and controller code, making them hard to inspect and update; the work proposes externalising domain knowledge, tool/action models, environment dynamics, self-model, other-agent model, task model, and causal model into knowledge graphs, PDDL specifications, state machines, capability registries, explicit opponent models, maintained goal specifications, and structural causal graphs. A design recommendation and literature synthesis, supported by work on explicit world models, precondition and effect prediction, SimuRA's planning by simulating candidate actions against an LLM-based world model, and work linking planning failures to the lack of an explicit problem representation.

It offers three complementary measurement modes—perturbation, deployment-time dynamic observation, and mechanistic interpretability—and defines "capability overstatement": when an aggregate score passes a benchmark threshold while a criterion score falls below its threshold, the score reflects an artefact more than the underlying capability. The perturbation method extends NLP behavioural testing to the trajectory level with perturbations grouped by criterion; dynamic observation captures commitment drift over weeks and deployment-scale calibration; interpretability uses probing classifiers and circuit analysis on internal states. The modes are paired per criterion and can extend existing benchmarks at a small constant multiple of one end-to-end evaluation. A methodological proposal without measured profiles; the cost argument rests on running one base task once per perturbation, dynamic observation adding little cost once telemetry exists, and interpretability being applied selectively.

The framework targets agentic systems whose full trajectory can be observed, adding per-criterion diagnostics on top of existing benchmarks for agent evaluation and architecture design. It is meant to apply when tasks are held out, runs are repeated, and trajectory logs distinguish changes from the model, scaffold, interface, controller, memory, or tool layer. The authors' suggested next experiment applies the five perturbation classes of Table 1 to frontier agents on held-out samples from GAIA, SWE-bench Pro, and ARC-AGI-3, reporting each profile beside the end-to-end score; ARC-AGI-3 is named a suitable testbed because its environments already demand the behaviours the criteria measure. On the design side, symbolic verifiers, structured memory, planner coupling, ontology-constrained tool routing, and difficulty-aware control are listed as neurosymbolic patterns for building and testing these capacities.

The proxies, profile hypotheses, and the worked trajectory in Section 6 are illustrative and await empirical test; the five criteria are measurable but joint sufficiency remains open, and they overlap, since one trajectory failure can lower several scores when memory, planning, and control flow fail together, so root-cause analysis still needs step-level evidence. Scoring details—thresholds, task selection, reliability checks, cost reporting, and aggregation—need further specification. The architectural mapping is abstract because real agents mix prompts, retrieval, tools, controller code, memory stores, and human input. The central hypothesis—that agents with similar aggregate scores separate on the profile and that those differences predict deployment failures—is falsifiable, but a direct test would need matched agents, held-out perturbation sets, and repeated runs. The text is complete, but some symbolic markers in the tables are lost in plain text, so the exact per-criterion coverage symbols in the matrix should be checked against the original tables.