Skip to main content
Back to timeline
arXivSource publication:

Knowledge tracing stops at one right-or-wrong score: DIKT splits the prediction by skill and gains 2.04 AUC points on average across three benchmarks

Synopsis

DIKT decomposes the probability that a student answers the next exercise correctly into a product of per-skill success probabilities, giving each skill a cognitive state bounded between 0 and 1 that evolves over time. Most deep knowledge-tracing models predict from implicit vectors and cannot say which skill is holding the result back. On ASSIST09, ASSIST17 and Algebra2005, DIKT's AUC exceeds every listed baseline, an average absolute gain of 2.04 percentage points.

AI-generated editorial illustration: DIKT: A Diagnostic and Interpretable Knowledge Tracing Framework for Prediction and Model-based Learning Analysis

Interpretation

A knowledge-tracing model can now return both the probability of a correct next response and skill-level diagnostic information, rather than a single overall prediction score. Most deep knowledge-tracing methods represent student state as implicit vectors, so explanations are derived passively from internal mechanisms, hard to align with specific skills, and unable to evaluate how a hypothetical state change would affect the prediction. On the three public benchmarks ASSIST09, ASSIST17 and Algebra2005, DIKT reaches AUC values of 0.8272, 0.7925 and 0.8485, above the listed BKT, DKT, SAINT, QIKT, DSN-KT, KQN and SAKT baselines, for an average absolute AUC gain of 2.04 percentage points.

Each skill carries a cognitive state that evolves over time, and the item-level success probability is the product of the skill-wise success probabilities. That pathway is built from Bayesian multi-head attention, conditional variational recurrent inference, a logistic-growth decoder and a multiplicative skill-wise prediction layer, with attention projection matrices treated as Gaussian random variables. Cognitive states are generated recursively through logistic growth and exponential decay at a skill-specific forgetting rate and are constrained to the 0 to 1 range; on representative trajectories from the three benchmark datasets the states stay inside that range and evolve smoothly, with different skills showing different evolution patterns.

The learned cognitive states also support two kinds of model-based probing: directly overwriting a selected skill's cognitive value, or perturbing effective learning duration and recomputing the subsequent trajectory. Intervention analysis reports how locally sensitive the prediction is to one skill's cognitive value, while what-if analysis measures how far the trajectory diverges after a duration perturbation propagates over time. In representative cases, Skill 31 shows a local intervention efficiency of about 0.33 while Skill 57 is close to 0.00; in what-if analysis the trajectory distances differ across the high-cognitive, low-cognitive and low-to-high transition cases, with the low-cognitive case the largest.

Perspective

The demonstrated scope is the three public knowledge-tracing benchmarks ASSIST09, ASSIST17 and Algebra2005, together with the trajectories, attributions and intervention curves inside models trained on them; intervention and what-if analysis are explicitly framed as tools for probing model behavior rather than formal causal identification. The authors propose future work that adds richer educational and contextual information, improves the visualization and practical usability of skill-level trajectories and attributions, and evaluates diagnostic value through expert assessment and real instructional applications.

Cognitive states are model-inferred probabilistic representations rather than directly observable variables; multiplicative fusion is described as a structured aggregation mechanism rather than a strict causal equation; intervention and what-if results reflect sensitivity under the learned model only, and the text gives no validity evidence for these diagnostic indicators in real instruction.