Decision Titan adds test-time training layers to a Decision Transformer and learns long-term dependencies 20x longer than its context window in X-Maze
The work augments a Decision Transformer with Test-Time Training (TTT) layers, forming Decision Titan, and analyzes its performance and memory mechanism in the X-Maze environment built to test sequential memory: the model learns long-term dependencies with ranges 20x longer than the context window and generalizes to lengths 1.7x the training data, but temporal generalization depends on the time embeddings used and the ability to learn long-term dependencies depends on how the relevant information is encoded.