Sarah Pan

Titan-Llama converts the first four layers of Llama-3.2-1B-Instruct to blockwise attention and attaches a fixed-size neural memory module to each KV head, using test-time training to compress historical key-value bindings while the backbone stays frozen. Against segmented attention windows alone, accuracy rises from 23.87% to 40.51% on QuALITY and from 14.50% to 40.75% on LongHealth. In the evaluated configurations, prefill is more than 20% faster, decode 37% faster at medium context lengths, and steady-state decode memory 24% lower.

Titan-Llama converts the first four layers of Llama-3.2-1B-Instruct to blockwise attention and attaches a fixed-size neural memory module to each KV head, using test-time training to compress historical key-value bindings while the backbone stays frozen. Against segmented attention windows alone, accuracy rises from 23.87% to 40.51% on QuALITY and from 14.50% to 40.75% on LongHealth. In the evaluated configurations, prefill is more than 20% faster, decode 37% faster at medium context lengths, and steady-state decode memory 24% lower.