RWKV fades on long horizons: alternating in attention blocks lets a 36M model match a 1B forecaster on GIFT long-horizon MASE
HART alternates Transformer and RWKV blocks so prediction positions can retrieve history directly, beating pure RWKV at medium and long horizons on GIFT-Eval and TIME-Benchmark and matching Toto-2.0-1B's long-horizon MASE on GIFT with 36M parameters.