# Autoregressive diffusion

Real-time video-to-audio has been held back by two bottlenecks: visual frontends shrink their receptive fields to cut look-ahead, and causal diffusion backends denoise each patch in isolation, so both alignment and audio quality trail offline models. LiveFoley uses a trailing sliding window to turn off-the-shelf non-causal segment-level encoders into streaming modules without retraining, cutting frontend look-ahead to 0.08 s, and widens the backend denoising window toward history. On the VGGSound test set, LiveFoley-48k reaches a DeSync of 0.48, on par with offline MMAudio-L at 0.47, with 156 ms end-to-end latency on a single A800 GPU.

Real-time video-to-audio has been held back by two bottlenecks: visual frontends shrink their receptive fields to cut look-ahead, and causal diffusion backends denoise each patch in isolation, so both alignment and audio quality trail offline models. LiveFoley uses a trailing sliding window to turn off-the-shelf non-causal segment-level encoders into streaming modules without retraining, cutting frontend look-ahead to 0.08 s, and widens the backend denoising window toward history. On the VGGSound test set, LiveFoley-48k reaches a DeSync of 0.48, on par with offline MMAudio-L at 0.47, with 156 ms end-to-end latency on a single A800 GPU.

Human motion generation has mostly been steered by naming an action or a place to move; StepControl instead takes only a future left/right foot-contact schedule, with no target root or body poses and no semantic prompt. It treats contacts as explicit temporal events, injects them through dedicated cross-attention with classifier-free guidance, and adds a stochastic mixture noise schedule and warmup for stable long-horizon autoregressive generation. Experiments report that it follows prescribed contact timing more accurately than adapting existing spatially guided generators to contact constraints, keeps contact control over long continuous rollouts, and responds to edited schedules such as cadence changes, stop-and-go pauses, limping gaits and bilateral jumping.

Human motion generation has mostly been steered by naming an action or a place to move; StepControl instead takes only a future left/right foot-contact schedule, with no target root or body poses and no semantic prompt. It treats contacts as explicit temporal events, injects them through dedicated cross-attention with classifier-free guidance, and adds a stochastic mixture noise schedule and warmup for stable long-horizon autoregressive generation. Experiments report that it follows prescribed contact timing more accurately than adapting existing spatially guided generators to contact constraints, keeps contact control over long continuous rollouts, and responds to edited schedules such as cadence changes, stop-and-go pauses, limping gaits and bilateral jumping.