Samsung Advanced Institute of Technology (South Korea)

The authors present ServeTwin, a closed-loop simulator that couples specification-driven analytical timing with a stateful serving loop, uses iSTAGE to derive per-iteration traces from model and engine specifications while avoiding target-hardware operator profiling, and implements a vLLM-compatible interface that runs unmodified serving benchmarks; against real deployments it reproduces InferenceX's steady-state throughput-interactivity frontier with a 3.6% mean error and predicts LMBenchmark's multi-turn performance with a 9.9% error while tracking KV-cache evolution, and pre-silicon sweeps show that the preferred HBM bandwidth-capacity tradeoff reverses across workload states and that increasing concurrency shifts the bottleneck from memory bandwidth to scheduler and runtime overheads.

The authors present ServeTwin, a closed-loop simulator that couples specification-driven analytical timing with a stateful serving loop, uses iSTAGE to derive per-iteration traces from model and engine specifications while avoiding target-hardware operator profiling, and implements a vLLM-compatible interface that runs unmodified serving benchmarks; against real deployments it reproduces InferenceX's steady-state throughput-interactivity frontier with a 3.6% mean error and predicts LMBenchmark's multi-turn performance with a 9.9% error while tracking KV-cache evolution, and pre-silicon sweeps show that the preferred HBM bandwidth-capacity tradeoff reverses across workload states and that increasing concurrency shifts the bottleneck from memory bandwidth to scheduler and runtime overheads.