When VRAM runs out, stop paging: Ember keeps cold data in host memory and beats UVM on six oversubscribed applications

Ember lets oversubscribed applications stop relying on page faults to move data, by mapping allocations into host memory by default and promoting only the regions that are genuinely hot into GPU memory. Unified virtual memory migrates on page faults by default, and its performance collapses once the working set exceeds GPU memory, which is why production systems either orchestrate placement by hand or disable it. Under oversubscription, Ember-static reaches a geometric-mean speedup over stock UVM across six applications on three PCIe GPUs, the RTX 5080, A40, and H200, with Ember-adaptive also faster.

It uses the GPU's hardware access counters to decide which regions are worth moving to the GPU, and writes cold regions back to host memory in batches before GPU memory fills. Stock UVM does not use those counters; it migrates on faults and reclaims on the critical path once memory is full, and that interleaving of eviction with execution lowers warp occupancy. On GH200, a kernel reads a 1 GiB window of an 8 GiB managed buffer in eight phases, sliding the window forward by a fixed share each phase; Ember-static matches prefetching when the hot set is static and still outperforms all baselines at full drift.

Ember targets discrete GPUs whose device memory is oversubscribed, and it preserves the UVM memory and programming model, so applications still allocate managed memory and dereference the returned pointer as before. Ember-static deploys on the stock driver through placement hints and needs no elevated privileges or bare-metal access; Ember-adaptive runs inside the UVM driver and requires hardware access counters and a machine where an administrator can load a module. It places only memory allocated through cudaMallocManaged, so buffers a library allocates internally stay outside its view.

Ember cannot directly observe that a resident region has gone cold; it infers this from the demand recorded before promotion, decayed over time, so a region that is still hot can be reclaimed by mistake and moved twice. Ember-adaptive needs bare-metal access to load a kernel module, and the module builds against the internal interfaces of an open-source driver release, so it must be rebased for each new one. On a multi-GPU node, peer memory forms a third tier, which the design does not yet cover.