Cloud gaming stops encoding whole frames alike: interest-aware bitrates keep key objects sharp and lift 5G QoE by at least 20%

Bitrate allocation for cloud gaming drops from whole chunks or whole frames to individual game objects, so different objects in one frame can receive different resolutions. Earlier adaptive bitrate schemes adjust over chunks lasting several seconds, giving every object in a frame the same bitrate, so bandwidth is still spent on regions the player ignores while attention sits on vehicles, obstacles and enemies. In simulations of 5G, 4G and WiFi scenarios the scheme beats comparison methods including BBA, BOLA, RobustMPC, Pensieve, DeepGame, T-Gaming and MetaABR on the proposed QoE metrics, with normalized average QoE at least 20% higher in 5G.

Key objects are inferred without extra hardware such as gaze trackers, from the player's in-game movement speed, object semantic labels, pixel area, motion speed and latent relationships among objects. Existing approaches either measure gaze with eye-tracking hardware or mark regions of interest by pre-defined rules such as colour, contrast and saliency, which cannot follow changes in the gameplay scene. On roughly 500 gameplay clips and more than 1500 images collected from GTA V, with key objects annotated by five independent annotators and aggregated across them, the scheme reaches 90.58% average key-object identification accuracy over 10 evaluation runs.

Bitrate decisions run in a non-blocking manner, staying in effect for subsequent frames while the rendering and streaming pipeline never waits for object identification or policy inference. Running segmentation and policy inference on every frame would eat into the end-to-end latency budget, which is far tighter for cloud gaming than for ordinary video streaming. On an RTX 4090 the semantic segmentation model averages about 90 ms of inference and the reinforcement learning model only 0.96 ms, both well under the 250 ms decision interval; in a fully integrated cloud gaming platform, object identification can instead read object metadata directly from the game engine's rendering pipeline.

The scheme targets wireless cloud gaming in which gameplay frames travel from a cloud server to thin clients such as smartphones and tablets, with decisions made on the server side and no extra client hardware such as gaze trackers. Evaluation covers simulated 5G, 4G and WiFi scenarios and gameplay clips from GTA V, Cyberpunk 2077, Spider-Man 2, The Witcher 3, Black Myth: Wukong and CS:GO. The scheme maps resolution decisions to a quantization parameter map through ROI interfaces for standard encoders, without changing codec standards. Future work plans evaluation on a broader range of cloud games with different visual styles and interaction dynamics, and a full interactive implementation on real-time cloud gaming platforms.

Training data comes mainly from open-world GTA V scenes, so the learned object-importance pattern may not transfer directly to genres whose key objects differ, such as sports games where attention centers on the ball, and applying it there may require additional data collection and fine-tuning. The subjective user study uses processed gameplay clips to isolate the perceptual impact of object-aware bitrate allocation and update frequency, and deployment-level validation on real-time cloud gaming systems remains to be done. Raising the object update frequency from 4 Hz to 8 Hz lifts the mean opinion score only from 4.05 to 4.17, a difference that is not statistically significant, so 4 Hz is chosen as the default setting.