Offloading Physical AI Inference from the Robot to Edge or Cloud GPUs: A Systematic Measurement Study of Mobile Manipulation Workloads

The authors describe this as the first systematic study of robotics workloads, focused on mobile robotic manipulation with a canonical task such as "check for rubbish in the kitchen and put it in the trash," covering three core capabilities: semantic mapping and planning, navigation, and manipulation. The prevailing approach provisions a GPU onboard the robot and confines inference to it, with higher-level planning possibly in the cloud but task execution tied to the robot; this work makes the inference infrastructure itself the object of study, comparing onboard, edge, and cloud configurations. Based on evaluation of representative models across a range of compute configurations, with specific test hardware details in the technical report; differences are reported as percentages, without sample sizes or statistical tests in the main text.

Offloading inference improves task performance: some smaller GPUs could not accommodate the mobile manipulation stack; on GPUs with sufficient memory, mapping and planning slowed by up to 383% compared to an A100; navigation showed a 30% drop in timely obstacle detection; VLA models did not dramatically slow down but their accuracies dropped by 50%. It turns the engineering intuition of insufficient onboard compute into quantified performance loss, indicating that onboard GPUs limit robot abilities in dynamic spaces while offloading to an on-premise or cloud GPU boosts operations. Relative comparison against an A100 across mapping and planning, navigation, and VLA manipulation workloads; no confidence intervals or repetition counts are given in the text.

Offloading inference extends battery lifetime: replacing an onboard GPU with a Raspberry Pi-5 board and shipping all data to the offloaded GPU increased battery lifetime; larger onboard GPUs such as Jetson Thor drained robot batteries by up to 160% (or a few hours) even for larger robots. It brings power and runtime into the physical AI inference architecture discussion alongside performance, noting that onboard GPUs add power consumption, cost, and weight. Based on comparison between onboard GPU and Raspberry Pi-5 with remote inference, reported as percentages and a "few hours" magnitude.

The authors built a toolset for automatic offload: it automatically containerizes and offloads robotics workloads using declarative specifications, distributes physical AI containers with smart policies using Kubernetes across robot compute, edge GPU, and cloud, and integrates with robotic simulators, LeRobot, and ROS2; this is announced as an industry-first capability for offloaded physical AI inference for robots in the open-source, production-ready Physical AI Toolchain, including example projects for SO-101 and UR10e and a demonstration of offloading Microsoft's Rho model to a Jetson Thor GPU controlling the Mobile Aloha robot. It turns measurement findings into a reusable deployment path, letting developers orchestrate distributed inference across robots, edge, and cloud without building an offload pipeline themselves. Rests on a released open-source toolchain and example projects, and states it has been tested with many real-world use cases; the text does not give quantitative results for those use cases.

The results target the mobile robotic manipulation setting, with a canonical task such as "check for rubbish in the kitchen and put it in the trash," covering semantic mapping and planning, navigation, and manipulation, and comparing onboard, edge, and cloud GPU configurations. They apply to robot deployments that need to run larger models in dynamic spaces while extending operating time, and to developers who want to orchestrate inference across robot, edge, and cloud with Kubernetes, with SO-101 and UR10e examples provided in the toolchain. The authors explicitly note that offloading involves a complex tradeoff among performance, network latency and bandwidth, and available GPU resources, so benefits depend on the network conditions and compute supply of the deployment setting.

The text reports differences as percentages but gives no sample sizes, repetition counts, or statistical tests, and specific test hardware details point to the technical report, so the stability of these numbers still needs confirmation against that report. Offloading benefits are tightly coupled to network latency, bandwidth, and available GPU resources, and the text does not give the boundary at which benefits would reverse. The authors state they have tested it with many real-world use cases but do not list quantitative results for them. In addition, the claim that benefits are "likely to become even more pronounced" as physical AI models grow is a forward-looking expectation rather than a verified conclusion.