NVIDIA used an AI coding agent to migrate a Depth Anything 3 ROS 2 node to the CUDA buffer backend, letting depth images move between nodes with zero-copy transport

NVIDIA contributed the CUDA buffer backend for ROS 2 Lyrical, implementing rosidl::Buffer storage with CUDA Virtual Memory Management (VMM) so co-located nodes can exchange GPU-resident payloads without serialization or host copies when runtime conditions allow. Previously, GPU-accelerated ROS 2 graphs could still serialize messages or copy them through CPU memory at node boundaries, eroding the benefit of keeping perception and AI workloads on the GPU; rosidl::Buffer moves memory sharing and data-lifetime management behind a standard ROS 2 field, letting platform vendors support externally managed storage without defining a separate ROS message type. The text states the backend was contributed by NVIDIA to ROS 2 Lyrical and lists the runtime requirements for the optimized path: the same host, CUDA device, and Linux user, plus a supported RMW implementation such as rmw_fastrtps_cpp and rmw_zenoh_cpp; otherwise ROS 2 automatically falls back to the CPU path compatible with any existing ROS 2 nodes.

Using the Depth Anything 3 TensorRT node as the example, the migrated ROS transport change stays small: one subscription option, one CUDA allocation, two stream-aware handle extractions, and one publish, with no custom message, duplicate CUDA topic, or CPU/CUDA publisher branch. The original node's callback converted the incoming ROS image to an OpenCV view, ran monocular metric-depth inference with TensorRT, and converted the resulting cv::Mat back to a ROS image, leaving a CPU boundary around a GPU-native algorithm; the migration keeps sensor_msgs/msg/Image and the existing image_transport and message_filters topology, only letting the output Image.data carry CUDA buffer-backed storage. The text shows the essential code: allocate_buffer() gives Image.data CUDA buffer-backed storage, from_input_buffer() supplies a handle safe to consume read-only on the TensorRT stream, and from_output_buffer() supplies a write handle so the postprocess writes its final 32FC1 result directly into the outgoing message's buffer, avoiding a device-to-host copy and an intermediate device-to-device output; the inner scope releases the write handle before publishing to record a write CUDA event.

The migration skill turns the audit into a repeatable agent workflow rather than replacing the node with a template or rewriting code automatically, and it requires independently verifying semantics, backend negotiation, separate-process transport, buffer lifetime, and actual memory-copy behavior. Identifying the correct boundaries to update requires a careful audit of allocations, serialization, stream ownership, and fallback behavior; the agent is directed to record the starting revision, target ROS environment, and existing local changes, confirm generated message field type compatibility and add cuda_buffer and cuda_buffer_backend dependencies, trace each message field from receipt to publication, run the read-only copy-boundary audit, make a per-field migration plan, and implement the smallest interface-preserving patch. The text lists seven steps for the skill and notes it also contains a verification step that produces custom source and sink nodes, creating two pipelines to test the same migrated node under both CPU and GPU setups without code changes.

Validation includes inspecting GPU activity and memory transfers with NVIDIA Nsight Systems and confirming backend negotiation on the subscriber side via msg->data.get_backend_type() reporting "cuda". This gives an actionable criterion for whether the CUDA transport path is actually enabled: on an eligible CUDA path, the migrated node should not show payload-sized host-to-device or device-to-host transfers at its ROS boundary, and comparable latency measurements should be recorded before and after the change. The text provides subscriber-side example code where get_backend_type() should report "cuda" when both endpoints meet the CUDA backend requirements, and notes production code will often accept CPU fallback without throwing the error; from_input_buffer() automatically handles CPU fallback internally, so the callback need not distinguish CPU and GPU paths.

The result targets GPU-accelerated robotics developers on ROS 2 Lyrical and above with supported RMW implementations such as rmw_fastrtps_cpp and rmw_zenoh_cpp, especially those with existing CUDA-accelerated nodes whose messages contain variable-length primitive fields. The optimized path requires publisher and subscriber on the same host, CUDA device, and Linux user; otherwise ROS 2 automatically falls back to the CPU path. The migration keeps core functions and boundary message types the same and only adds cuda_buffer and cuda_buffer_backend dependencies, so the build and setup process stays similar to the original node, making the workflow reusable for other CUDA-accelerated ROS 2 nodes and deployable on NVIDIA Jetson AGX Thor.

The text gives no concrete before-and-after latency or throughput numbers and only recommends recording comparable latency measurements, so the actual size of the gain still needs to be measured by the reader. Point-cloud construction and debug visualization remain local CPU consumers that may still require a device-to-host copy and synchronization when enabled, and the migration leaves them as explicit optional boundaries. The subscriber-side validation example throws when CUDA is not negotiated, while production code will often accept CPU fallback, a difference to watch when reusing the test code. The piece is also tutorial in nature and provides no independent third-party reproduction or cross-hardware comparison data.