Jetson DLA inference moves into Python: cuda-bindings 13.3.0 exposes cuDLA so GPU and DLA stages queue on one stream
Synopsis
NVIDIA released the cuda.bindings.cudla module in cuda-bindings 13.3.0, a thin Cython-generated Python layer over the cuDLA C++ runtime that lets Jetson AGX Orin applications create DLA devices, load networks, register buffers and submit inference without writing C++. The module targets hybrid mode, where DLA tasks are enqueued on a CUDA stream alongside GPU preprocessing and postprocessing, and the same API applies on RTX Spark. The release notes that standalone mode, which needs NvSci buffers and sync primitives, is not supported in this version.
Interpretation
Edge AI applications often need several neural networks within limited compute and power budgets, and the DLA on Jetson AGX Orin is dedicated hardware for deep learning inference that can offload work from the GPU, improving power efficiency and leaving GPU capacity for other tasks. Managing DLA execution previously meant working through the cuDLA C++ runtime API, with the application handling devices, networks and buffers in C++. cuda.bindings.cudla ships in cuda-bindings 13.3.0 as a thin Python layer over the cuDLA C++ API, exposing thirteen functions, nine data types and eight enums, while inference still executes on DLA hardware.
The workflow starts by creating a DLA device, loads a compiled network, allocates and registers buffers according to the tensor descriptors the module reports, then submits the inference task onto a CUDA stream. cudla.Mode.CUDA_DLA selects hybrid mode, so preprocessing kernels, DLA inference and postprocessing kernels can be enqueued on the same stream and executed in order. The example runs a ResNet-50 backbone compiled as a DLA loadable, and its output lists one input tensor, resnetv17_pool0_fwd, and one output tensor, resnetv17_stage4_activation2, with shapes (1,64,56,56) and (1,2048,7,7).
Buffer sizes are read from queried tensor descriptors rather than hardcoded, so the same code paths handle any loadable and only preprocessing and postprocessing change with the model. The loadable is passed as a Python bytes object directly to module_load_from_memory, with no file path or filesystem handle. On the build side, trtexec with --buildDLAStandalone produces a raw DLA loadable that needs no TensorRT at inference time, and the example targets TensorRT 11.4.1 with CUDA 13.4 Update 1 on Jetson AGX Orin running JetPack 7.2.2 or later.
Perspective
The demonstrated scope is an image classification example in a JetPack environment on Jetson AGX Orin, covering the full lifecycle from compiling a loadable to reading the output; the same API holds on RTX Spark, so the cuDLA interface portion applies there directly.
This release does not support standalone mode, and requesting it raises ErrorUnsupportedOperation; not every network layer is DLA-compatible, so preprocessing the ONNX graph to trim incompatible head and tail layers is often necessary; TensorRT 11 derives precision from the model's tensor types, and --fp16 and --int8 are no longer supported.