DexDICE splits safety and performance into two demonstration sets: 10 of 10 real-robot navigation trials reach the goal with zero violations
Synopsis
Offline imitation learning usually demands demonstrations that are both safe and high-performing, or a hand-written cost function, and neither is easy to obtain in real settings. DexDICE separates the data into safe demonstrations and performant demonstrations, uses the safe set to define a safe support region in which the policy may act, and then matches the performant behavior's distribution inside that region, training without reward or cost labels. On 38 re-curated DSRL tasks it keeps group-averaged cost below the constraint and attains the highest mean return among safe methods; on a SUMMIT-XL mobile robot trained from 20 human teleoperation episodes, it reached the goal in all 10 trials with zero safety violations.
Interpretation
Offline imitation learning can now constrain safety and performance together using two demonstration sets with different jobs, without first writing a cost function or finding a single expert that is both safe and efficient. Earlier approaches either relied on explicitly specified cost functions or required demonstrations that are themselves both safe and high-performing, which are costly to collect. On the re-curated DSRL benchmark, DexDICE keeps group-averaged cost below the constraint across all three families, Safety Gymnasium, Bullet Safety Gym and MetaDrive, totaling 38 tasks, and attains the highest mean episode return among constraint-satisfying methods.
The method treats safe demonstrations as a boundary on where the policy may act, restricting location rather than how often the policy moves inside the safe region. Conventional DICE-style offline imitation learning pulls the policy toward a demonstration distribution with a divergence penalty, so a suboptimal safe expert drags the policy toward suboptimal behavior. In a tabular gridworld with exact solutions, the distribution constraint inherits suboptimal behavior as safe-demonstration quality drops, while the support constraint lets the policy align with performant demonstrations inside the safe region.
The safe boundary is estimated by a discriminator whose threshold is set from a target false-negative rate, so safety strength is tuned by one parameter. Directly estimating safe density in high-dimensional state-action space is intractable, and estimators trained only on safe demonstrations are less robust. A discriminator trained on safe versus unlabeled data supplies the threshold as a quantile at the target false-negative rate; across 11 tasks comparing support estimators, this estimator had the highest AUROC on 8.
Perspective
The demonstrated scope is the re-curated DSRL simulated benchmark and indoor obstacle-avoidance navigation on one SUMMIT-XL mobile robot, with real-robot data consisting of 10 aggressive, 10 conservative and 94 remaining episodes recorded by a single operator; the theoretical guarantee concerns the ideal support-constrained policy, while the deployed policy is extracted by weighted behavioral cloning.
The dual-expert problem needs a feasibility condition: non-trivial overlap between safe and performant demonstrations and a separable safe support, and controlled studies show degradation when overlap is extreme. The exact zero-visitation guarantee covers only the ideal policy, whereas the weighted-BC policy is a Gaussian with full support over actions; deployment-time projection onto the estimated support substantially reduces but does not eliminate this gap. The theorem's conditions are population-level and monitored through empirical diagnostics rather than certified, and results depend on dataset-construction thresholds.