Alignment without per-case preference labels: triadic viability alignment holds task performance in a controlled testbed but trails risk-sensitive control and multi-step planning

Alignment training can drop per-case preference labels and instead write three roles, the acting subject, affected others and shared conditions, together with their hard floors into the decision procedure, excluding floor-breaching actions before task benefit is compared. Earlier preference-alignment methods, including reinforcement learning from human feedback, reward modeling and direct preference optimization, all depend on demonstrations or comparisons that mark which output is preferable case by case; such labels have been linked to sycophancy, proxy overoptimization, surface-feature bias and compression of value disagreement. Across 21,600 episodes in the Triadic Commons Environment, zero-label TVA reached 96.94% task success, 3.06 points below a preference-supervised baseline built from 4,000 simulated preference states and 36,000 action rows, with an interval of −3.82 to −2.36 points that stays inside the −10-point non-inferiority margin; joint viability rose from 0.00% to 83.68%.

The core structure is a feasibility filter at decision time, followed by scoring only within the feasible set on task gain, changes in the three role states, change in damage and uncertainty cost. A continuous-penalty variant, B5, receives the same role representation and the same four floors but replaces the feasibility gate with a weighted penalty, testing whether the result comes merely from being given more normative information. B5 reached only 16.88% joint viability, 66.81 points below TVA with an interval of 64.65 to 68.82; in the ablations, removing shared conditions cut joint viability by 51.94 points and removing hard floors by 83.54 points, the two largest components.

The method is not expensive to run: on the same hardware within the same Python reference implementation, TVA took 21.65 s end to end, against 65.93 s for risk-sensitive control and 120.34 s for multi-step model-predictive control. Risk-sensitive control performed 1,244,160 proxy evaluations per step and multi-step model-predictive control 3,839,175, while TVA performed 148,329 with at most 36 floor comparisons. The comparison covers only wall-clock time and evaluation counts inside this reference run, peak-memory differences were small, and the authors state explicitly that it is not a claim about training cost, deployment latency or energy.

The conclusions rest on the Triadic Commons Environment, a controlled discrete-time multi-agent testbed in which roles, floors and scenario dynamics are researcher-defined and the evaluator sees outcomes that the acting policy only partially observes; the authors leave external validity to a later open-world systems study.

The authors report that TVA trails risk-sensitive control and multi-step model-predictive control by 10.42 and 8.68 points respectively, and name missing multi-step planning as a main failure source; structured data selection passed the equivalence test only at 60%, later than random selection at 40%, so the minimum-sufficient-data hypothesis was rejected. Because the environment did not independently label every state that should trigger escalation, Open-Gap trigger precision and recall cannot be reported.