# Instruction Hierarchy

Existing instruction-hierarchy training sets cover constraint-constraint and constraint-goal conflicts but not two goals that conflict with each other. Decomposing a task into goals and constraints yields 9,544 goal-goal preference pairs built from two text-processing instructions that each process the other's message, and fine-tuning Qwen3.5-9B and Gemma4-12B-it on them with direct preference optimization raises mean conflict resistance by 34.3 and 19.4 points on four instruction-hierarchy benchmarks unseen in training, with small mean utility changes across eight benchmarks.

Existing instruction-hierarchy training sets cover constraint-constraint and constraint-goal conflicts but not two goals that conflict with each other. Decomposing a task into goals and constraints yields 9,544 goal-goal preference pairs built from two text-processing instructions that each process the other's message, and fine-tuning Qwen3.5-9B and Gemma4-12B-it on them with direct preference optimization raises mean conflict resistance by 34.3 and 19.4 points on four instruction-hierarchy benchmarks unseen in training, with small mean utility changes across eight benchmarks.