Public articles linked to the same research event.
arXiv The work proposes using the variance of specialists' task vectors across specialists as a measure of interference, yielding a pre-merge score that predicts collapse, and designs PRISM: an operator that averages task vectors first and then soft-thresholds each layer at a level set by that layer's interference; across twenty-two merge configurations from four model families only destructive merges exceed the score threshold, twelve of fourteen pre-evaluation predictions were correct, and PRISM keeps all five destructive merges above the threshold within evaluation noise of the base model without data or tuning, where plain averaging falls at least 14.4 points below it or collapses entirely.
The work proposes using the variance of specialists' task vectors across specialists as a measure of interference, yielding a pre-merge score that predicts collapse, and designs PRISM: an operator that averages task vectors first and then soft-thresholds each layer at a level set by that layer's interference; across twenty-two merge configurations from four model families only destructive merges exceed the score threshold, twelve of fourteen pre-evaluation predictions were correct, and PRISM keeps all five destructive merges above the threshold within evaluation noise of the base model without data or tuning, where plain averaging falls at least 14.4 points below it or collapses entirely.
The work proposes using the variance of specialists' task vectors across specialists as a measure of interference, yielding a pre-merge score that predicts collapse, and designs PRISM: an operator that averages task vectors first and then soft-thresholds each layer at a level set by that layer's interference; across twenty-two merge configurations from four model families only destructive merges exceed the score threshold, twelve of fourteen pre-evaluation predictions were correct, and PRISM keeps all five destructive merges above the threshold within evaluation noise of the base model without data or tuning, where plain averaging falls at least 14.4 points below it or collapses entirely.
The work proposes using the variance of specialists' task vectors across specialists as a measure of interference, yielding a pre-merge score that predicts collapse, and designs PRISM: an operator that averages task vectors first and then soft-thresholds each layer at a level set by that layer's interference; across twenty-two merge configurations from four model families only destructive merges exceed the score threshold, twelve of fourteen pre-evaluation predictions were correct, and PRISM keeps all five destructive merges above the threshold within evaluation noise of the base model without data or tuning, where plain averaging falls at least 14.4 points below it or collapses entirely.