Samsung Research's Net Utility allocates LoRA merging rank budgets per singular direction, lifting vision tasks by 2.1% and language tasks by 2.2% on average

The paper attributes the merging performance gap to how rank is allocated rather than to the merging operator itself, reporting that relaxing the uniform-budget assumption yields roughly +3.4% average performance across seven vision datasets with task arithmetic merging. Most prior merging methods (such as TSV-Merging, PRIME, and Core Space) fix the retained rank as a single constant shared across layers, and some require an equal split across tasks; this work isolates rank-budget allocation as its own object of study. Based on a controlled observation over seven vision datasets with task arithmetic, reported by the paper; per-dataset detail is not given in the loaded text.

It proposes Net Utility: for each singular direction of each task LoRA's SVD, the score is the normalized source-task benefit minus cross-task interference, dimensionless so scores compare across tasks, with the highest-scoring directions selected globally under a total budget. Unlike AdaRank, which needs unlabelled calibration data and gradient descent, or PRIME, which resolves to one shared rank, the metric depends only on adapter weights, is closed-form, data-free, and requires no optimization loop, and it selects no rank at all: the rank profile falls out of a single global top-k selection. The derivation includes a separable upper bound (Proposition 1) and a net-utility decomposition (Proposition 2), with complexity analysis in the appendix; SVDs can be computed directly from the LoRA factors without materializing dense updates.

At matched budgets, Net Utility allocation generally outperforms uniform allocation across three merging spaces (full, core, KnOTS) and multiple merging methods, averaging +2.1% on vision and +2.2% on language, with some method-space combinations reaching as much as +3.8%. Gains tend to grow as the budget shrinks (for example, TSV's gain on language tasks rises from one budget to a tighter one), indicating that which directions are kept matters more under tight budgets. Covers 7 vision tasks (ViT-B/32) and 6 language tasks (Qwen3-4B), with each method evaluated at three budgets using normalized accuracy; DARE on language tasks in core and KnOTS space is an exception, within about 1% of uniform or slightly below it.

Analysis shows the optimal direction set need not be a prefix: because interference is governed by cross-task geometry, a low-singular-value direction orthogonal to other tasks can outrank a dominant but heavily shared one; the budget also need not be fully spent, since net utility can be negative. This challenges the prefix truncation used by existing SVD-based merging methods and gives a picture of non-uniform allocation across modules and layers (for example, later-layer value and output projections in ViT-B/32 require more rank). The paper reports that most chosen sets on vision are non-prefix and provides a correlation analysis between net utility and task difficulty (on seven Hendrycks MATH topics, relative accuracy is inversely correlated with mean net utility).

The result targets offline merging of a fixed set of adapters: given a total rank budget, it selects which singular directions to retain for vision (ViT-B/32, 7 tasks) and language (Qwen3-4B, 6 NLI tasks) adapters. It suits deployers who must avoid hot-swapping at inference while operating under a rank budget, such as on-device or batched serving. The method can be layered on top of merging operators including task arithmetic, TSV, DARE, TIES, and Iso-C, and across the full, core, and KnOTS merging spaces, without task data, a validation set, or a forward pass.

The net-utility derivation rests on linear merging and a separable upper bound; the paper notes it does not re-derive the metric for a specific merging algorithm and does not tune the hyperparameter α, and tuning with a validation set could further change results. DARE on language tasks in core and KnOTS space does not exceed uniform allocation, indicating gains depend on the method-space combination. The loaded text is full text but tables appear as plain text, so some numbers and per-task detail cannot be fully verified, leaving the exact gain ranges per method-space combination an open question. In addition, the correlation between net utility and task difficulty comes from a single controlled Hendrycks MATH experiment, so its generality awaits testing on more task sets.