# GRAM

Multi-task deployment often requires merging independently fine-tuned models into one, but existing merging methods compress all tasks into a single shared low-rank space, so tasks whose subspaces are misaligned lose task-relevant directions. GRAM uses only the fine-tuned weights, first partitioning tasks by the principal-angle geometry of their output subspaces and then running a standard SVD-based merge inside each group. On GLUE with Flan-T5-Base it reaches 83.8% average accuracy, above the 81.9% of global merging and the 81.7% of the strongest baseline TSV-Merge, at the cost of one extra deployed model.

Multi-task deployment often requires merging independently fine-tuned models into one, but existing merging methods compress all tasks into a single shared low-rank space, so tasks whose subspaces are misaligned lose task-relevant directions. GRAM uses only the fine-tuned weights, first partitioning tasks by the principal-angle geometry of their output subspaces and then running a standard SVD-based merge inside each group. On GLUE with Flan-T5-Base it reaches 83.8% average accuracy, above the 81.9% of global merging and the 81.7% of the strongest baseline TSV-Merge, at the cost of one extra deployed model.