Public articles linked to the same research event.
arXiv The work proposes GPart (Global Partition fine-tuning), a highly parameter-efficient fine-tuning method that maps a d-dimensional trainable vector directly into the full weight space through a sparse, isometric partition matrix, retains a fixed global parameter-sharing prior while removing the additional low-rank reconstruction used by LoRA-based methods, and thereby yields a single main hyperparameter (d), exact end-to-end isometry, and a minimal checkpoint representation consisting of the trainable vector and a random seed, matching or improving over existing PEFT methods at ultra-low parameter budgets across natural language understanding, computer vision, and mathematical reasoning benchmarks.
The work proposes GPart (Global Partition fine-tuning), a highly parameter-efficient fine-tuning method that maps a d-dimensional trainable vector directly into the full weight space through a sparse, isometric partition matrix, retains a fixed global parameter-sharing prior while removing the additional low-rank reconstruction used by LoRA-based methods, and thereby yields a single main hyperparameter (d), exact end-to-end isometry, and a minimal checkpoint representation consisting of the trainable vector and a random seed, matching or improving over existing PEFT methods at ultra-low parameter budgets across natural language understanding, computer vision, and mathematical reasoning benchmarks.
The work proposes GPart (Global Partition fine-tuning), a highly parameter-efficient fine-tuning method that maps a d-dimensional trainable vector directly into the full weight space through a sparse, isometric partition matrix, retains a fixed global parameter-sharing prior while removing the additional low-rank reconstruction used by LoRA-based methods, and thereby yields a single main hyperparameter (d), exact end-to-end isometry, and a minimal checkpoint representation consisting of the trainable vector and a random seed, matching or improving over existing PEFT methods at ultra-low parameter budgets across natural language understanding, computer vision, and mathematical reasoning benchmarks.
The work proposes GPart (Global Partition fine-tuning), a highly parameter-efficient fine-tuning method that maps a d-dimensional trainable vector directly into the full weight space through a sparse, isometric partition matrix, retains a fixed global parameter-sharing prior while removing the additional low-rank reconstruction used by LoRA-based methods, and thereby yields a single main hyperparameter (d), exact end-to-end isometry, and a minimal checkpoint representation consisting of the trainable vector and a random seed, matching or improving over existing PEFT methods at ultra-low parameter budgets across natural language understanding, computer vision, and mathematical reasoning benchmarks.