# LoRA Adapters

Existing secure fine-tuning protocols hand the data owner hidden representations or logits from every step, and from those the data owner can copy the provider's LoRA adapter: from the hidden representations of a single mini-batch of 32 samples, the copy agrees with the original on 99.2% of test inputs, and logits alone give about 94%. VeilTune instead runs the forward pass, backpropagation and the Adam update entirely in secure multi-party computation and reveals only the trained adapter to the model owner, averaging 5.3 points above SecP-Tuning on four few-shot tasks with RoBERTa-large and staying within 0.3 to 1.7 points of plaintext training.

Existing secure fine-tuning protocols hand the data owner hidden representations or logits from every step, and from those the data owner can copy the provider's LoRA adapter: from the hidden representations of a single mini-batch of 32 samples, the copy agrees with the original on 99.2% of test inputs, and logits alone give about 94%. VeilTune instead runs the forward pass, backpropagation and the Adam update entirely in secure multi-party computation and reveals only the trained adapter to the model owner, averaging 5.3 points above SecP-Tuning on four few-shot tasks with RoBERTa-large and staying within 0.3 to 1.7 points of plaintext training.

Audio language model benchmark scores can mistake language ability for acoustic understanding. Using synthetic readings and transcripts as references, the study measures the recording gain of the original audio over words across 92 tasks from 6 benchmarks, finding that 19 open models trail Gemini 3.1 Pro by 28.6 and 41.9 percentage points on global-attribute and within-recording structure tasks, with about half or more of each gap lying in the recording gain. Training LoRA adapters with a content anchor in the readout-decline layers of Gemma-4 12B and Qwen2.5-Omni-7B raises attribute and structure accuracy by 12.8 to 44.4 points while mean lexical accuracy stays within 1 point of the base models.

Audio language model benchmark scores can mistake language ability for acoustic understanding. Using synthetic readings and transcripts as references, the study measures the recording gain of the original audio over words across 92 tasks from 6 benchmarks, finding that 19 open models trail Gemini 3.1 Pro by 28.6 and 41.9 percentage points on global-attribute and within-recording structure tasks, with about half or more of each gap lying in the recording gain. Training LoRA adapters with a content anchor in the readout-decline layers of Gemma-4 12B and Qwen2.5-Omni-7B raises attribute and structure accuracy by 12.8 to 44.4 points while mean lexical accuracy stays within 1 point of the base models.

AutoAdapt is a modular instruction-tuning framework that first discovers latent domains with three methods (K-means, BERTopic, MNLI), then trains one LoRA adapter per domain independently and in parallel, and routes at inference with a parameter-free centroid mechanism; across 14 domain-specific benchmarks and GPT-4o pairwise judgements it reaches parity with a single LoRA adapter trained on all domains, while adding or updating a domain requires training only one adapter instead of full-model retraining.

AutoAdapt is a modular instruction-tuning framework that first discovers latent domains with three methods (K-means, BERTopic, MNLI), then trains one LoRA adapter per domain independently and in parallel, and routes at inference with a parameter-free centroid mechanism; across 14 domain-specific benchmarks and GPT-4o pairwise judgements it reaches parity with a single LoRA adapter trained on all domains, while adding or updating a domain requires training only one adapter instead of full-model retraining.

In a controlled setting, the authors trained low-rank adapters to make binary decisions for fictitious characters according to latent linear preference functions, found that accurate self-reporting of learned preferences emerges from decision-task fine-tuning alone without explicit self-report supervision, and showed that faithful models have significantly higher attribution similarity between the decision and self-report tasks (mean 0.34 versus 0.08; paired-bootstrap 95% CI [0.16, 0.36]), allowing faithful and unfaithful self-report to be distinguished without understanding the report content.

In a controlled setting, the authors trained low-rank adapters to make binary decisions for fictitious characters according to latent linear preference functions, found that accurate self-reporting of learned preferences emerges from decision-task fine-tuning alone without explicit self-report supervision, and showed that faithful models have significantly higher attribution similarity between the decision and self-report tasks (mean 0.34 versus 0.08; paired-bootstrap 95% CI [0.16, 0.36]), allowing faithful and unfaithful self-report to be distinguished without understanding the report content.

The work proposes AuthorMix, a lightweight, modular, and interpretable authorship style transfer framework that first trains individual style-specific LoRA adapters on a small set of high-resource authors and then, via reinforcement-learning-based layer-wise adapter mixing, rapidly builds a specialized adaptation model for each new target using only a handful of target-style training examples; according to the abstract, AuthorMix ranks first on the combined style-meaning score among all baselines including GPT-5.1, substantially improves meaning preservation over the trained baselines, and is the only method best-or-tied on every dimension under human evaluation.

The work proposes AuthorMix, a lightweight, modular, and interpretable authorship style transfer framework that first trains individual style-specific LoRA adapters on a small set of high-resource authors and then, via reinforcement-learning-based layer-wise adapter mixing, rapidly builds a specialized adaptation model for each new target using only a handful of target-style training examples; according to the abstract, AuthorMix ranks first on the combined style-meaning score among all baselines including GPT-5.1, substantially improves meaning preservation over the trained baselines, and is the only method best-or-tied on every dimension under human evaluation.

This study introduces GenEHR, an autoregressive generative model trained on electronic health records from millions of patients that explicitly represents irregular inter-visit time intervals via RAdix Time Encoding and combines parameter-efficient gated low-rank adaptation for supervised fine-tuning, improving prediction of a first cancer diagnosis within a five-year horizon across five large EHR cohorts and supporting risk-based screening for aggressive cancers such as pancreatic and ovarian cancer.

This study introduces GenEHR, an autoregressive generative model trained on electronic health records from millions of patients that explicitly represents irregular inter-visit time intervals via RAdix Time Encoding and combines parameter-efficient gated low-rank adaptation for supervised fine-tuning, improving prediction of a first cancer diagnosis within a five-year horizon across five large EHR cohorts and supporting risk-based screening for aggressive cancers such as pancreatic and ovarian cancer.