# Hypernetwork

Bifidelity modeling uses cheap low-fidelity simulations to supplement scarce high-fidelity experiments, but the usual product is a single deterministic fit that says little about how trustworthy a prediction is. This work extends a single-fidelity Langevin sampling hypernetwork to bifidelity data, using a shared-latent data model that predicts both the low-fidelity response and the high-fidelity correction while a learned drift network generates parameter samples from the posterior score. On a synthetic hyperelastic case, conditional staged training cut the SPD-drift RMSE from 0.075 to 0.024 and the convex-potential drift from 0.081 to 0.017; on experimental and DNS data, predicting the high-fidelity response directly gave lower error than an additive discrepancy.

Bifidelity modeling uses cheap low-fidelity simulations to supplement scarce high-fidelity experiments, but the usual product is a single deterministic fit that says little about how trustworthy a prediction is. This work extends a single-fidelity Langevin sampling hypernetwork to bifidelity data, using a shared-latent data model that predicts both the low-fidelity response and the high-fidelity correction while a learned drift network generates parameter samples from the posterior score. On a synthetic hyperelastic case, conditional staged training cut the SPD-drift RMSE from 0.075 to 0.024 and the convex-potential drift from 0.081 to 0.017; on experimental and DNS data, predicting the high-fidelity response directly gave lower error than an additive discrepancy.

This work studies hypernetworks conditioned only on an LLM's input query to estimate LoRA, and introduces distributional hypernetworks that output a distribution over LoRAs via an end-to-end loss with a differentiable Monte Carlo approximation; results show that even the mean of the learned distribution can outperform deterministic hypernetworks, that sampling weight updates yields a form of test-time scaling distinct from token sampling with performance improving as more weight samples are considered and remaining stronger than corresponding token-sampling adaptation baselines, and that generated updates can transfer across queries.

This work studies hypernetworks conditioned only on an LLM's input query to estimate LoRA, and introduces distributional hypernetworks that output a distribution over LoRAs via an end-to-end loss with a differentiable Monte Carlo approximation; results show that even the mean of the learned distribution can outperform deterministic hypernetworks, that sampling weight updates yields a form of test-time scaling distinct from token sampling with performance improving as more weight samples are considered and remaining stronger than corresponding token-sampling adaptation baselines, and that generated updates can transfer across queries.

HyperThink proposes a text-to-parameter approach in which a lightweight hypernetwork reads the question and predicts updates to a small subset of the base LLM's parameters, while a vector-quantized decoder constrains those updates to a finite set of reusable patterns; trained end-to-end on outputs from the base model itself, it eliminates long thinking traces at test time, so that after one hypernetwork forward pass the adapted model generates a concise step-by-step solution and final answer without an intermediate trace, using far fewer tokens while retaining strong reasoning performance, and it improves the low-latency region of the accuracy-latency trade-off on mathematical and general reasoning tasks, with its strongest gains in the near-non-thinking regime.

HyperThink proposes a text-to-parameter approach in which a lightweight hypernetwork reads the question and predicts updates to a small subset of the base LLM's parameters, while a vector-quantized decoder constrains those updates to a finite set of reusable patterns; trained end-to-end on outputs from the base model itself, it eliminates long thinking traces at test time, so that after one hypernetwork forward pass the adapted model generates a concise step-by-step solution and final answer without an intermediate trace, using far fewer tokens while retaining strong reasoning performance, and it improves the low-latency region of the accuracy-latency trade-off on mathematical and general reasoning tasks, with its strongest gains in the near-non-thinking regime.

The authors propose HyperNSDE, a continuous-time generative model that encodes heterogeneous static covariates with an HI-VAE, maps the static latent representation through a hypernetwork to subject-specific drift parameters driving a latent Neural SDE, and jointly models observation times with a latent-state-dependent intensity process, trained via a deterministic–stochastic path decomposition with a signature-kernel objective; on simulated data and the VELOUR and PPMI cohorts it achieves better observation-time fidelity and several distributional and privacy metrics, while forecasting and correlation performance is mixed and affected by observation-grid regularity and trajectory smoothness.

The authors propose HyperNSDE, a continuous-time generative model that encodes heterogeneous static covariates with an HI-VAE, maps the static latent representation through a hypernetwork to subject-specific drift parameters driving a latent Neural SDE, and jointly models observation times with a latent-state-dependent intensity process, trained via a deterministic–stochastic path decomposition with a signature-kernel objective; on simulated data and the VELOUR and PPMI cohorts it achieves better observation-time fidelity and several distributional and privacy metrics, while forecasting and correlation performance is mixed and affected by observation-grid regularity and trajectory smoothness.

The work proposes an "infinite-parameter LLM" architecture in which a compact hypernetwork encodes run-time data (facts, instructions, demonstrations) into a low-dimensional latent code, that code generates a low-rank additive modulation of a shared base feed-forward network so each token's "expert" is generated rather than drawn from a stored bank, and a Bayesian belief over the latent code is carried and updated online so the effective weight keeps evolving within a session; the authors also specify an evaluation protocol that pits carrying knowledge in the weights against carrying it in the prompt at matched budget.

The work proposes an "infinite-parameter LLM" architecture in which a compact hypernetwork encodes run-time data (facts, instructions, demonstrations) into a low-dimensional latent code, that code generates a low-rank additive modulation of a shared base feed-forward network so each token's "expert" is generated rather than drawn from a stored bank, and a Bayesian belief over the latent code is carried and updated online so the effective weight keeps evolving within a session; the authors also specify an evaluation protocol that pits carrying knowledge in the weights against carrying it in the prompt at matched budget.