# Importance Sampling

DINGO-TGR infers binary parameters and six post-Newtonian phase-deformation parameters jointly with neural posterior estimation, then reweights the network's approximate posterior by importance sampling. For GW150914 the inference takes about 38 minutes against roughly 177 hours for bilby-tgr with dynesty, and about 4.4 hours against 238 hours for the higher signal-to-noise GW250114. GW150914 shows a mild offset from general relativity in the leading principal-component mode, but 115 of 1,000 general-relativity injections resembling GW150914 reach or exceed that offset, an empirical upper-tail p-value of about 0.115 that does not reject the general-relativity null at the 5% level.

DINGO-TGR infers binary parameters and six post-Newtonian phase-deformation parameters jointly with neural posterior estimation, then reweights the network's approximate posterior by importance sampling. For GW150914 the inference takes about 38 minutes against roughly 177 hours for bilby-tgr with dynesty, and about 4.4 hours against 238 hours for the higher signal-to-noise GW250114. GW150914 shows a mild offset from general relativity in the leading principal-component mode, but 115 of 1,000 general-relativity injections resembling GW150914 reach or exceed that offset, an empirical upper-tail p-value of about 0.115 that does not reject the general-relativity null at the 5% level.

Asynchronous reinforcement learning runs rollout and training at the same time, but the learner trains on samples from older weights, and importance sampling alone had not stopped the drift. REINFORCE Zero, from NVIDIA Research and Inferact, first aligns the rollout engine and trainer so both compute bit-identical per-token logprobs at the same weights, then handles the remaining weight lag with unclipped token-level importance sampling. On 16 GPUs over 12K updates it reaches 2.40× the training-step throughput of synchronous training with aligned kernels (68.55 versus 164.77 hours) at comparable quality, and Qwen3-4B with Python tools trains at lag 64 through 5,000 updates, with mean reward rising from 0.287 to about 0.78.

Asynchronous reinforcement learning runs rollout and training at the same time, but the learner trains on samples from older weights, and importance sampling alone had not stopped the drift. REINFORCE Zero, from NVIDIA Research and Inferact, first aligns the rollout engine and trainer so both compute bit-identical per-token logprobs at the same weights, then handles the remaining weight lag with unclipped token-level importance sampling. On 16 GPUs over 12K updates it reaches 2.40× the training-step throughput of synchronous training with aligned kernels (68.55 versus 164.77 hours) at comparable quality, and Qwen3-4B with Python tools trains at lag 64 through 5,000 updates, with mean reward rising from 0.287 to about 0.78.

SRAM yield sign-off needs per-cell failure probabilities of 10⁻⁶ to 10⁻¹² at every PVT condition, and running importance sampling at each one is expensive. The method transfers each failure lobe's design point across PVT, corrects it with one SPICE run per lobe, and adds split-conformal margins computed from importance sampling on a random subset of sign-off conditions, escalating undecided conditions to importance sampling. Across all 242 unseen conditions of a SkyWater 130-nm cell and an ASAP7 7-nm cell, no calibrated decision was unsafe, the minimum operating voltage read from the conformal upper limit lay −1 to +34 mV from importance sampling, and a 1-Mb macro delivered at least 99.88% yield.

SRAM yield sign-off needs per-cell failure probabilities of 10⁻⁶ to 10⁻¹² at every PVT condition, and running importance sampling at each one is expensive. The method transfers each failure lobe's design point across PVT, corrects it with one SPICE run per lobe, and adds split-conformal margins computed from importance sampling on a random subset of sign-off conditions, escalating undecided conditions to importance sampling. Across all 242 unseen conditions of a SkyWater 130-nm cell and an ASAP7 7-nm cell, no calibrated decision was unsafe, the minimum operating voltage read from the conformal upper limit lay −1 to +34 mV from importance sampling, and a 1-Mb macro delivered at least 99.88% yield.

The authors propose a covariate-dependent consecutive-ratio Markov random field (CR-MRF) that jointly models multivariate ordinal ratings, show that comparison models such as Bradley-Terry and Plackett-Luce arise as restrictions of it under independence and an exponential-race argument, and develop maximum likelihood inference with an importance-sampling estimator of the intractable normalizer; in simulations and on MovieLens, modeling ordinal scores directly yields more accurate and better-calibrated out-of-sample pairwise preference predictions than fitting comparison models to derived win-loss data.

The authors propose a covariate-dependent consecutive-ratio Markov random field (CR-MRF) that jointly models multivariate ordinal ratings, show that comparison models such as Bradley-Terry and Plackett-Luce arise as restrictions of it under independence and an exponential-race argument, and develop maximum likelihood inference with an importance-sampling estimator of the intractable normalizer; in simulations and on MovieLens, modeling ordinal scores directly yields more accurate and better-calibrated out-of-sample pairwise preference predictions than fitting comparison models to derived win-loss data.

FERPO is an on-policy maximum-entropy reinforcement learning algorithm that derives an optimal target action distribution from a policy-improvement objective regularized by entropy and KL divergence, then fits the actor to that target by minimizing a forward-KL objective estimated with self-normalized importance sampling (SNIS), thereby performing policy improvement using critic values without differentiating the critic with respect to actions; experiments and ablations on MuJoCo Playground and ManiSkill show competitive performance and sample-efficiency gains, and computational benchmarks show faster actor updates than REPPO.

FERPO is an on-policy maximum-entropy reinforcement learning algorithm that derives an optimal target action distribution from a policy-improvement objective regularized by entropy and KL divergence, then fits the actor to that target by minimizing a forward-KL objective estimated with self-normalized importance sampling (SNIS), thereby performing policy improvement using critic values without differentiating the critic with respect to actions; experiments and ablations on MuJoCo Playground and ManiSkill show competitive performance and sample-efficiency gains, and computational benchmarks show faster actor updates than REPPO.