# Low-Precision Transformer Inference

The work extends the PRISM vectorized rounding library to arbitrary virtual precision via a variable-precision stochastic rounding (VPSR) algorithm and, holding the numerical format fixed while varying only the rounding rule at individual operation sites, emulates DistilGPT-2; it supports the site-level trade-off with a probabilistic forward-error bound for linear projections and a second-order decomposition of expected cross-entropy at the softmax, finding that at t=6 significand bits SR in the MLP gives 1.15x the full-precision reference perplexity versus 2.21x for RN, that the ordering reverses at the language-model head, and that a mixed-precision configuration (MLP output at t=6, SR in the MLP, RN at the head) brings perplexity within 1.10x, a 28% reduction over matched-bit RN.

The work extends the PRISM vectorized rounding library to arbitrary virtual precision via a variable-precision stochastic rounding (VPSR) algorithm and, holding the numerical format fixed while varying only the rounding rule at individual operation sites, emulates DistilGPT-2; it supports the site-level trade-off with a probabilistic forward-error bound for linear projections and a second-order decomposition of expected cross-entropy at the softmax, finding that at t=6 significand bits SR in the MLP gives 1.15x the full-precision reference perplexity versus 2.21x for RN, that the ordering reverses at the language-model head, and that a mixed-precision configuration (MLP output at t=6, SR in the MLP, RN at the head) brings perplexity within 1.10x, a 28% reduction over matched-bit RN.