Mixed-precision quantization stops treating each layer as one bit width: A2BQ assigns bits per block by attention sensitivity, reaching 7.884 perplexity on TinyLlama

Precision allocation moves from one bit width per layer to a separate decision for each 128-row weight block, so a single transformer layer can hold 4-, 6-, and 8-bit blocks at once. Earlier mixed-precision or layer-wise quantization methods assign one bit width per layer from aggregated importance metrics and do not capture intra-layer variation. Across five models from 1.1B to 8B parameters, A2BQ reaches an actual average of 5.69 bits against a 5.7-bit target, yet the block counts differ by model: TinyLlama-1.1B assigns 123, 159, and 70 of its 352 blocks to 4, 6, and 8 bits.

Low-rank adaptation capacity is no longer uniform across layers but varies with each layer's quantization sensitivity and assigned bit width. Conventional qLoRA uses a fixed rank across all target layers, implicitly assuming every layer benefits equally from extra adaptation capacity. The TinyLlama-1.1B rank map contains 154 module-specific assignments with ranks from 11 to 19 and 12,257,280 trainable adapter parameters, versus 6,307,840 for the fixed-rank-8 qLoRA baseline, so the comparison evaluates the full adaptive design rather than a parameter-matched one.

Under the same final mixed-bit policy and rewrite scope, adding qLoRA+ adaptation lowers test perplexity below mixed-bit rewriting alone on all five models. Mixed-bit rewriting alone gives TinyLlama-1.1B a perplexity of 8.994, Qwen2.5-3B 9.077, Phi-3-mini-4k 7.884, Mistral-7B 6.938, and LLaMA-3.1-8B 8.258. The corresponding A2BQ+qLoRA+ perplexities are 7.884, 8.335, 6.922, 6.571, and 7.501, all measured on the held-out WikiText-2 test split with the allocation map derived from the validation split.

Estimated packed-weight storage and runtime GPU memory are separate quantities, the first from the allocation map and the second from dequantized-buffer execution. The reference backend reconstructs rewritten weights into floating-point tensors before inference and runs standard floating-point linear operations rather than matrix multiplication on physically packed low-bit weights. The estimated whole-model packed payload under the deployed rewrite scope falls 4.02% to 7.90% relative to FP16, while the principal TinyLlama-1.1B configuration measures about 2.08 GB peak GPU memory against 2.09 GB for FP16.

The demonstrated setting covers five models, TinyLlama-1.1B, Qwen2.5-3B, Phi-3-mini-4k, Mistral-7B, and LLaMA-3.1-8B, with calibration and allocation on the first 200 WikiText-2 validation documents at maximum length 512, perplexity reported on the held-out test split, and a single 80 GB H100. The principal configuration rewrites only the Q and O projections in the attention scope that satisfy the output-dimension condition, with the first and last three layers protected.

Attention-derived sensitivity correlates only moderately and architecture-dependently with post-quantization layer-wise KL divergence, with Pearson −0.667 and Spearman −0.400 over 16 active layers on TinyLlama-1.1B, so it is positioned as an allocation signal rather than a predictor of distribution shift. Mistral-7B improves perplexity after adaptation while mean KL divergence rises, showing that proximity to the FP16 predictive distribution is not a sufficient indicator of downstream quality. In the block-size ablation the active-region average precision differs across the three settings, so that experiment targets the full pipeline rather than an equal-payload comparison.