Turning Quantum Circuits into Per-Token Bespoke Objects: A Width Study of HyperQ on a Frozen Diffusion Language Model

Introduces token-conditioned circuit emission as an architecture: instead of searching for one circuit for a whole task, a hypernetwork maps each token's hidden state to continuous coordinates on a shared sparse circuit skeleton, so every token gets its own circuit. Earlier integrations of quantum circuits with transformers mostly studied parameter-efficient adaptation or inserted circuits at a fixed resource level, often independently of the language model and for a fixed task; here the circuit coordinates are conditioned directly on the language model's hidden states and placed as a residual branch parallel to the frozen fused query-key-value projection. The construction is specified in full: a low-rank adapter implements the hypernetwork, emitting single-qubit rotation angles, two-body coupling angles on the edges, and a per-qubit readout axis; the branch is attached to all transformer blocks, the backbone stays frozen, only the branch and its low-rank projections are trained, and the only training signal is the masked-diffusion objective.

Restricting the emitted circuits to a strict two-body IQP family on a fixed edge set (one ring layer plus one chord layer, giving every qubit degree four) yields both an affordable readout and an initialisation gradient that does not shrink with width. A general IQP interior contains all non-empty subsets of generators, and exact simulation of a generic -qubit pure state requires amplitudes; keeping only generators of weight at most two on a fixed edge set gives an exact closed form for the readout with total cost , and a coordinate-gradient variance of under the specified independent reference initialisation that stays constant across 16, 32 and 64 qubits because the degree is fixed at four. The closed-form expectation value and the gradient-variance expression are given and verified against brute-force statevector calculations at to qubits; the text also states that this characterises circuit-coordinate gradients for the specified circuit family and reference initialisation, not the frozen backbone, and does not guarantee trainability in general.

Runs a width sweep at 16/32/64 qubits inside a 1.1-billion-parameter frozen backbone, observing continued improvement as the register widens, with the emitted circuit outperforming hand-designed circuits and discrete motif search. Existing quantum-enhanced models each evaluate their quantum component at a fixed resource level and use different evaluation suites, so they cannot answer whether increasing the number of qubits makes a language model better; here the width sweep is run on one backbone under one evaluation suite, and HyperQ is reported as the first among these quantum-enhanced models to report the standard six-benchmark downstream suite. The six-benchmark average (ARC-e, HellaSwag, PIQA, BoolQ, RACE, GSM8K) goes 47.65 to 52.80 to 54.30; at 64 qubits it reaches 54.30 against 49.59 for the frozen backbone and 50.63 for the same backbone with a classical low-rank adapter; at 16 qubits it scores 47.65 and remains below the frozen backbone, so the gain appears only as the register widens; the emitted circuit beats the fixed ansätze and the 3-motif and 6-motif searches at every width, and at 32 qubits its single-pass 52.80 exceeds the best three-circuit ensemble at 52.24.

Compiles the emitted circuits into executable gate sequences under fixed gate-count and depth budgets, performs a bounded paired validation on a 156-qubit superconducting processor, and preserves the masked-diffusion backbone's any-order decoding. Per-token circuits would be impractical inside a billion-parameter model if each required a full statevector or an unconstrained compilation procedure; here one per-token compilation pipeline serves both model evaluation and hardware transpilation, and the fixed edge set allows routing once per width rather than once per token. Execution on ibm_quebec through Qiskit Runtime EstimatorV2 shows the mean absolute deviation from the analytic value rising with width from 16 to 32 and 64 qubits, against a shot-noise floor of ; on a fixed 256-item subset per benchmark the average score drops by a bounded amount at each width, with GSM8K losing the most and PIQA the least; the authors explicitly claim no quantum advantage and the hardware pass does not evaluate native hardware-based training.

This work speaks to researchers and practitioners doing parameter-efficient adaptation inside a frozen masked-diffusion backbone: it shows that, at 16 to 64 qubits, in a 1.1-billion-parameter backbone, with 20,000 prompt-response pairs, per-token continuous circuit emission can be trained, compiled, and executed on superconducting hardware, and that the downstream average improves with width. It enables follow-up work to ask whether the same trend continues at wider registers, in other circuit families, on other backbones such as autoregressive models, and under native hardware training; the hardware portion is a bounded paired validation confirming that the emitted circuits can run on noisy devices while preserving most of their downstream performance.

The boundaries the authors themselves draw are worth watching: the gradient-variance result characterises circuit-coordinate gradients for the specified circuit family and reference initialisation, does not explain why downstream accuracy increases with width, and does not guarantee trainability in general; the width sweep covers only three points, 16, 32 and 64, and does not establish a scaling law beyond that range; the readout remains classically efficient at every tested width, so this is an architectural result rather than a computational one, and the authors explicitly claim no quantum advantage. On the hardware side, readout error increases with width, the paired subset is only 256 items per benchmark, and native hardware-based training is not evaluated. Direct comparison with earlier quantum-augmented models is not possible because they use different evaluation suites, and HyperQ is the first among them to report the standard six-benchmark suite; WikiText perplexity is the one column where it does not lead. A reader who sees only the homepage summary without the methods and tables will miss the circuit family, the closed-form readout, the gradient conditions, and the hardware pairing protocol, which affects how the scope of the conclusions can be judged.