Anthropic open-sources FlashProve: one H100 proves a 400B model's inference at 110 tokens per second
Synopsis
When a user queries a large model, nothing in the reply shows it came from the model the developer claims, so a routing service swapping in a cheaper model or an attacker tampering with the token stream goes unnoticed. Anthropic's open-source FlashProve attaches a zero-knowledge succinct argument (zkSNARK, a short certificate a verifier can check quickly without the weights) to inference. On 4-bit Llama 4 Maverick, a 400B-parameter mixture of experts, a single H100 generates proofs at about 110 tokens per second, roughly 19 MB per 1,024-token request, at about 70 to 95 times the GPU time vLLM needs to run the model itself.
Interpretation
A model provider can now attach a short proof to each reply, letting a user with no weights and limited compute confirm that those tokens were computed from one previously committed set of weights. Previously a user could judge model identity only from output quality, and a third-party router substituting a cheaper model or an attacker altering the token stream could not be detected from the reply itself. The proof is bound to a weight commitment published in the offline phase, and anyone holding the commitment, prompt and reply can check it; a reply that departs from the committed model fails the check.
The system splits proving into an offline commitment and an online proof, where the online phase reads finished text in one parallel pass and issues a proof for each token. It builds on a zero-knowledge succinct argument (zkSNARK), compiled from an interactive oracle proof (IOP, an information-theoretic proof object) using hash functions and assembled from code proximity tests with linear constraints and lookup arguments drawn from coding theory. On the public 4-bit weights of Llama 4 Maverick, a server with one H100 and 24 CPU cores proves a batch of 24 requests in about 223 seconds, or 110 tokens per second, with each proof 18.78 MB; verification takes about 17 seconds on a 24-thread CPU.
Perspective
The demonstrated setting is the 4-bit integer version of Llama 4 Maverick, a 400B-parameter mixture of experts, proved and verified on one H100 with 24 CPU cores within 1,024-token request windows; the system hides weights but not model architecture, and it proves after the fact rather than alongside inference.
The proof covers an integer model under greedy decoding, so a floating-point implementation or sampled decoding could diverge at some position and no longer match the proved computation; the proof reveals nothing about weights, but side channels such as proving time are out of scope; the fast C++ verifier carries no formal guarantee and agrees with the Lean verifier only on the proofs in their tests.