Perseus folds the homomorphic inference stack into one public Python library, and its Llama-3-8B decode beats the fastest known implementation, Sylph

Encrypted model inference now has a public Python library that puts the cryptographic backend, data packing and bootstrap placement in one environment, so adopters can write a new encrypted model, packing layout or bootstrap scheme in pure Python on top of a Rust and CUDA CKKS runtime. Most prior optimizations were implemented in different, sometimes closed-source frameworks, making integration difficult and time-consuming and preventing orthogonal optimizations from compounding. Across the implemented GPT-2, Llama-3-8B, ViT and BERT, the combination reports state-of-the-art latency; results for competing methods are reproduced when their code is public and cited from publication when it is not.

The latency reduction comes from three changes, one per level: half of each key-switching key is uniformly random, so it is recomputed in-kernel with a counter-mode ChaCha12 expander inside the two kernels that use the key, halving key storage; packing extends Cachemir to the complex slots of CKKS so each ciphertext carries two real vectors; and bootstrap placement first captures the computation graph, annotates each ciphertext by coefficient magnitude and packing periodicity, then places bootstraps with an iterated minimum cut. Cachemir previously exploited only the real slots of a ciphertext, and prior min-cut placers decided from depth and scale alone rather than from the values themselves. Complex-Cachemir roughly doubles the throughput of many operations and layers and roughly halves memory requirements; the placer executes fewer bootstraps on average than comparable placers; all measurements use one NVIDIA RTX PRO 6000 with a 32-bit RNS CKKS chain at 128-bit security.

On Llama-3-8B decoding, under a fixed bootstrap schedule decided by its own planner, Perseus reduces latency relative to Sylph, the fastest known implementation, which runs on roughly the same hardware but on the closed-source HEaaN. Sylph is the prior fastest encrypted Llama-3-8B implementation and serves as the comparison point. Encrypted perplexity for decoding is reported alongside the plaintext reference, and quality metrics are computed on held-out data from the approximations' calibration set; prefill is likewise the fastest implementation, with normalized accuracy reported over held-out prompts.

Evaluation covers four models, GPT-2-small, Llama-3-8B, ViT and BERT; Llama-3-8B is tested in both decode and prefill modalities, GPT-2-small is reported for prefill, and ViT and BERT are classification tasks. All measurements use one NVIDIA RTX PRO 6000 with a 32-bit RNS CKKS chain at 128-bit security, and quality metrics are computed on held-out data from the approximations' calibration set. The authors also publish a public leaderboard for encrypted GPT-2 inviting community improvements.

Encrypted perplexity is reported alongside the plaintext reference, and normalized accuracy serves as a proxy for degradation from CKKS approximate arithmetic, bootstraps and required nonlinearity approximations. The authors state that CKKS execution of large models remains far from practical despite frequent optimizations across the stack.