Homomorphic encryption can run a model directly on ciphertext, but the latency introduced by bootstrapping, the costly operation that refreshes a ciphertext's available multiplication depth, keeps encrypted large models impractical, and existing optimizations sit in different, sometimes closed-source frameworks that are difficult and time-consuming to integrate. Perseus is a public Python library spanning the cryptographic backend, data packing and bootstrap placement, with in-kernel key regeneration, complex-Cachemir packing and bootstrap planning driven by coefficient magnitude and packing periodicity on GPUs. On Llama-3-8B decoding its latency is lower than the fastest known implementation, Sylph, which runs on roughly the same hardware but on the closed-source HEaaN.
Homomorphic encryption can run a model directly on ciphertext, but the latency introduced by bootstrapping, the costly operation that refreshes a ciphertext's available multiplication depth, keeps encrypted large models impractical, and existing optimizations sit in different, sometimes closed-source frameworks that are difficult and time-consuming to integrate. Perseus is a public Python library spanning the cryptographic backend, data packing and bootstrap placement, with in-kernel key regeneration, complex-Cachemir packing and bootstrap planning driven by coefficient magnitude and packing periodicity on GPUs. On Llama-3-8B decoding its latency is lower than the fastest known implementation, Sylph, which runs on roughly the same hardware but on the closed-source HEaaN.