In our last blog post, we showed how to break privacy-preserving LLM schemes — decoding permuted model states with near-perfect accuracy. Today, we present our defense: Cascade 🕵️‍♂️✨

Cascade secures LLM inference by splitting hidden states across parties via "token sharding"—each node only sees a few tokens in sequence. The farther apart a node’s tokens are, the higher the cost of an attack—growing exponentially with distance.

Cascade retains full fidelity in inference—no approximations, no fixed-point math. It splits computation between CompNodes (for MLPs) and AttnNodes (for attention), preserving exact results across distributed stages.

The crux of Cascade is a token-sharding scheme that is robust to both our reconstruction attack and known learning-based attacks. We demonstrate in the paper that our approach, called c-δ sharding, achieves strong privacy without compromising fidelity.

We perform extensive experiments on Cascade’s security against learning-based attacks. While it lacks SMPC’s formal guarantees, we demonstrate that with enough nodes, Cascade is empirically robust to these attacks.

Cascade’s most significant benefit is speed and scalability. Compared to existing SMPC schemes like MPCFormer and Puma, Cascade is 2 orders of magnitude faster, even the most secure setting we tested (72 participants):

Cascade was motivated by the need to find a new paradigm in the tradeoff between privacy, security and scalability. We believe its core idea—token-sharding—strikes that balance and opens a new direction for privacy-preserving LLMs.

Our work on both the attack and Cascade was accepted at ICML 2025, and will be presented on 7/16 at 4:30pm PST in East Exhibition Hall A-B, Room E-2612, in the Vancouver Convention Center. Come and chat with us! 🔗

Come join us for the Ritual social at ICML. Meet our AI and crypto research team, unwind with great people, and end your day the right way. It’s the perfect Ritual. Register here: