The Architecture: A Superficial Confidentiality Layer
LLM providers, including Anthropic, OpenAI, and Google, have been returning "encrypted" step-by-step LLM reasoning traces—the chain-of-thought (CoT)—directly to clients. This architectural vulnerability, where encrypted LLM reasoning traces could be replayed outside its original context, was first identified by Matthew Green in May 2026. This might appear to protect proprietary IP or user privacy while offering auditability or replayability. However, these encrypted blocks were designed with full compatibility across different sessions, users, and even models within a provider's ecosystem.
This design choice has significant implications: a block generated for one user could be replayed by another. A trace from a high-end "frontier" model could be processed by a smaller, cheaper "decoder" model. This architectural decision points to a critical flaw: either a single, global encryption key was used across model families and user contexts, or the cryptographic binding mechanisms were too weak to offer real contextual isolation. This confuses obfuscation with actual cryptography. If your "encryption" allows any authorized client to decrypt any block, regardless of its origin, you lack confidentiality. You have a puzzle trivial to solve once you have the key, or in this case, a weaker model that acts as the key.
The Core Issue: Trust and Contextual Integrity of LLM Reasoning Traces
The core issue here is not performance; it is a breakdown in *trust* and *contextual integrity*, stemming from a failure to establish adequate architectural boundaries for sensitive or proprietary data. These boundaries dictate who can access what, when, and under what conditions. The LLM providers' architecture failed to establish these boundaries for LLM reasoning traces.
The attack mechanism is simple: inject an encrypted block from a powerful model into a weaker, less-guarded model from the same provider. Then, prompt the weaker model to transcribe it in plaintext. This works because the weaker model, being part of the same ecosystem, possesses the necessary decryption capability to process these LLM reasoning traces. For instance, researchers demonstrated this by using Anthropic's Claude Haiku 4.5 to decode traces from Opus 4.8, OpenAI's GPT-5.6 Luna to decode GPT-5.6 Sol, and Google's Gemini Robotics 1.6 to decode Gemini 1.5 Flash. This capability effectively grants universal decryption access within the provider's ecosystem, allowing a less secure entry point to compromise data from more protected systems.
This architectural flaw yields immediate, severe consequences:
- Intellectual Property Theft: This flaw enables the distillation of proprietary reasoning from expensive "teacher" models into cheaper "student" models. Decoding 10,000 LLM reasoning traces costs an estimated $720. This represents a highly cost-effective method for illicitly acquiring proprietary model reasoning, circumventing significant research and development investments.
- Privacy Breaches: Publicly shared logs containing these blocks become rich sources of sensitive data. We have recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials (including 62 API keys, 33 email addresses, 33 passwords, and private keys) from GitHub and Hugging Face. Some of these secrets were *only* in the LLM reasoning traces, never in the visible chat.
- Jailbreaking and Hazardous Knowledge: Models might internally process dangerous information but refuse to output it. These LLM reasoning traces expose that internal processing, allowing extraction of prohibited instructions.
- Invisible Prompt Injection: Malicious instructions can be embedded within an encrypted block. When a victim uses it, the model interprets these as its own prior reasoning, completely hidden from the user. This presents a critical vector for covert data exfiltration or system manipulation, posing a significant security risk.
Discussions on platforms like Hacker News and Reddit, for instance, indicate significant concern regarding the perceived prioritization of rapid deployment over robust security, particularly among developers building agents on these proprietary APIs.
The Trade-offs: Availability at the Expense of Consistency
This situation highlights a fundamental trade-off: prioritizing client-side flexibility and developer experience (ease of use) over robust data confidentiality and integrity (consistent access control) for LLM reasoning traces.
- Client-side Flexibility and Developer Experience (Availability/Ease of Use): Allowing clients to store and replay reasoning traces locally simplifies state management for developers and potentially reduces server load for providers. This makes the system feel more "available" to the client.
- Data Confidentiality and Integrity (Consistency of Access Control): Ensuring that sensitive LLM reasoning traces remain private and are only accessible under strict, cryptographically enforced conditions.
Providers prioritized the former; whether this was a conscious decision or an oversight, the outcome is clear. By making reasoning blocks universally compatible and client-side, they sacrificed strong consistency in access control. The system *should* have consistently enforced that a reasoning trace generated for User A, Model X, Session Y, could *only* be decrypted and processed within that exact context. It did not. This is not a subtle bug; it is a fundamental architectural decision that undermined the very purpose of "encryption" for sensitive LLM reasoning traces.
The "encryption" was a weak attempt to achieve confidentiality without the necessary architectural robustness. It was a superficial fix that failed to address the underlying design flaw, which inherently lacked context-specific cryptographic links. Effective distributed system design necessitates anticipating malicious actors and network partitions, building resilience against potential failures. Here, the failure was in assuming that client-side data, even "encrypted," could maintain its integrity and confidentiality without solid server-side enforcement or strong cryptographic binding to its origin context.
Recommendations: Restoring Integrity Through Robust Design
To address this, LLM providers should implement architectural patterns that prioritize data integrity and confidentiality over mere obfuscation. My recommendations for a review based on sound architectural principles are:
Mandating Server-Side Storage
The most direct and architecturally sound solution involves storing LLM reasoning traces exclusively on provider servers. Clients would then receive a unique, ephemeral session identifier to reference a trace, rather than the trace data itself. This approach shifts the access control boundary definitively to the server, aligning with established principles for managing sensitive state in distributed systems, rather than relying on client-side storage, which inherently lacks adequate security mechanisms for sensitive state.
Implementing Cryptographic Contextual Binding
Should client-side storage be deemed an unavoidable requirement, each LLM reasoning trace block must be cryptographically bound to its specific user, session, and model identifier. This necessitates that the decryption key or mechanism is intrinsically derived from, or inextricably linked to, these precise contextual identifiers. Consequently, any attempt to decrypt or utilize the block outside its original context would result in cryptographic failure. Such an implementation demands a robust Key Management System (KMS) and meticulously planned key rotation strategies.
Enforcing Strict API Gateway Isolation
Furthermore, strict isolation must be enforced at the API gateway level. Requests attempting to inject LLM reasoning traces from one model into another, or from one user's context into another's, must be rejected outright. This measure establishes stringent application-level isolation, ensuring that distinct operational contexts remain genuinely separate and do not intermingle.
Integrating Model-Level Defense-in-Depth
Finally, a crucial defense-in-depth layer involves training models to recognize and refuse requests to decode or transcribe LLM reasoning traces. While not a primary security control, this mechanism serves as an essential last line of defense. Even if an attacker circumvents other architectural safeguards, the model itself should be equipped to identify and reject such malicious prompts.
Beyond the immediate concern of IP theft, this incident highlights a significant architectural oversight in the design of these systems regarding LLM reasoning traces. It underscores that the rapid deployment of advanced AI capabilities has, in this instance, compromised fundamental distributed systems security principles. It is imperative that LLM providers enhance their security practices. The long-term trust and transparency of AI systems hinge on such improvements.
