Running a model requires decrypting both the user's prompt and the model's weights, because, as the whitepaper notes, techniques such as homomorphic encryption are not yet usable at the scale and speed inference needs (footnote 2). The post "Confidential Inference via Trusted Virtual Machines", published on 18 June 2025 under Anthropic's Announcements label, asks how to keep that decrypted data confined to a small, verifiable part of the system. It accompanies a whitepaper written with the AI security company Pattern Labs, "Confidential Inference Systems: Design principles and security risks" (Version 1.0, June 2025, 32 pages). The post gives two reasons for the work: model weight security, as one component of protecting frontier models against capable attackers, and user security, proving that sensitive user data is kept private. It calls itself "just a sketch of our research to start a conversation" and says it is too early to say what designs or features will follow. Neither document reports experiments or measurements.
Terms used in this note
- Confidential computing. The whitepaper quotes the Confidential Computing Consortium's definition: protecting data in use by computing inside a hardware-based, attested trusted execution environment (Overview).
- Trusted execution environment (TEE). An isolated environment whose memory the operating system cannot read, with debugging and monitoring features disabled, and which can produce a signed attestation that it runs the expected code (Overview, "Meeting the requirements").
- Attestation. A signed record of measurements (hashes) of the boot process, operating system and workload, produced with a hardware root of trust such as a trusted platform module (TPM), that a remote party can check before trusting the environment ("Secure cryptographic attestation").
- Key management system (KMS) or keyserver. The service that holds decryption keys and releases them only to environments whose attestation it accepts.
The design in the post
The post states the guiding principle that sensitive data should stay encrypted except where it is processed ("Inference service"). User data has to be in cleartext at two points: the API server, which tokenizes prompts and handles request logic, and the inference server, which runs the model on accelerators. Model weights need to be in cleartext only on the inference server. The post focuses on the inference server.
Because not all accelerators fully support confidential computing yet, the planned design puts a small, trusted "model loader and invoker" in a separate virtual machine isolated by the hypervisor. The loader decrypts data, sends it to the accelerator, invokes computations and returns encrypted results. It presents itself to the rest of the inference server as a "virtual accelerator" and accepts only programs signed by Anthropic's secure continuous-integration server, which ensures the code was reviewed by multiple engineers. The rest of the inference server is untrusted and can change often without affecting confidentiality. The post's aim is that confidentiality holds "no matter what the rest of the system does," provided the loader runs correctly ("Inference service").
The loader's environment is meant to have encrypted, hardware-isolated memory, disabled debugging and cryptographic proof of the code being run ("Trusted environment"). The post is explicit about a gap in the first property: the features needed to share encrypted host memory with an accelerator "aren't well established" yet, so for now Anthropic relies on its compute providers for physical datacenter and hypervisor security. The other two are achieved with a TPM that measures each boot stage; a keyserver releases keys only when the measurements match. The keyserver therefore decides what counts as trusted, and the post says Anthropic is exploring arrangements in which other parties validate the trusted code and run independent keyservers.
The post's end-to-end flow for a request: encrypted before it reaches Anthropic; decrypted, processed and re-encrypted at the API server; handled encrypted by the inference server; decrypted only inside the loader; and encrypted again before the completion leaves it ("Conclusions"). Weights are stored encrypted, decrypted only in the loader, and never released from it. Possible future additions at the loader layer include egress bandwidth limits on servers holding cleartext weights and requiring "a signature from a safety classifier" before inference runs ("Future directions"). The post closes by suggesting that hardware designers who have not already done so consider building confidential computing into their chips, since a hardware root of trust on the accelerator would shrink the trust boundary.
What the whitepaper covers
The whitepaper is general rather than a description of Anthropic's systems.
Parties and goals. Inference involves a model owner, a data owner (the user) and a service provider; each wants confidentiality without having to trust the other two (Overview). The paper separates the confidential data aspect (inputs and outputs) from the confidential model aspect (weights and architecture); a system may implement either or both ("The two aspects of confidential inference").
Link to weight security levels. It draws on RAND's report Securing AI Model Weights (Nevo et al., 2024), which defines five security levels. For the top two, SL4 and SL5, meant to resist state-level attackers, the whitepaper says RAND strongly recommends confidential computing where available, with physical-attack protections, keys generated and kept inside the TEE, and prespecified, audited, signed code ("Confidential computing for SL4 and SL5").
Accelerators. Two approaches are described: accelerators with native TEE support (NVIDIA H100 confidential computing is the example), or bridging a CPU enclave to an accelerator that lacks it, which the paper says "is not airtight" because data passes to the GPU unencrypted ("Trusted execution environments with AI accelerators").
Components. A reference design has three parts: the inference service (a model-agnostic enclave that loads and invokes the model and must hold no model-owner secrets); the model provisioning procedure (encrypting weights, with a trade-off analysis of three places to run the KMS); and the enclave build environment, where binary assurance can come from reproducible builds, binary analysis, or a pre-assured third-party or open-source binary ("Components and their design principles").
Threat model. Attackers are assumed to control everything outside the TEE, and each party treats the others as potential threats ("Threat model"). The paper separates systemic risks in the components relied on (hardware side channels, hypervisor flaws, vendor supply-chain compromise, cryptographic breaks including future quantum computers) from introduced risks in a particular implementation (debug-mode enclaves, parsing bugs, man-in-the-middle attacks on enclave traffic, permissive KMS policies, decommissioned vulnerable enclaves that still have key access, excessive key reuse, "harvest now, decrypt later" theft of encrypted weights, and compromised build or development environments).
Limitations and open questions
- The post is a design sketch and reports no deployment, performance cost, or security evaluation.
- It relies on compute providers for physical and hypervisor security until accelerators can share encrypted memory.
- The whitepaper says its threat section is "not a comprehensive security review" and that confidentiality of weights outside inference, such as backups and training, is out of scope ("Limitation of scope").
- Trust ultimately rests on the keyserver and on code review of the signed loader, so the guarantee is only as strong as those processes.
Research questions
- What is the performance cost of routing inference through a trusted loader? Neither document reports overhead. Latency and throughput measurements with and without the loader would show whether the design is practical at scale.
- How would a safety-classifier signature requirement work in practice? The post mentions it as a possible future safeguard. A specification of which classifier signs what, and how a refusal is enforced at the loader, would show how a content policy could be tied to cryptographic enforcement.
- Can outside parties verify the trusted code? The post mentions independent keyservers and outside validation. A published reproducible build of the loader, checked by a third party, would test whether users can verify the guarantee rather than rely on Anthropic.
- Which of the whitepaper's introduced risks are most likely in real deployments? The list is qualitative. A red-team exercise against a reference implementation, reporting which risks were exploitable, would rank them.
How it connects
This item is about infrastructure security and is not connected by either document to other notes in the library. The post links to Anthropic's ASL-3 activation announcement for the egress bandwidth controls it mentions.
References
- Anthropic. (2025, June 18). Confidential Inference via Trusted Virtual Machines. https://www.anthropic.com/research/confidential-inference-trusted-vms
- Pattern Labs and Anthropic. (2025, June). Confidential Inference Systems: Design principles and security risks (Version 1.0). https://assets.anthropic.com/m/c52125297b85a42/original/Confidential_Inference_Paper.pdf
- Nevo, S., Lahav, D., Karpur, A., Bar-On, Y., Bradley, H. A., and Alstott, J. (2024). Securing AI model weights: Preventing theft and misuse of frontier models. RAND. https://www.rand.org/pubs/research_reports/RRA2849-1.html (cited as the whitepaper describes it; not read for this note).