Confidential Computing for AI: TEE Without the Hype

Confidential computing aims to protect data while it is in use—when a processor must operate on plaintext in memory. For AI systems, the proposition is compelling: an organisation can process sensitive data and proprietary weights in the cloud while reducing trust in host administrators, the hypervisor, or the infrastructure operator. A “confidential VM” label alone does not close the threat model.

In one paragraph: an effective design needs a hardware-isolated TEE, trustworthy remote attestation, a policy that compares measurements with an approved configuration, and a mechanism that releases keys only after successful verification. A TEE does not fix a vulnerable application, prompt injection, excessive agent permissions, output leakage, or flawed data governance. It shifts part of the trust boundary and reduces selected infrastructure attack classes.

The Confidential Computing Consortium defines the field around protecting data in use by performing computation inside a hardware-based, attested Trusted Execution Environment. IETF RFC 9334: Remote ATtestation procedureS defines vendor-neutral attestation roles and data flows.

The three states of data

The conventional model distinguishes:

State	Typical control	Example
data at rest	disk or object encryption	checkpoint in storage
data in transit	TLS or mTLS	prompt sent to an inference API
data in use	TEE and memory isolation	weights and batch during computation

Disk encryption stops protecting content after a volume is mounted. TLS terminates in a process or proxy. A model and its input must be decrypted somewhere before a CPU or GPU can compute. A TEE is intended to make that “somewhere” inaccessible to parts of the privileged platform software.

It does not make everything invisible. Depending on the platform, an operator may still observe metadata, size, timing, availability, network endpoints, and resource consumption. The application inside the TEE sees plaintext and can disclose it if malicious or vulnerable.

What a TEE actually is

A Trusted Execution Environment is a protected execution context with defined isolation, confidentiality, and integrity properties. Implementations differ in scope:

an enclave isolates selected application code and memory;
a confidential VM includes a larger virtual machine;
a CPU TEE plus GPU TEE aims to protect an accelerated computation path;
confidential container services build on a protected VM and workload policy.

A larger TEE can simplify migration, but it expands the Trusted Computing Base. The guest kernel, libraries, drivers, and runtime inside the protected environment become components that must be trusted. A smaller enclave can reduce the TCB while requiring application redesign and careful handling of calls to the untrusted world.

Microsoft’s Trusted Execution Environment documentation describes a protected CPU and memory area backed by hardware and distinguishes rehosting inside confidential VMs from enclave-oriented application design.

Attestation: do not trust a label

Encrypted memory does not answer the question: which code will receive my data? Remote attestation provides evidence about the execution environment. RFC 9334 defines three central roles:

the Attester produces Evidence about its environment;
the Verifier appraises Evidence against policy and reference values;
the Relying Party makes an access decision using an Attestation Result.

Measurements may cover hardware, firmware, boot chain, VM image, configuration, or workload. A verifier compares them with approved state. Evidence freshness matters, so protocols rely on timestamps, nonces, or another anti-replay mechanism.

Attestation does not declare that a system is absolutely secure. It states that presented evidence satisfies a specific policy. If the policy accepts a vulnerable image, the result can be formally valid while the system remains vulnerable. If an attacker compromises the reference-value pipeline, the trust decision also fails.

Releasing a key after attestation

A practical architecture works as follows:

The workload starts in a TEE without access to the data or model key.
It creates fresh attestation evidence bound to the instance and a session key.
A verifier checks the chain, measurements, freshness, and policy.
The relying party or KMS releases a short-lived key only to the approved environment.
The workload decrypts data inside the TEE and erases the key after use.
A changed or expired environment must attest again.

RFC 9334 includes a reference use case for confidential ML model protection: a service releases a model or decryption key only after successful device attestation. This distinction matters. An encrypted checkpoint stored beside a key that is available without attestation does not provide the same property.

CPU and GPU in one trust chain

Large models need accelerators. Protecting CPU memory is insufficient if weights move in plaintext to a GPU or a driver outside the TEE can inspect them. The design needs a protected CPU–GPU channel, attestation of both ends, and policy that binds the result to a particular workload.

NVIDIA describes an on-die root of trust, GPU TEE, and a protected session to a driver in a CPU TEE for H100 in Confidential Computing on NVIDIA H100 GPUs. Vendor documentation is an important implementation source, but teams must verify the exact mode, hardware generation, firmware version, and protected operation set.

An accelerated task does not automatically become confidential. An unprotected kernel fallback, preprocessing outside the TEE, or temporary tensors written to ordinary storage can break end-to-end assumptions.

Practical AI use cases

Inference over regulated data

An organisation may process medical, financial, or trade-secret data while reducing trust in host operators. The TEE limits some infrastructure access, but the application still needs data minimisation, access control, retention, and privacy filtering.

Protecting proprietary weights

A provider can deploy a model at a customer’s site and release its key only to an attested environment. This helps prevent a system administrator from simply reading the file or memory, but it does not guarantee that API responses cannot support model extraction.

Joint data processing

Two organisations can run an agreed workload over combined data and release their keys after attestation. They still need an output policy: approved code may reveal individual records through an overly detailed result.

Agent and tool isolation

A TEE may protect agent secrets from the host, but it will not stop a prompt injection that persuades approved code to use a secret incorrectly. Separate controls remain necessary: an AI agent sandbox, least privilege, and egress policy.

What confidential computing does not solve

Workload vulnerabilities

If the application has RCE, SQL injection, insecure deserialization, or broken authorization, an attacker may operate inside the trusted environment. The TEE protects against a defined adversary outside it, not against accepted application code.

Leakage through legitimate interfaces

A model can disclose data through output, logs, metrics, or tool calls. A TEE does not decide whether a token is allowed. RAG security still requires document-level authorization and context filtering.

Side channels and denial of service

Protection from side channels depends on the technology and threat model. Timing, access patterns, cache behaviour, paging, or resource use may leak information. The infrastructure operator can usually stop the workload or deny resources. Confidentiality and integrity do not guarantee availability.

Supply-chain compromise

Attesting a malicious but approved image proves its identity, not its benevolence. The build still needs signing, provenance, SBOM, and review. SBOM guidance and SLSA/Sigstore controls answer a different question and should form one evidence chain with attestation.

Metadata and outputs

Model size, request frequency, inference time, and network destinations may remain visible. Output leaves the TEE and needs TLS, authorization, and recipient controls.

Designing an attestation policy

Treat policy as versioned, reviewed code. Define:

allowed hardware types and minimum firmware levels;
boot-chain, image, and configuration measurements;
workload artefact signature and provenance;
permitted debug modes, normally disabled in production;
Evidence freshness and maximum result age;
binding to tenant, region, and session key;
behaviour during verifier failure, revocation, or outage.

Failing open during an attestation outage can destroy the security property. Failing closed can stop critical business operations. Make the decision explicitly using availability requirements, short-lived cached results, TTLs, and an emergency procedure.

An operational deployment model

Phase	Control	Evidence
build	reproducible image, scanning, signing	digest, provenance, SBOM
deploy	policy permits exact measurements	policy-as-code version
start	fresh TEE attestation	Evidence and Attestation Result
key release	key bound to approved instance	KMS log and nonce
runtime	egress, IAM, telemetry, quotas	events tied to workload identity
update	reference values approved first	authorization and rollback plan
revoke	block vulnerable firmware or image	revocation record and incident

RFC 9334 notes that attestation material itself may contain device identifiers or personal information. Limit its retention and access. Do not place raw attestation tokens in ordinary application telemetry.

Testing confidential AI controls

A security test should do more than confirm a green status in the console. In a controlled environment, attempt to start an unapproved image, change configuration, replay old Evidence, alter a nonce, enable debug mode, restore a snapshot, and request a key after attestation expires.

Then verify data boundaries: temporary files, swap, core dumps, logs, preprocessing, CPU–GPU communication, output, and backups. Secrets management should bind KMS policy to the attestation result rather than only to a cloud VM identity.

Decision checklist

Which adversary are we addressing: host administrator, cloud provider, customer, or malware?
Which code and hardware are included in the TCB?
Is the entire CPU–GPU–storage path protected?
Is each key released only after fresh verified attestation?
Does policy validate workload, firmware, debug state, and provenance?
Does verifier failure result in fail-open or fail-closed behaviour?
Does the application still enforce least privilege, sandboxing, egress, and output controls?
Is there a revocation process for a vulnerable image or firmware build?
Can an independent test prove the property rather than a product label?

Confidential computing is valuable when the threat model includes privileged infrastructure and data in use. It is not a magic “secure AI” seal. A strong deployment combines attestation, secure builds, key management, workload containment, and end-to-end testing. Book an AI security assessment — we will verify the TEE architecture and whether keys genuinely reach approved code only.