96% throughput changes the confidential-inference trade-off. Benchmark the protected path.
NVIDIA measured confidential DeepSeek-R1 inference at 96.1–98.2% of its unprotected throughput on eight B200 GPUs. Security teams can make CC-on performance a deployment test instead of assuming privacy requires a large inference penalty.
NVIDIA measured confidential DeepSeek-R1 inference on eight B200 GPUs at 96.1–98.2% of its confidential-computing-off output throughput. Mean time per output token stayed within 1.2–4.3% of the unprotected baseline across concurrency levels from one to 16.
That does not prove confidential inference is nearly free for every workload. It changes the deployment question. Teams handling proprietary models, enterprise context or sensitive prompts can benchmark the protected execution path directly instead of treating a large performance penalty as inevitable.
The benchmark isolates the security cost
NVIDIA held the model, hardware, framework, sequence lengths, parallelism and concurrency constant, changing confidential computing state between runs. The test used DeepSeek-R1-0528-NVFP4 with a 32K-token input and 1K-token output on one DGX B200 system with eight B200 GPUs and Intel TDX.
The workload was deliberately unfriendly to hidden overhead: long context, extended generation and low concurrency expose encryption and transfer costs that busier workloads can mask. That makes the result more useful as a method than as a universal percentage.
Confidential inference should be a measured production property, not a performance penalty accepted on faith.
Secure execution changes the runtime
The small measured gap depends on software adapting to the protected hardware path. NVIDIA says B200 confidential computing routes host-to-device transfers through an encrypted bounce buffer, changes timing assumptions used by kernel autotuning and removes NVLink SHARP multicast from the tested configuration.
TensorRT LLM compensates by choosing memory paths that fit confidential execution, moving repeated readback work to an asynchronous worker and using a different timing source for autotuning. Multi-GPU communication also has to account for unavailable multicast capability. Turning on confidential computing without checking those runtime assumptions can therefore produce a different result.
Make CC-on versus CC-off part of acceptance testing
Platform teams deploying private inference should add a paired benchmark to the release gate. Run the intended model, sequence lengths, concurrency, parallelism and framework version with confidential computing both disabled and enabled. Record retained throughput and per-token latency, then keep those measurements beside the security configuration.
The security gate remains separate: NVIDIA recommends attesting the environment before serving sensitive workloads. The operational change is to test attestation and performance together. If the protected path meets the workload's latency and throughput budget, confidential inference becomes an enforceable deployment baseline rather than an exception reserved for workloads willing to absorb an assumed cost.
Sources
Written by the Devence Lab research team.
