NVIDIA Vera Rubin: 10x More Tokens per Megawatt

NVIDIA announced on 21 July 2026 that Vera Rubin NVL72 systems are entering production and already running at initial partners including CoreWeave, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure. The headline number is 10 times more tokens per second per megawatt than Grace Blackwell NVL72 in a DeepSeek-R1 benchmark run by CoreWeave. The multiplier is less important than the accounting change behind it: AI capacity is increasingly constrained not by the number of available GPUs but by power, networking, cooling, memory bandwidth, and the cost of a useful result.

NVIDIA’s Vera Rubin deployment and architecture announcement describes a co-designed rack-scale system. The published results come from NVIDIA and its partners and cover particular configurations and workloads. Buyers should not copy the 10x figure into a business case for a different model, cloud region, or service. An honest evaluation must use the real context length, concurrency, output quality, and total service cost.

What was confirmed on 21 July

Vera Rubin NVL72 is no longer merely an architectural roadmap. NVIDIA says production is ramping, racks are live at cloud operators, and its supply chain spans more than 350 factory sites in 30 countries. Google Cloud has a first A5X instance based on NVL72 running, while CoreWeave has published early measurements from live hardware.

The platform joins seven new chips across five rack trays. Rubin GPUs and the Vera CPU provide the compute layer, while NVLink 6, ConnectX-9, Spectrum-6, and BlueField-4 contribute communication and infrastructure services. The composition matters because an AI workload is more than matrix multiplication. CPUs orchestrate agents, the fabric synchronizes experts and nodes, DPUs handle and isolate data paths, and cooling determines whether the system can sustain its target power envelope.

Component	System role	A buyer should ask
Rubin GPU	training, post-training and inference	which model and precision were measured?
Vera CPU	orchestration, tools and agent logic	does CPU time limit concurrency?
NVLink 6	scale-up communication inside a system	what topology and oversubscription apply?
Spectrum-X / Spectrum-6	scale-out communication across racks	how much network cost is in each token?
BlueField-4	DPU, networking, security and isolation	where is the tenant trust boundary?
liquid cooling	heat removal at high density	who owns water, maintenance and failures?

What 10x tokens per megawatt means

CoreWeave compared Vera Rubin NVL72 with Grace Blackwell NVL72 on DeepSeek-R1 and reported a tenfold increase in tokens per second per megawatt. This is an energy-throughput measure: how much generative work the system performs inside a constrained power budget. It matters where another megawatt requires years of utility planning, a new substation, or a cooling redesign.

It is not a complete value metric. Tokens do not have constant quality or business value. A short output from a small model, a long reasoning trace, and a multimodal token are not interchangeable. A valid comparison must hold constant at least the model and version, precision, input and output length, batch size, latency SLO, concurrency, quality, and inference features such as speculative decoding.

If a platform generates more tokens but the application retries more work or rejects more answers, useful economics may deteriorate. The approach in our article on measuring AI ROI still applies: count an accepted process outcome, not raw inference volume.

Why DeepSeek-R1 stresses communication

DeepSeek-R1 uses a mixture-of-experts architecture. Each token activates selected expert subnetworks that may reside on different accelerators. A router distributes tokens, and the system creates intensive all-to-all communication. When the fabric cannot keep up, expensive GPUs wait for data and theoretical FLOPS do not become useful results.

NVIDIA states that NVLink 6 supplies 260 TB/s of all-to-all bandwidth inside NVL72 so a rack can behave as a unified accelerator. CoreWeave also describes a fabric based on 102.4 Tb/s Spectrum-6 with no oversubscription in its spine-leaf deployment. These are architecture and partner-implementation properties, not guarantees for every cloud offering. A customer needs to know whether it is buying a whole rack, a partitioned cluster, or a profile subject to placement constraints.

The same complexity affects security. More components participating in one inference mean more management planes, firmware, telemetry, and tenant boundaries. A cloud security review should cover the service API, management network, node images, model registry, checkpoint storage, and automation roles.

CPUs return to the center of agentic AI

The announcement is not only about GPUs. The Vera CPU uses Olympus cores and, according to NVIDIA, delivers twice the single-threaded performance, three times the core-to-core bandwidth, and 40% lower memory latency than the compared chiplet designs. DeepInfra reported up to 2.2x faster orchestration and as many as 1.6x more concurrent agents at the same quality of service in its environment.

The result is plausible because an agent executes control code, parses model output, talks to databases, evaluates policy, and waits for tools. A GPU can finish generation quickly while a slow orchestration thread still limits the workflow. A customer benchmark should separate model time, queues, CPU, tools, network, and storage. Otherwise one component can improve dramatically without changing application latency.

Agent scale also amplifies mistakes. Higher concurrency produces more parallel operations against data and APIs. The controls in our guide to AI agent identities are as important as performance: a distinct workload identity, short-lived credentials, least privilege, and a complete tool-call audit.

Cooling, water, and power density

Vera Rubin NVL72 uses a tray design with no cables, fans, or hoses inside the compute tray, which NVIDIA says cuts tray assembly from hours to a minute. A 45°C liquid inlet design can support dry coolers without a conventional chiller. NVIDIA estimates that higher-temperature dry cooling and a closed loop can save millions of gallons of water per megawatt annually in new AI factories.

This does not mean the platform consumes no resources or fits every existing data center. Facility teams must account for climate, ambient temperature, fluid quality, pump redundancy, leak detection, maintenance, rack density, and emergency shutdown. PUE can hide non-IT energy, while WUE needs local water context. Sustainability reporting should distinguish operator measurements from generalized vendor estimates.

Data sovereignty does not follow from GPU location

NVIDIA tied the production news to European deployments involving Microsoft and Mistral. The announcement describes public-cloud, connected, and disconnected private-cloud modes. That is relevant to government, finance, and healthcare, but a “sovereign-ready” label is not evidence of compliance on its own.

Organizations must determine where data, logs, backups, keys, support, and administrative personnel reside. They need subprocessors, retention, international-transfer, key-management, and deletion controls. For especially sensitive models, confidential computing and attestation may help, while remembering that a TEE has limits and cannot fix an insecure application.

Securing rack-scale AI

High integration can improve performance while increasing dependence on a single stack. A firmware update for a switch, DPU, BIOS, driver, communication library, or model runtime can affect the cluster. Operators need a version inventory and controlled canary deployment, not a simultaneous update of every rack.

Priority controls include:

separation of management, storage, training, and inference networks;
phishing-resistant MFA and just-in-time elevation for operators;
signed node images, drivers, and model artifacts;
encryption at rest and in transit with clear key ownership;
tenant limits, quotas, and resource-exhaustion protection;
aggregate fabric anomaly detection, not just single-host monitoring;
checkpoint recovery plans with tested restoration;
a process for firmware and library vulnerability remediation.

These requirements fit the Zero Trust model. Physical presence in the same rack does not create a trusted zone, and one operator account should not have permanent control over every plane.

Running a buyer-controlled benchmark

Select three representative workflows first: perhaps a low-latency interactive assistant, batch processing, and a tool-using agent. Freeze the model version, decoding parameters, and evaluation set. Measure tokens per second but also time to first token, p95/p99 tail latency, successful task rate, energy, network cost, and downtime.

Then test resilience. Constrain a link, remove a node, simulate a storage error, and observe how quickly the scheduler recovers work. Test isolation: can one tenant infer another tenant’s queue, telemetry, checkpoint, or job identifier? Finally, calculate the cost of an accepted result at realistic utilization. GPU list price without reservation, energy, network, storage, licenses, and operating staff is not TCO.

What the launch means for an ordinary company

Most organizations will not buy an NVL72 system. They will consume it indirectly through a model API, managed endpoint, or cloud provider. The generation still changes negotiations: customers can request evidence about token cost, energy, data location, isolation, and availability of specific hardware profiles.

Migration solely because a new platform has a larger multiplier makes little sense. If the application waits on a database, has weak caching, sends excessive context, or retries failed jobs, workflow redesign may be cheaper. A model router can direct demanding jobs to a strong model and simple jobs to a smaller one under explicit quality and confidentiality rules.

FAQ

Is Vera Rubin 10 times faster for every workload?

No. The published result is tokens per second per megawatt for a CoreWeave DeepSeek-R1 benchmark against Grace Blackwell NVL72. Another model, batch size, SLO, or configuration can behave differently.

Does 10x tokens per megawatt mean a 10x lower bill?

Not automatically. A bill includes amortization or cloud pricing, reservation, power, networking, storage, software, operations, and utilization. NVIDIA also claims as little as one-tenth the cost per million tokens relative to GB200 NVL72, but the customer’s price depends on contract and workload.

Does an on-premises rack solve data protection?

No. Administrative accounts, logs, service providers, updates, keys, and telemetry export remain. The full responsibility model still needs to be documented and tested.

The main conclusion

Vera Rubin demonstrates that AI economics has moved from an individual GPU to a complete system: CPU, fabric, DPU, cooling, and software. The 10x result deserves attention, but it should begin validation rather than end it. The winning platform is the one that produces the most correct, accepted outcomes inside the organization’s energy, time, risk, and cost limits.

Planning a private or cloud AI platform? Before signing a long reservation, combine the performance benchmark with an AI security assessment and architecture review. Contact BreachRoad to build a measurable test plan.