AMD and Google TPU Closing the Gap on NVIDIA

AMD, Google TPU, and NVIDIA are competing across hardware benchmarks, software infrastructure governance, and supply dynamics in mid-2026. AMD and Red Hat have a formal strengthened strategic collaboration [17], which complicates AMD's known absence from llm-d's supported hardware list [18] — the two companies are active partners even as AMD GPU support in Red Hat AI 3's inference stack remains unconfirmed against the official hardware configuration documentation [19]. Red Hat AI 3 launched in October 2025 [8] integrating llm-d into OpenShift AI, subsequently donated to CNCF governance in March 2026 [9], while NVIDIA holds a 3.6 million unit B200/GB200 backlog with enterprise buyers facing 16–24 month waits [21][22].

NVIDIA's incumbency rests on ecosystem lock-in and hardware availability, but its own supply backlog creates a structural window for cost-competitive alternatives. AMD leads on published inference cost efficiency and holds a formal Red Hat partnership, yet lacks confirmed hardware support on the inference stack Red Hat ships at enterprise scale. Whether AMD converts that partnership into llm-d hardware support — and whether NVIDIA's backlog drives enterprises to substitute rather than wait — are the defining questions for AI infrastructure in 2026.

AMD's competitiveness in AI inference has moved from vendor claims to formalized benchmark territory. In April 2026, MLCommons officially published MLPerf Inference v6.0 results [1], prompting Forbes to directly ask whether AMD had beaten NVIDIA in AI performance [2] — a question that captures the genuine competitive ambiguity of the round. Spheron's analysis positioned the results as providing clear GPU performance rankings for AI workloads [3], while Nebius simultaneously highlighted top-tier NVIDIA performance from the same data [4]. AMD's results were strong enough to force the question while NVIDIA retained the absolute top position. This benchmark context builds on AMD's earlier confirmation of a 40% cost advantage over NVIDIA's B200 on single-node GLM5 FP8 inference, achieved just 14 weeks after GLM5's launch using SGLang v0.12 [5] — a result SemiAnalysis framed as evidence that ROCm's software maturity has crossed a threshold where AMD hardware can realize its cost advantages in production. AMD's pricing strategy is selective: the MI355X carries higher manufacturing costs than NVIDIA's competing chip while priced below cost to gain inference market share [6], but AMD's MI350 GPU separately saw a 66.7% price increase [7], revealing a targeted rather than uniformly aggressive posture.

On the software infrastructure front, the llm-d coalition's timeline spans enterprise product launch, then open-source governance. Red Hat AI 3 launched in October 2025 [8], integrating llm-d — a Kubernetes-native distributed inference framework — directly into OpenShift AI for enterprise production workloads. In March 2026, IBM, Red Hat, and Google co-donated llm-d and its TPU drivers to the CNCF [9][10][11], moving the framework into neutral governance while it was already shipping in a major enterprise Linux distribution. Google confirmed the CNCF sandbox acceptance on its own Cloud Blog [12], and separately added nightly CI coverage for TPU hardware in llm-d [13], with SemiAnalysis noting TPU has reached code quality parity with NVIDIA on the framework [14]. CoreWeave, a major NVIDIA-centric cloud provider, publicly endorsed llm-d's CNCF acceptance as significant for production inference infrastructure [15]. theCUBE Research has analyzed Red Hat AI 3 as a platform for distributed inference and agentic intelligence [16].

AMD's relationship with the llm-d stack is more nuanced than simple absence. In May 2025, AMD and Red Hat announced a strengthened strategic collaboration [17] — a formal partnership that predates Red Hat AI 3's October 2025 launch. This makes AMD's absence from llm-d's supported hardware list [18] a transitional gap within an active partnership rather than a clean exclusion. The Red Hat AI 3 hardware configurations documentation [19] is the authoritative reference for exactly which AMD hardware, if any, is currently certified. AMD also contributed upstream to NVIDIA's AIPerf benchmarking sub-project within the Dynamo repository [20] — believed to be the first cross-competitor open-source merge — suggesting shared interest in measurement infrastructure even as hardware and pricing strategies diverge sharply.

NVIDIA's supply picture adds a structural dimension that complicates the standard incumbency narrative. NVIDIA's B200 and GB200 are sold out through mid-2026 with a backlog of 3.6 million units [21], and enterprise buyers face 16–24 month wait times for GB200 systems [22]. This supply constraint is double-edged: it confirms extraordinary demand, sustaining the incumbency narrative Jensen Huang articulated with his $1 trillion GPU purchase order projection through 2027 [23] and his 'low MFU by design' philosophy that frames over-provisioned capacity as strategic flexibility [24]. But enterprises that need AI inference capacity now face a genuine substitution decision. AMD's cost advantage [5] and Google TPU's availability through Google Cloud both become practically relevant when NVIDIA hardware is simply unavailable. The competitive picture in mid-2026 is thus multi-layered: AMD leads on published inference cost efficiency and holds a formal Red Hat partnership but awaits hardware validation on the enterprise inference stack; Google has achieved CI parity on llm-d and occupies the governing coalition; and NVIDIA holds both extraordinary projected demand and a supply backlog that simultaneously validates its position and opens a substitution window.