Version 3

AMD, Google, IBM, and NVIDIA are competing across hardware benchmarks, software infrastructure, and market economics in mid-2026. AMD's MI355 delivers 40% lower inference cost than NVIDIA's B200 on GLM5 FP8 workloads [1], and AMD has now posted self-described 'breakthrough' results in the standardized MLPerf Inference 6.0 round [2]. Google and IBM jointly donated llm-d and its TPU drivers to the CNCF in March 2026 [6][7], with CoreWeave publicly endorsing the move as significant for production inference [10]. Against this, NVIDIA projects $1 trillion in GPU purchase orders through 2027 [15] — a macro-level demand signal suggesting incumbency advantage. AMD's concrete gap persists: llm-d images still do not support AMD MI300 GPUs [13], leaving it behind both NVIDIA and Google TPU on the Kubernetes-native inference stack enterprise operators increasingly require.

NVIDIA's dominance has rested on raw hardware performance and a mature CUDA software ecosystem. AMD is demonstrating cost-competitive inference hardware and standardized benchmark milestones while pricing below cost to gain share. Google and IBM are constructing a neutral, CNCF-governed Kubernetes inference stack that TPU and NVIDIA have joined but AMD has not yet. Whether AMD can close its software gap before NVIDIA's projected demand wave locks in a new hardware cycle is the central question for enterprise AI infrastructure in 2026.

The competition for AI accelerator dominance entered a multi-front phase in 2026, with AMD, Google, IBM, and NVIDIA each posting concrete moves across hardware economics, benchmark transparency, and software infrastructure governance.

The headline hardware result is AMD's: its MI355 GPU is 40% cheaper than NVIDIA's B200 for single-node FP8 inference on the GLM5 architecture, using speculative decoding via SGLang v0.12 across both MTP and non-MTP configurations [1]. SemiAnalysis, which reported the finding, emphasized it was achieved just 14 weeks after GLM5's initial launch and holds on both CUDA and ROCm backends — framing it as evidence that ROCm's software maturity has crossed a threshold where AMD hardware can realize its cost advantages in production inference. AMD has added a standardized benchmark milestone: the company published what it describes as 'breakthrough' results in the MLPerf Inference 6.0 round [2], situating AMD more formally alongside NVIDIA [3] in the industry's primary hardware benchmarking process and extending the cost-advantage claim beyond a single architecture test. AMD's MI355X carries higher manufacturing costs than NVIDIA's competing chip while selling at a substantially lower price [4], a deliberate margin sacrifice to gain inference market share. One complicating pricing signal: AMD's MI350, an earlier GPU in the same family, has seen its price jump 66.7% amid NVIDIA rivalry [5], suggesting AMD's pricing posture is not uniformly below-cost across its accelerator portfolio and likely reflects demand dynamics on specific SKUs.

On the software infrastructure front, Google and IBM jointly donated llm-d — an open-source Kubernetes-native distributed inference framework — and its TPU drivers to the Cloud Native Computing Foundation in March 2026 [6][7], placing the project under neutral governance and lowering the barrier for enterprise operators to standardize on it. The CNCF sandbox acceptance process for llm-d was formally filed [8], and Google presented GKE and open-source inference innovations at KubeCon EU 2026 [9], deepening the Kubernetes-native inference ecosystem around its hardware. IBM's involvement as a co-originator of the donation is significant: it broadens the coalition behind the neutral-governance inference stack beyond hyperscalers to enterprise IT incumbents. CoreWeave — a major NVIDIA-centric cloud provider — publicly framed llm-d's CNCF acceptance as significant for production inference infrastructure [10], a notable cross-vendor endorsement from a company whose business model is built on NVIDIA hardware. Google further added nightly continuous integration for llm-d on TPU hardware [11] and published 3X inference speedups using diffusion-style speculative decoding [12], adding throughput dimensions beyond Kubernetes tooling. AMD's position on this stack remains a concrete liability: an open GitHub issue confirms llm-d images do not yet support AMD MI300 GPUs [13], creating a specific production blocker while NVIDIA and Google TPU both have active CI coverage.

AMD's upstream contribution to NVIDIA's AIPerf benchmarking sub-project within the Dynamo repository [14] — believed to be the first such cross-competitor upstream merge — signals some willingness to collaborate on shared measurement infrastructure, but the inference-framework gap that matters most to enterprise Kubernetes operators remains unresolved. Against the AMD and Google TPU competitive narrative, Jensen Huang has projected $1 trillion in GPU purchase orders through 2027 [15], a macro-level demand claim suggesting NVIDIA expects its incumbency advantages to sustain at scale regardless of cost-per-token comparisons. Huang has also articulated a 'low MFU by design' philosophy [16], arguing that deliberate GPU over-provisioning is a strategic asset rather than waste — a framing that, if adopted broadly by hyperscalers, would favor capacity incumbents over cost-efficiency challengers and blunt AMD's inference cost advantage.