Version 1

Over five days in mid-May 2026, three separate competitive signals converged: AMD's MI355 GPU achieved 40% lower inference cost than NVIDIA's B200 on the GLM5 architecture just 14 weeks after launch [1]; Google added nightly CI for the llm-d Kubernetes inference framework on its TPU hardware, gaining parity with NVIDIA in code quality and coverage [2]; and AMD made what is believed to be its first accepted upstream contribution to an NVIDIA-owned open-source repository, the AIPerf benchmarking sub-project [3]. Meanwhile, Jensen Huang offered a counterintuitive infrastructure thesis — that low GPU utilization (MFU) is a feature, not a failure [4].

For years NVIDIA's dominance rested on two pillars: raw hardware performance and a mature software ecosystem (CUDA). These developments suggest both pillars are under pressure simultaneously — AMD is demonstrating cost-competitive inference on ROCm, Google is closing the software infrastructure gap for TPUs, and cross-competitor open-source collaboration is beginning to erode the moat-by-fragmentation dynamic that historically benefited NVIDIA.

The competitive landscape for AI accelerators shifted noticeably in the week of May 16–21, 2026, with AMD and Google each posting concrete milestones against NVIDIA's incumbency.

The headline result came from AMD: its MI355 GPU is now 40% cheaper than NVIDIA's B200 when serving the GLM5 architecture on a single node in FP8 precision, covering both MTP and non-MTP configurations with speculative decoding on SGLang v0.12 [1]. SemiAnalysis, which published the finding, emphasized that this result holds on both CUDA and ROCm backends and was achieved just 14 weeks after GLM5's initial launch — framing it as evidence that ROCm's software maturity has reached a point where AMD hardware can realize its cost advantages in production inference settings. The analyst house's tagline for the moment: "SPEED IS THE MOAT."

On the software infrastructure front, Google added nightly continuous integration (CI) for llm-d — an open-source Kubernetes-native distributed inference framework — on its TPU hardware [2]. SemiAnalysis characterized TPU as now catching up to NVIDIA in terms of llm-d CI coverage and code quality, while noting that AMD's official llm-d support lags behind both. Nightly CI is a meaningful signal of production-readiness intent: it means Google is committing engineering resources to keep TPU compatibility green on each code push, rather than treating OSS support as a periodic patch exercise.

Perhaps the most symbolically charged development was AMD submitting — and having accepted — an upstream contribution to AIPerf, a benchmarking sub-repository within NVIDIA's Dynamo project [3]. SemiAnalysis described this as likely the first time a contribution from AMD has been accepted into any NVIDIA-owned repository, calling it "awesome" and historically significant. Cross-competitor open-source collaboration of this kind is rare in the AI hardware space, where software ecosystems have been deliberately siloed; the willingness of both companies to merge such a contribution suggests at least a narrow alignment of interests around shared benchmarking infrastructure.

Running against the competitive-pressure narrative is an observation from NVIDIA's own CEO. Speaking at Stanford's CS153 Frontier Systems course, Jensen Huang stated that he would prefer to operate at low Model FLOP Utilization (MFU) at all times [4]. His reasoning inverts the conventional framing: low MFU is not a sign of waste but of deliberate over-provisioning of compute, networking, and memory — a posture that leaves headroom to absorb spikes. SemiAnalysis drew a pointed comparison to xAI's kernel engineering culture, implying a similar philosophy may be at work there. If this view diffuses broadly, it could reshape how buyers think about GPU procurement — favoring capacity and flexibility over raw utilization efficiency, which would tend to benefit the incumbent with the most headroom to provision.