The Information Machine

NVIDIA vs. Custom ASICs: GPU Dominance Persists Despite Startup Performance Claims · history

Version 8

2026-06-30 08:07 UTC · 101 items

What

NVIDIA has held or grown AI compute market share against custom ASICs as of mid-2026 [1], while its NVLink Fusion ecosystem now spans chip designers, foundries (Samsung Foundry [9]), connectivity silicon (Astera Labs [10]), and consumer chip makers (MediaTek [11]). On June 24, 2026, OpenAI and Broadcom announced Jalapeño — OpenAI's first custom LLM inference chip, built in nine months, claiming better performance per watt and lower model-serving costs than current accelerators, targeting gigawatt-scale deployment with Microsoft starting in 2026 [12][13][14]; it has not been independently benchmarked. SemiAnalysis argues AI chip competition is decided by software ecosystem depth rather than hardware specs [3], and a 2.5x serving cost reduction on NVIDIA's GB200 NVL72 in under 70 days via CUDA kernel rewrites is the clearest production evidence for that claim [4].

Why it matters

OpenAI's entry into custom silicon is structurally different from hardware startup claims: OpenAI owns both the models and the software stack, which is exactly what SemiAnalysis argues strands most ASIC projects. Whether that ownership is sufficient to overcome the software moat is now an empirical question with a real chip in production, not a theoretical one.

Open questions

  • Can Jalapeño's claimed performance-per-watt advantage survive independent benchmarking, or will it follow the benchmark-slide pattern SemiAnalysis describes for hardware startups? [12][3]

  • Does the 2.5x serving cost reduction through CUDA kernel rewrites in under 70 days [4] represent a rate of software improvement that inference-specific ASICs cannot match on near-term timelines?

  • Will Tensordyne's 13x throughput claim [5] or DeepAdapt's 82% cost reduction [6] survive independent testing, given both rest on unverified internal benchmarks?

  • Does NVLink Fusion's expansion to Samsung Foundry and MediaTek [9][11] mean NVIDIA captures value across manufacturing and consumer chip design layers, not just hyperscaler ASIC development?

Narrative

For roughly two years, the dominant expectation in AI compute markets was that custom ASICs would gradually absorb workloads from NVIDIA GPUs. Mid-2026 data has not confirmed that prediction: NVIDIA has held or grown its market share against ASICs [1], and CEO Jensen Huang has publicly projected continued rapid growth [2]. The analytical explanation that has gained the most traction centers on software. SemiAnalysis argues that AI chip competition is decided by software ecosystem depth, not hardware benchmarks — that 99% of chip startups publish impressive simulated numbers but fail in production because building a competitive software stack is the actual hard problem, one that NVIDIA's CUDA platform has largely solved over two decades [3].

A concrete data point supports this argument: serving costs on NVIDIA's GB200 NVL72 dropped 2.5x in under 70 days through CUDA kernel rewrites applied to the Kimi model architecture [4], illustrating software-layer compounding that hardware-first competitors cannot replicate on short timelines. Hardware startups Tensordyne and DeepAdapt have published large performance claims — 13x rack throughput versus NVIDIA's NVL72 GB300 on DeepSeek-R1 [5] and 82% AI operating cost reduction by shifting workloads from GPUs to CPUs [6], respectively — both based on internal simulations without independent validation, fitting the benchmark-slide pattern SemiAnalysis describes.

NVIDIA has structured custom silicon as a participant in its own ecosystem rather than an external competitor. NVLink Fusion allows hyperscalers and chip designers to integrate non-NVIDIA compute into NVIDIA-led clusters [7]. The ecosystem spans Marvell (hyperscaler ASICs), SiFive (RISC-V AI data center chips) [8], Samsung Foundry as a manufacturing partner [9], Astera Labs (connectivity silicon) [10], and MediaTek [11], covering design, manufacturing, and connectivity layers. Milk Road AI frames this as making custom silicon makers ecosystem participants rather than competitors [1][7].

On June 24, 2026, OpenAI and Broadcom announced Jalapeño — OpenAI's first custom Intelligence Processor, designed for LLM inference and built from design to tape-out in nine months [12][13]. OpenAI claims better performance per watt than current accelerators in early testing and lower model-serving costs [14], targeting gigawatt-scale deployment with Microsoft starting in 2026, with OpenAI leading architecture and Broadcom handling silicon [15]. Secondary coverage frames the effort as a move to reduce NVIDIA reliance [16]. SemiAnalysis noted the nine-month development timeline requires 'making no mistakes' [17], extending its ASIC skepticism to OpenAI's claim. What differentiates Jalapeño from startup hardware claims is that OpenAI brings captive workloads and model-development depth — exactly the software-stack ownership SemiAnalysis argues most ASIC projects lack. No independent benchmarks have been published.

Timeline

  • 2026-01-15: SiFive announces integration of NVLink Fusion for next-generation RISC-V AI data center chips. [8]
  • 2026-03: Meta announces expansion of its MTIA custom silicon program to power next-generation AI workloads. [21]
  • 2026-06-14: NVIDIA CEO Jensen Huang states publicly that next year's growth will remain rapid. [2]
  • 2026-06-15: Analysis published on China's growing supply chain independence in AI hardware, covering compound semiconductors and Huawei's architecture work. [23]
  • 2026-06-17: Tensordyne announces an inference rack claiming 13x rack throughput versus NVIDIA's NVL72 GB300 on DeepSeek-R1, based on internal simulations. [5][24]
  • 2026-06-18: Milk Road AI publishes analysis arguing data shows NVIDIA has held or grown market share against ASICs, contrary to two years of consensus predictions. [1]
  • 2026-06-19: SemiAnalysis states that 99% of custom ASIC projects fail and that AI chip success is determined by software depth, not hardware specifications. [3]
  • 2026-06-19: Jensen Huang appears at a Marvell keynote framing NVLink Fusion as enabling custom silicon coexistence within NVIDIA-led clusters. [7]
  • 2026-06-19: Amazon reported to be entering the AI chip market. [22]
  • 2026-06-19: DeepAdapt claims its runtime layer cuts AI operating costs by 82% and delivers 33x faster inference by shifting workloads from GPUs to CPUs, without independent validation. [6]
  • 2026-06-21: Samsung Foundry added to NVIDIA's NVLink Fusion ecosystem as a manufacturing partner for custom silicon. [9]
  • 2026-06-22: SemiAnalysis documents 2.5x serving cost reduction on GB200 NVL72 in under 70 days via CUDA kernel rewrites. [4]
  • 2026-06-24: OpenAI and Broadcom announce Jalapeño, OpenAI's first custom LLM inference chip developed in nine months, claiming better performance per watt than current accelerators; OpenAI led architecture, Broadcom handled silicon. [12][13][19][15]
  • 2026-06-24: SemiAnalysis notes the Jalapeño nine-month development timeline with the observation that it requires 'making no mistakes.' [17]

Perspectives

NVIDIA / Jensen Huang

Projects continued rapid growth; NVLink Fusion positions custom silicon makers as ecosystem participants rather than competitors, with NVIDIA as the interconnect backbone of AI compute.

Evolution: Consistently bullish; NVLink Fusion has expanded from a few large hyperscalers to chip designers, foundries, consumer silicon makers, and connectivity vendors.

[2][7][18]

OpenAI / Broadcom

Jalapeño is OpenAI's first custom Intelligence Processor for LLM inference, co-developed with Broadcom in nine months; early tests claim better performance per watt and lower serving costs than current accelerators, targeting gigawatt-scale deployment with Microsoft in 2026.

Evolution: First appeared June 24; secondary coverage now explicitly frames the effort as NVIDIA dependency reduction; no independent benchmarks yet.

[12][13][19][15][14][16]

SemiAnalysis

AI chip competition is decided by software ecosystem depth; 99% of ASIC projects fail in production regardless of simulation results. The 2.5x serving cost reduction via CUDA rewrites in 70 days is direct production evidence; Jalapeño's nine-month timeline requires 'making no mistakes.'

Evolution: Consistent skeptic of hardware-first ASIC narratives; June 2026 CUDA cost-reduction data sharpens the software moat argument; the Jalapeño comment extends that skepticism to OpenAI's timeline claim.

[3][4][17]

Milk Road AI

Data shows NVIDIA has held or grown market share against ASICs; NVLink Fusion makes custom silicon makers ecosystem participants rather than competitors.

Evolution: Contrarian on ASIC displacement; consistently reframes competition as coexistence via NVLink Fusion.

[1][7][20]

NVLink Fusion ecosystem (Marvell, SiFive, Samsung Foundry, MediaTek, Astera Labs)

Chip designers, foundries, and connectivity silicon makers treat NVLink Fusion as shared infrastructure, operating within NVIDIA's interconnect framework rather than against it.

Evolution: Expanded from Marvell alone to a set spanning manufacturing (Samsung), consumer chip design (MediaTek), RISC-V AI chips (SiFive), and connectivity (Astera Labs).

[20][7][9][10][8][11]

Meta / Amazon (hyperscalers)

Both treating custom silicon as critical for scaling AI workloads — Meta expanding MTIA, Amazon entering the AI chip market — with captive workloads and engineering scale that differs structurally from hardware-first startups.

Evolution: Deepening commitment to custom silicon through 2026; structurally distinct from startups in their ability to develop accompanying software.

[21][22]

Tensordyne / DeepAdapt

Both claim large performance advantages over NVIDIA-centric inference (13x throughput and 82% cost reduction respectively), based on internal simulations without independent validation.

Evolution: Fit the benchmark-slide pattern SemiAnalysis describes; neither claim has been independently verified.

[5][6]

Sky Rain

China's positions in compound semiconductors and Huawei's architecture work are building a supply chain track that operates outside the NVIDIA-dominated ecosystem.

Evolution: Consistent focus on China supply chain independence as a structurally distinct competitive thread.

[23]

Tensions

  • SemiAnalysis argues the CUDA software moat is real and deepening — 2.5x serving cost reduction in 70 days via kernel rewrites — while OpenAI claims Jalapeño delivers better performance per watt through workload-specific hardware design; SemiAnalysis's 'making no mistakes' comment signals skepticism of OpenAI's timeline. [4][12][17]
  • SemiAnalysis argues AI chip competition is decided by software depth and 99% of ASIC projects fail in production, while Tensordyne and DeepAdapt publish large unverified numbers framed as improvements over NVIDIA hardware. [3][5][6]
  • Milk Road AI and NVIDIA argue the 'NVIDIA versus everyone' frame is wrong — NVLink Fusion makes custom silicon a coexistent participant; SemiAnalysis supports NVIDIA's resilience through a different mechanism: software moat, not ecosystem embrace. [7][1][3]
  • Milk Road AI argues mid-2026 data shows NVIDIA holding or growing market share, directly contradicting two years of consensus predictions that custom silicon would erode NVIDIA's position. [1]
  • The software moat argument holds that startups cannot overcome CUDA's depth; Meta, Amazon, and OpenAI — organizations with captive workloads and engineering scale — test whether sufficient resources can develop the required software stack alongside custom hardware. [3][21][22][12]

Sources

  1. [1] Everyone assumed Nvidia would get crushed by ASICs but the data says the opposite just happened and the reason why chang… — Milk Road AI Twitter (2026-06-18)
  2. [2] 🚨 $NVDA -NVIDIA CEO JENSEN HUANG: NEXT YEAR’S GROWTH WILL REMAIN RAPID 🚀🧠 — reactive:asic-gpu-market-dynamics (2026-06-14)
  3. [3] 100% of AI chip startups have slides/“simulated performance data” showing that their chip is way better, but 99% of cust… — SemiAnalysis Twitter (2026-06-19)
  4. [4] CUDA MOAT ALERT 🔥: In less than 70 days, GB200 NVL72 serving costs decreased by 2.5x through software improvements alone… — SemiAnalysis Twitter (2026-06-22)
  5. [5] Quite a massive inferencing rack breakthrough from @TensordyneInc . — Rohan Paul Twitter (2026-06-17)
  6. [6] DeepAdapt has launched a runtime intelligence layer that cuts AI operating costs by up to 82% and 33X faster inference b… — Rohan Paul Twitter (2026-06-19)
  7. [7] Jensen Huang showed up at a Marvell keynote to explain why connectivity matters (Save this). — Milk Road AI Twitter (2026-06-19)
  8. [8] SiFive To Integrate Nvidia NVLink Fusion For Next-Gen AI Data Centers — reactive:asic-gpu-market-dynamics
  9. [9] NVIDIA Adds Samsung Foundry to NVLink Fusion Ecosystem for Custom Silicon Manufacturing : r/hardware — reactive:asic-gpu-market-dynamics
  10. [10] Why Connectivity is the New Frontier of AI Infrastructure—and What NVLink Fusion Means for the Future — reactive:asic-gpu-market-dynamics
  11. [11] MediaTek | NVLink Fusion | Custom AI ASIC Innovation — reactive:asic-gpu-market-dynamics
  12. [12] OpenAI and Broadcom unveil LLM-optimized inference chip — OpenAI Blog (2026-06-24)
  13. [13] OpenAI and Broadcom announce chip designed for LLM inference at scale — Ars Technica AI (2026-06-24)
  14. [14] 🟡 Hollywood's AI trick work — Semafor Technology (2026-06-26)
  15. [15] OpenAI and Broadcom unveiled first chip Jalapeno for LLM inference. OpenAI led architecture, Broadcom handled silicon an... — reactive:asic-gpu-market-dynamics (2026-06-25)
  16. [16] OpenAI unveils "Jalapeno" custom AI chip to reduce Nvidia reliance, promising superior inference efficiency and 2026 dep... — reactive:asic-gpu-market-dynamics (2026-06-26)
  17. [17] Chat develop a chip from initial design to tape out in 9 months, make no mistakes. https://t.co/DaNCCPJebw — SemiAnalysis Twitter (2026-06-24)
  18. [18] Build Semi-Custom AI Infrastructure | NVIDIA NVLink Fusion — reactive:asic-gpu-market-dynamics
  19. [19] Broadcom and OpenAI unveil custom-built Jalapeño inference ... — reactive:asic-gpu-market-dynamics
  20. [20] In 2016, Marvell's largest design win was a Wi-Fi chip in the Barbie Dream House (Save this). — Milk Road AI Twitter (2026-06-20)
  21. [21] Expanding Meta's Custom Silicon to Power Our AI Workloads — reactive:asic-gpu-market-dynamics
  22. [22] 🚨 AMAZON IS ENTERING THE AI CHIP MARKET — reactive:asic-gpu-market-dynamics (2026-06-19)
  23. [23] China’s Quiet Dominance in the AI Supply Chain: InP, Semiconductor Independence, and Huawei’s Logic Folding Wildcard — reactive:asic-gpu-market-dynamics (2026-06-15)
  24. [24] Tensordyne Announces Breakthrough Inference System - LinkedIn — reactive:asic-gpu-market-dynamics