Version 9

NVIDIA has held or grown AI compute market share against custom ASICs in mid-2026 [1], while its NVLink Fusion ecosystem spans foundries, chip designers, and connectivity vendors [7][9]. OpenAI's Jalapeño, an LLM inference chip built with Broadcom in nine months, entered production targeting gigawatt-scale deployment with Microsoft starting in 2026, claiming better performance-per-watt and lower serving costs than current accelerators — claims without independent benchmarks [10][11][12]. Inference startup Etched drew technical attention for a cluster-scale memory (CSM) architecture; Andrej Karpathy praised its extreme low-voltage engineering for maximizing tokens-per-watt at interactive speeds [18], while SemiAnalysis noted the high SerDes count the design requires [17].

The central question — whether software ecosystem depth or workload-specific hardware design delivers better AI compute economics — is now being tested by multiple credible actors simultaneously: OpenAI with captive model workloads, Etched with a technically novel memory architecture that has attracted credible endorsement, and NVIDIA with compounding software optimization that cut serving costs 2.5x in under 70 days [4]. Outcomes will determine whether CUDA's software moat holds as the dominant structural advantage or whether hardware-first approaches can accumulate enough workload ownership to overcome it.

For roughly two years, the dominant expectation in AI compute markets was that custom ASICs would gradually absorb workloads from NVIDIA GPUs. Mid-2026 data has not confirmed that prediction: NVIDIA has held or grown its market share against ASICs [1], and CEO Jensen Huang has publicly projected continued rapid growth [2]. The analytical explanation that has gained the most traction centers on software. SemiAnalysis argues that AI chip competition is decided by software ecosystem depth, not hardware benchmarks — that 99% of chip startups publish impressive simulated numbers but fail in production because building a competitive software stack is the actual hard problem, one that NVIDIA's CUDA platform has largely solved over two decades [3]. A concrete data point supports this argument: serving costs on NVIDIA's GB200 NVL72 dropped 2.5x in under 70 days through CUDA kernel rewrites applied to the Kimi model architecture [4], illustrating software-layer compounding that hardware-first competitors cannot replicate on short timelines.

NVIDIA has structured custom silicon as a participant in its own ecosystem rather than an external competitor. NVLink Fusion allows hyperscalers and chip designers to integrate non-NVIDIA compute into NVIDIA-led clusters [5]. The ecosystem spans Marvell (hyperscaler ASICs), SiFive (RISC-V AI data center chips) [6], Samsung Foundry as a manufacturing partner [7], Astera Labs (connectivity silicon) [8], and MediaTek [9], covering design, manufacturing, and connectivity layers.

On June 24, 2026, OpenAI and Broadcom announced Jalapeño — OpenAI's first custom Intelligence Processor, designed for LLM inference and built from design to tape-out in nine months [10][11]. OpenAI claims better performance per watt than current accelerators in early testing and lower model-serving costs [12], targeting gigawatt-scale deployment with Microsoft starting in 2026 [13]. SemiAnalysis noted the nine-month development timeline requires 'making no mistakes' [14], extending its ASIC skepticism to OpenAI's claim. What differentiates Jalapeño from startup hardware claims is that OpenAI brings captive workloads and model-development depth — exactly the software-stack ownership SemiAnalysis argues most ASIC projects lack. No independent benchmarks have been published. Hardware startups Tensordyne and DeepAdapt have each published large unverified performance claims — 13x rack throughput versus NVIDIA's NVL72 GB300 on DeepSeek-R1 [15] and 82% AI operating cost reduction by shifting workloads from GPUs to CPUs [16] — both based on internal simulations, fitting the benchmark-slide pattern SemiAnalysis describes.

Inference startup Etched has attracted credible technical attention for a distinct architectural approach. Its cluster-scale memory (CSM) creates a shared low-latency memory pool across an entire scale-up domain using a proprietary interconnect, combining HBM and SRAM in a hybrid designed to achieve both high throughput and interactive inference speeds without the tradeoffs of SRAM-only designs or 3D DRAM [17]. Andrej Karpathy publicly praised the engineering approach, describing the design as operating at 'very low voltage domains' with 'cluster scale memory' and framing efficient LLM inference as the electrical inverse of power transmission — very low voltage, high current at tiny distances [18]. SemiAnalysis noted the high SerDes count the architecture requires [17], a technically neutral observation pointing to both the novelty and the complexity of the approach. No independent benchmarks of Etched's system have been published.