The Information Machine

Agentic Workloads Rewriting LLM Inference Economics · history

Version 5

2026-05-24 19:01 UTC · 136 items

What

Agentic AI workloads are generating token volumes far exceeding industry planning assumptions — empirical analysis of 432k coding-agent requests finds a median input of 96k tokens with roughly half exceeding 128k.[1][2] The engineering response has matured into three documented technical pillars (tiered KV cache management, IO-aware attention optimization, speculative decoding), with NVIDIA adding a fourth hardware-level approach — SRAM-Decode — unveiled at GTC 2026.[21] A significant factual clarification has emerged on the NVIDIA-Groq deal: Groq's official press release describes it as a 'non-exclusive inference technology licensing agreement,'[22] while multiple analyst sources continue to frame it as a $20B acquisition.[23][24] Gartner's primary forecast that inference costs will fall over 90% by 2030[29] is now confirmed at the source, yet a dedicated analysis frames why total enterprise bills keep rising regardless — a Jevons paradox dynamic that may be the defining economic constraint of the agentic era.[33]

Why it matters

The central tension is that per-token inference prices are falling sharply while total enterprise AI bills keep climbing — efficiency gains are absorbed by increased agentic consumption, not banked as savings.[33][29] Whether the NVIDIA-Groq deal is a licensing arrangement or de facto acquisition determines the shape of the specialist inference hardware market, which in turn shapes whether cost mitigation remains competitive or consolidates under NVIDIA's control.[22][26]

Open questions

  • Is the NVIDIA-Groq deal a non-exclusive licensing arrangement[22] or effectively a de facto acquisition as framed by analysts at $20B?[23][24] The distinction is material for Cerebras's competitive positioning and the independence of specialist inference silicon — Cerebras bumped up its IPO range but faced skepticism about whether it can credibly challenge NVIDIA's ecosystem.[26][27]

  • Can speculative decoding's claimed 3× throughput gains[17] be sustained at the 96k+ token contexts dominating agentic workloads, where draft-model acceptance rates may fall as context complexity grows?[15][19]

  • Agentic AI project failure rates are reported variously at 40%[35] and 90–95%[36] — how much of that failure rate is attributable to hidden inference cost overruns (tool call overhead, retry loops, context window consumption[37][39]) vs. other factors, and what distinguishes the deployments that survive?

  • Does the Jevons paradox hold at the enterprise level? Gartner officially confirms inference costs will fall over 90% by 2030,[29] but the 'Inference Cost Paradox' analysis argues cheaper tokens drive more agentic deployment, keeping total bills elevated regardless of efficiency gains.[33]

Narrative

Empirical analysis of 432k coding-agent requests by SemiAnalysis found the median input token count is 96k — roughly the length of a novel — with approximately half of all requests exceeding 128k tokens.[1][2] An independently published academic paper at OpenReview examines the same question from a research angle, studying how coding agents allocate token expenditure across task steps and building predictive models for consumption.[3] The structural driver is not user verbosity but the scaffolding agents assemble automatically before a user types a word: system prompts, tool and skill definitions, MCP schemas, and the rolling context of prior conversation turns and file contents agents must carry to stay coherent. A community observer responding to the SemiAnalysis data captured the dynamic precisely: "the 'task' is often a giant read of the codebase before any actual work begins."[4] A separate observer framed it as a memory problem the market systematically underprices.[5]

The infrastructure response has matured into four documented technical approaches. The first is tiered KV cache management: when a model processes a 96k-token request, the key-value tensors overflow GPU HBM at scale,[6] and the mitigation is spilling to slower memory tiers. NVIDIA's Dynamo framework productizes hierarchical offloading (HBM → DRAM → NVMe),[7] the Tutti paper documents making SSD-backed KV cache practical for long-context serving,[8] and Dell has published an enterprise storage approach to the same offloading problem.[9] Open-source stacks including vLLM, SGLang, and lmdeploy are implementing KV cache quantization in INT4, INT8, and FP8 formats to reduce memory footprints.[10][11][12] The second pillar is IO-aware attention optimization: at long contexts, attention computation — not just storage — becomes a primary cost driver, and algorithms like FlashAttention restructure computation to minimize HBM reads/writes, with 1.4–1.7× wall-clock speedups observed at 98k tokens.[13][14] The third pillar is speculative decoding, where a small draft model proposes tokens that the full model verifies in parallel; practitioners report throughput gains up to 3×, and deployment guides from Together.ai, NVIDIA, AWS, Snowflake, BentoML, and Nebius document approaches for long-context and decode-heavy workloads.[15][16][17][18][19][20] A fourth hardware-level approach emerged at GTC 2026: NVIDIA's SRAM-Decode architecture specifically targets the decode phase of inference, distinct from KV cache tiering.[21]

At the competitive level, the NVIDIA-Groq relationship has been clarified by Groq's own press release, which describes the deal as a 'non-exclusive inference technology licensing agreement to accelerate AI inference at global scale.'[22] Multiple analyst and media sources continue to frame the deal as a $20B acquisition or IP consolidation,[23][24][25] creating a direct tension between official and secondary characterizations. The non-exclusive framing matters: a licensing deal leaves Groq as an independent operator free to license its LPU technology to others, which is a materially different competitive outcome than absorption into NVIDIA's portfolio. Cerebras, the other major specialist inference hardware player, bumped up its IPO valuation range heading into its May 2026 pricing, but market observers characterized the offering as having 'too much hair' as Cerebras attempts to sell Wall Street on a credible NVIDIA alternative.[26][27] NextPlatform frames Cerebras's post-IPO strategy as a return to pushing inference performance limits independent of NVIDIA's ecosystem.[28]

The macro-level economic tension is now well-documented at the primary source level. Gartner's official March 2026 press release predicts that performing inference on a trillion-parameter LLM will cost providers over 90% less by 2030 than in 2025,[29] a forecast widely amplified across industry outlets.[30][31][32] A dedicated blog post titled 'The Inference Cost Paradox' frames the countervailing dynamic: as models get cheaper per token, enterprises deploy more agents and more complex pipelines, such that total AI bills keep rising even as unit costs fall — a Jevons paradox at the enterprise level.[33] McKinsey separately estimates a $7 trillion race to scale data centers, providing a capital investment backdrop to the same infrastructure scaling dynamic.[34] Analyst estimates of agentic project failure rates now range from 40% (Galileo and Trullion) to 90–95% (Beam AI),[35][36] and multiple practitioner analyses specifically enumerate hidden cost structures — tool call overhead, retry loops, and context window consumption — that are not visible at the pilot stage but cause cost overruns when projects reach production scale.[37][38][39]

Timeline

  • 2026-03-25: Gartner officially forecasts that inference costs for trillion-parameter LLMs will fall over 90% by 2030 compared to 2025; forecast amplified across multiple industry outlets [29][30][31][32]
  • 2026-04-14: Blog post 'The Inference Cost Paradox' frames the Jevons paradox dynamic for inference: cheaper tokens drive more agentic deployment, keeping total enterprise bills elevated regardless of per-unit improvements [33]
  • 2026-04-24: DeepSeek V4 released, described as best open-source model for coding tasks [61]
  • 2026-04-27: GitHub Copilot announces move to usage-based billing and retirement of annual plans [47]
  • 2026-05-14: Cerebras prices its IPO; analysts cover it as a direct NVIDIA rival play for specialized inference hardware; Cerebras bumps up its valuation range but faces skepticism about whether it can credibly challenge NVIDIA's ecosystem [48][51][52][53][26][27]
  • 2026-05-15: NextPlatform covers Cerebras's post-IPO strategy as a return to pushing AI inference performance limits independent of NVIDIA [28]
  • 2026-05-16: Researchers note attention computation is the primary cost driver at long context, with 1.4–1.7× wall-clock speedups observed at 98k tokens when attention is IO-optimized [13][14][64]
  • 2026-05-19: Community observers flag context memory as AI infrastructure's emerging bottleneck [75][60]
  • 2026-05-20: Observers note token-maxing creates qualitatively different constraints than prior bandwidth bottlenecks [77]
  • 2026-05-21: SemiAnalysis highlights KV cache overflow beyond GPU HBM as the central scaling constraint for agentic and long-context workloads [6]
  • 2026-05-22: SemiAnalysis publishes analysis of 432k coding-agent requests: median input is 96k tokens, ~50% exceed 128k, driven by agentic prefill context assembly [1][2]
  • 2026-05-22: Community observers amplify SemiAnalysis findings, framing agentic inference as primarily a memory problem the market systematically underprices [4][5]
  • 2026-05-22: SemiAnalysis predicts proliferation of fast-tier pricing, specialized inference chips, and KV cache management as next competitive frontier [40]
  • 2026-05-22: Observers frame the shift as AI entering a 'cloud economics' phase where serving efficiency rivals model capability [76]
  • 2026-05-23: Ben Thompson (Stratechery) publishes 'The Inference Shift,' bringing mainstream tech-strategy analysis to the inference economics debate [55]
  • 2026-05-23: Observer surfaces Gartner projection that inference costs will fall 90% by 2030 but notes total enterprise AI bills are rising — framing a Jevons paradox dynamic [74]
  • 2026-05-24: Groq publishes official press release describing its NVIDIA deal as a 'non-exclusive inference technology licensing agreement,' correcting earlier Reddit-sourced acquisition framing [22]
  • 2026-05-24: Multiple analyst and media outlets frame the NVIDIA-Groq deal as a $20B acquisition or IP consolidation, in direct tension with Groq's official licensing characterization [23][24][25]
  • 2026-05-24: GTC 2026 preview coverage highlights NVIDIA's SRAM-Decode architecture as a new hardware-level approach targeting inference decode-phase efficiency [21]

Perspectives

SemiAnalysis

Data-driven and prescriptive: empirical evidence from 432k requests shows real-world agentic token consumption far exceeds industry assumptions; the next competitive frontier is serving infrastructure and economics, not model intelligence; KV cache management and fast-tier pricing will define winners; model labs are best positioned to capture value from the transition

Evolution: Consistent and intensifying — built progressively from KV cache as bottleneck to specific usage statistics to market-structure predictions; academic work at OpenReview independently examining coding-agent token consumption provides corroboration

[6][40][1][2][41][42][43][3]

NVIDIA

Infrastructure solution provider across multiple layers: tiered KV cache architectures (HBM → DRAM → NVMe) via Dynamo, Blackwell hardware for the agentic cost model, speculative decoding documentation, and SRAM-Decode architecture at GTC 2026 targeting the inference decode phase specifically; the Groq licensing deal extends NVIDIA's reach into LPU-class inference technology on a non-exclusive basis

Evolution: Expanding — SRAM-Decode is a new hardware-level technique not present in previous passes, adding a fourth approach to NVIDIA's inference optimization portfolio; the Groq relationship is now officially characterized as licensing rather than acquisition, which is a softer but still significant competitive extension

[44][7][6][16][45][21][22][46]

Groq

The NVIDIA deal is a 'non-exclusive inference technology licensing agreement to accelerate AI inference at global scale' — Groq retains independence, the arrangement is not exclusive, and Groq remains free to license its LPU technology to other parties

Evolution: New voice with direct official statement — previous synthesis relied on secondhand Reddit reports of an 'acquisition'; Groq's press release materially changes the framing of the competitive situation

[22]

GitHub / Microsoft

Usage-based billing is the right model for agentic AI — retiring flat annual plans in favor of per-token or per-use pricing to align revenue with actual consumption

Evolution: Action-oriented and consistent; enterprise analyst community has independently confirmed the same dynamic with 'sticker shock' framing

[47]

Cerebras

Specialized inference hardware is ready for public markets and independent operation — the IPO is a bet that fast, cost-efficient inference silicon will capture durable market share as agentic workloads scale; Cerebras bumped its IPO range upward, though market observers note the offering carries significant uncertainty as it competes against NVIDIA's ecosystem

Evolution: Position is consistent; competitive context has partially clarified — if Groq's deal with NVIDIA is truly non-exclusive licensing rather than acquisition, Cerebras faces a different landscape than if NVIDIA had absorbed Groq entirely, potentially preserving more independent specialist hardware competition

[48][49][50][51][52][53][54][28][26][27]

Ben Thompson / Stratechery

The inference economics transition deserves mainstream tech-strategy framing — 'The Inference Shift' positions this as a structural market transition, not merely an engineering optimization problem

Evolution: Consistent; Thompson's engagement signals the inference economics story has crossed from specialist infrastructure discourse into mainstream technology strategy analysis

[55][56][57][58][59]

Open-source / practitioner community

Four technical approaches now address the cost problem: KV cache quantization (INT4/INT8/FP8) in vLLM, SGLang, lmdeploy; IO-aware attention algorithms like FlashAttention; speculative decoding with practitioners reporting 3× throughput gains; and NVIDIA's SRAM-Decode hardware architecture targeting decode-phase efficiency. Small models with structured guardrails can reach near-frontier agentic accuracy at far lower cost. SSD-backed KV cache is becoming practically deployable.

Evolution: Expanding — SRAM-Decode from GTC 2026 adds a hardware-level technique to the practitioner toolkit alongside three prior software/algorithmic pillars

[60][61][62][63][10][11][12][14][64][15][16][65][17][66][18][19][20][67][8][21]

Enterprise / analyst observers (HFS Research, CX Today, Galileo, Trullion, Beam AI, Quantiphi)

Agentic AI deployments carry hidden costs and complexity that cause 40–95% of projects to fail before reaching production; organizations face 'sticker shock' when moving from pilot to scale; TCO framing — not per-token pricing — is the correct lens; hidden cost traps include tool call overhead, retry loops, and context window consumption that are invisible at the pilot stage

Evolution: Expanding — failure rate estimates now range from 40% (Galileo/Trullion) to 90–95% (Beam AI), and multiple new practitioner analyses specifically enumerate hidden cost structures, adding operational specificity to the 'sticker shock' framing

[68][69][70][71][35][36][37][38][39]

Macro analysts (Gartner, McKinsey, Tianpan.co)

Gartner's official primary forecast (March 2026) confirms inference costs for trillion-parameter LLMs will fall over 90% by 2030; McKinsey estimates a $7 trillion race to scale data centers; independent blog analysis frames 'The Inference Cost Paradox' — cheaper inference per token drives more agentic usage, keeping total enterprise bills elevated in a Jevons paradox dynamic

Evolution: Gartner's 90% forecast now has a primary source citation rather than only secondhand amplification; McKinsey's capital investment estimate adds a data center infrastructure dimension; the Jevons paradox framing has moved from community observation to dedicated analytical treatment

[33][72][29][30][31][73][32][34][74]

Infrastructure observers (bitfid, Crypto Exponentials, Sahil/TalkinIdeas, 38twelveDaily)

Context memory is the new bottleneck; AI is entering a cloud-economics phase where unit economics and serving efficiency matter as much as capability; token-maxing is a qualitatively different constraint than prior bandwidth bottlenecks; the Jevons paradox dynamic means falling per-token prices do not translate to falling total bills

Evolution: Consistent — the Jevons paradox framing they introduced is now corroborated by Gartner's primary source data and the dedicated 'Inference Cost Paradox' analysis

[75][76][77][74]

Tensions

  • Groq's official press release characterizes the NVIDIA deal as a 'non-exclusive inference technology licensing agreement,' while multiple analyst and media sources frame it as a $20B acquisition or IP consolidation — the distinction is material for Cerebras's competitive positioning and the independence of specialist inference hardware market structure [22][23][24][25][26][27]
  • Tiered KV cache offloading (NVIDIA's Dynamo, Dell storage engines, Tutti SSD-backed paper) vs. the fundamental latency cost: spilling to slower memory tiers reduces capacity pressure but degrades time-to-first-token — whether the tradeoff is acceptable for interactive agentic workloads remains unresolved [6][7][8][9]
  • Prompt caching as a near-term cost fix (practitioners report 40–90% savings) vs. the agentic hit-rate problem: agentic workloads mutate context so frequently that cache hit rates may be systematically low in exactly the use cases that need savings most [62][78][63]
  • Large frontier models consuming 96k+ token contexts vs. small models with guardrails achieving near-equivalent agentic accuracy at far lower cost — cost pressure may bifurcate the market rather than resolve toward one architecture [2][60][79]
  • Model labs capturing disproportionate value from the agentic shift (SemiAnalysis's AI Value Capture thesis) vs. specialized inference hardware vendors (Cerebras, and NVIDIA via the Groq licensing arrangement) and cloud infrastructure providers who control the serving bottleneck [41][40][48][51][45][22]
  • Falling per-token prices (Gartner: -90% by 2030) vs. rising total enterprise AI bills — the Jevons paradox where efficiency gains are absorbed by increased agentic usage, leaving organizations facing higher total spend even as unit economics improve [29][33][68][69][70][36]
  • Speculative decoding's promise of 3× throughput gains vs. its applicability to agentic long-context workloads: the technique depends on draft-model acceptance rates that may degrade as context grows complex, limiting gains precisely where agentic workloads are most expensive [17][15][19]

Sources

  1. [1] Even more striking: ~50% of requests already exceed 128k tokens. The driver isn't user prompts getting longer. It's ever… — SemiAnalysis Twitter (2026-05-22)
  2. [2] Agentic workloads are quietly rewriting inference economics. We pulled data from 432k real coding agent requests at Semi… — SemiAnalysis Twitter (2026-05-22)
  3. [3] How Do Coding Agents Spend Your Money? Analyzing and Predicting Token Consumptions in Agentic Coding Tasks | OpenReview — reactive:agentic-inference-economics
  4. [4] @SemiAnalysis_ This is the inference-economics part of agents that gets under-discussed: the 'task' is often a giant rea... — reactive:agentic-inference-economics (2026-05-22)
  5. [5] SemiAnalysis put a hard number on something the market keeps underpricing: agentic inference is mostly a memory problem ... — reactive:agentic-inference-economics (2026-05-22)
  6. [6] With modern agentic workloads and long context windows, a common bottleneck in serving LLMs at scale is where to store a… — SemiAnalysis Twitter (2026-05-21)
  7. [7] How to Reduce KV Cache Bottlenecks with NVIDIA Dynamo — reactive:agentic-inference-economics
  8. [8] Tutti: Making SSD-Backed KV Cache Practical for Long-Context LLM Serving — reactive:agentic-inference-economics
  9. [9] Scalable KV Cache Offloading with Dell AI Storage Engines — reactive:agentic-inference-economics
  10. [10] Add INT8 Support for KV Cache Quantization (Currently FP8-Only ... — reactive:agentic-inference-economics
  11. [11] INT4/INT8 KV Cache — lmdeploy - Read the Docs — reactive:agentic-inference-economics
  12. [12] Quantized KV Cache - SGLang Documentation — reactive:agentic-inference-economics
  13. [13] @NousResearch Attention is the primary cost driver at long context. 1.4-1.7x wall-clock speedup at 98K doesn't just cut ... — reactive:agentic-inference-economics (2026-05-16)
  14. [14] FlashAttention | LLM Inference Handbook — reactive:agentic-inference-economics
  15. [15] Speculative decoding for high-throughput long-context inference — reactive:agentic-inference-economics
  16. [16] An Introduction to Speculative Decoding for Reducing ... — reactive:agentic-inference-economics
  17. [17] Get 3× Faster LLM Inference with Speculative Decoding Using the ... — reactive:agentic-inference-economics
  18. [18] Fastest Speculative Decoding in vLLM with Arctic Inference and ... — reactive:agentic-inference-economics
  19. [19] Why large MoE models break latency budgets and what speculative decoding changes in production systems — reactive:agentic-inference-economics
  20. [20] Accelerating decode-heavy LLM inference with speculative ... - AWS — reactive:agentic-inference-economics
  21. [21] GTC 2026 Preview | Implications of Nvidia's SRAM-Decode ... — reactive:agentic-inference-economics
  22. [22] Groq and Nvidia Enter Non-Exclusive Inference Technology Licensing Agreement to Accelerate AI Inference at Global Scale | Groq is fast, low cost inference. — reactive:agentic-inference-economics
  23. [23] Nvidia's $20B Groq Acquisition: Why It Paid 2.9x Valuation for LPU Tech | IntuitionLabs — reactive:agentic-inference-economics
  24. [24] NVIDIA’s $20 Billion ‘Shadow Merger’: How the Groq IP Deal Cemented the Inference Empire — reactive:agentic-inference-economics
  25. [25] NVIDIA Groq Deal: How a $20B AI Patent Strategy Is ... - Lexology — reactive:agentic-inference-economics
  26. [26] NVDA Rival Cerebras Bumps Up Its IPO Range, Targets ... — reactive:agentic-inference-economics
  27. [27] Cerebras IPO has ‘too much hair’ as AI chipmaker tries to sell Wall Street on Nvidia alternative – NBC New York — reactive:agentic-inference-economics
  28. [28] With Its IPO Done, Cerebras Can Get Back To Pushing The AI ... — reactive:agentic-inference-economics
  29. [29] Gartner Predicts That by 2030, Performing Inference on an LLM With ... — reactive:agentic-inference-economics
  30. [30] AI inference costs set to plunge: Gartner | Channel Dive — reactive:agentic-inference-economics
  31. [31] Gartner Forecasts 90% Drop in LLM Inference Costs by 2030 - AIwire — reactive:agentic-inference-economics
  32. [32] LLM inference costs to fall 90% by 2030 (Gartner)—what it means for Cloud providers — reactive:agentic-inference-economics
  33. [33] The Inference Cost Paradox: Why Your AI Bill Goes Up as Models ... — reactive:agentic-inference-economics
  34. [34] The cost of compute: A $7 trillion race to scale data centers - McKinsey — reactive:agentic-inference-economics
  35. [35] Why over 40% of agentic AI projects will fail – and which will survive — reactive:ai-demand-bubble-debate
  36. [36] Agentic AI: Why 95% Fail & How to Be the 10% That Succeed — reactive:agentic-inference-economics
  37. [37] The Hidden Cost Structure of Agentic AI: A Practical Guide for ... — reactive:agentic-inference-economics
  38. [38] The Hidden Agentic AI Tax - understand the true costs of autonomy — reactive:agentic-inference-economics
  39. [39] The Hidden Cost Traps Lurking in Agentic AI Projects - ChannelE2E — reactive:agentic-inference-economics
  40. [40] Inference economics are shifting. Expect more "fast tier" pricing (Opus Fast, Gemini Flash), more specialized inference … — SemiAnalysis Twitter (2026-05-22)
  41. [41] AI Value Capture - The Shift To Model Labs - SemiAnalysis — reactive:ai-demand-bubble-debate
  42. [42] The Coding Assistant Breakdown: More Tokens Please - SemiAnalysis — reactive:agentic-inference-economics
  43. [43] Claude Code Psychosis: How SemiAnalysis Is Token Mogging Meta | Ep. 008 — reactive:agentic-inference-economics
  44. [44] Agentic AI Cost Model: NVIDIA Blackwell, Dynamo & Tokens — reactive:agentic-inference-economics
  45. [45] Nvidia acquired Groq, but why not Cerebras? Cerebras is 3x times ... — reactive:agentic-inference-economics
  46. [46] NVIDIA Groq 3 LPU Explained: How the Non-GPU Inference Chip ... — reactive:agentic-inference-economics
  47. [47] GitHub Copilot is moving to usage-based billing and retiring annual plans — reactive:agentic-inference-economics (2026-04-27)
  48. [48] Cerebras Priced The Biggest IPO Of 2026 Into A Market Huang ... — reactive:agentic-inference-economics
  49. [49] Pricing - Cerebras — reactive:agentic-inference-economics
  50. [50] Cerebras Launches the World's Fastest AI Inference : r/LocalLLaMA — reactive:agentic-inference-economics
  51. [51] Nvidia competitor Cerebras' wild IPO: Here's what you need to know — reactive:agentic-inference-economics
  52. [52] Nvidia rival Cerebras discloses US IPO filing as AI boom drives listings — reactive:agentic-inference-economics
  53. [53] Cerebras Systems IPO Signals Growing Demand for AI Chip ... — reactive:agentic-inference-economics
  54. [54] Can Cerebras Systems Challenge Nvidia? A Deep Dive into the AI ... — reactive:agentic-inference-economics
  55. [55] The Inference Shift – Stratechery by Ben Thompson — reactive:agentic-inference-economics
  56. [56] Please Listen to My Podcast – Stratechery by Ben Thompson — reactive:agentic-inference-economics
  57. [57] 2026.20: Shifting Alliances in a Changing World - Stratechery — reactive:agentic-inference-economics
  58. [58] Losing in the Attention Economy – Stratechery by Ben Thompson — reactive:agentic-inference-economics
  59. [59] AI and the Human Condition – Stratechery by Ben Thompson — reactive:agentic-inference-economics
  60. [60] Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks — reactive:open-model-capability-gap (2026-05-19)
  61. [61] DeepSeek V4 is out. the best open-source on coding. here's the breakdown — reactive:agentic-inference-economics (2026-04-24)
  62. [62] An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks — reactive:agentic-inference-economics
  63. [63] How We Cut LLM Costs by 59% With Prompt Caching — reactive:agentic-inference-economics
  64. [64] FlashAttention: IO-Aware Exact Attention for Long-Context Language Models - Interactive | Michael Brenndoerfer | Michael Brenndoerfer — reactive:agentic-inference-economics
  65. [65] Speculative decoding: how it works & when to use it — reactive:agentic-inference-economics
  66. [66] Speculative Decoding in vLLM: Complete Guide to Faster LLM ... — reactive:agentic-inference-economics
  67. [67] hemingkx/SpeculativeDecodingPapers: 📰 Must-read ... — reactive:agentic-inference-economics
  68. [68] Cost of Agentic AI:Expenses & ROI — reactive:agentic-compute-cpu-gpu
  69. [69] The Agentic AI Cost Problem: Calculating TCO for ... - CX Today — reactive:agentic-inference-economics
  70. [70] The Hidden Costs of Agentic AI: Why 40% of Projects Fail Before Production — reactive:agentic-inference-economics
  71. [71] How to avoid agentic sticker shock - HFS Research — reactive:agentic-inference-economics
  72. [72] Gartner Predicts 90% Drop in AI Inference Costs by 2030 - LinkedIn — reactive:agentic-inference-economics
  73. [73] How much will AI cost in 2030? Forecasts for companies - Brandsit — reactive:agentic-inference-economics
  74. [74] The math looks broken: token prices are falling (Gartner says inference costs drop 90% by 2030), but total bills keep ri... — reactive:agentic-inference-economics (2026-05-23)
  75. [75] AI infrastructure is hitting a new bottleneck: context memory. — reactive:agentic-inference-economics (2026-05-20)
  76. [76] AI is entering its “cloud economics” phase. — reactive:agentic-inference-economics (2026-05-22)
  77. [77] @vijayshekhar The 3G to 4G parallel works for bandwidth, but token maxing is a different constraint. Inference costs and... — reactive:agentic-inference-economics (2026-05-20)
  78. [78] Low cache hit rate for large fixed System Prompt in Azure OpenAI ... — reactive:agentic-inference-economics
  79. [79] AI open models have benefits. So why aren’t they more widely used? — reactive:agentic-inference-economics