Headlines about AI infrastructure spending have started to feel almost hyperbolic. Companies are committing hundreds of billions of dollars, with Nvidia suggesting as much as $3-4 trillion in annual AI spending by the end of the decade. Against this backdrop, investors are asking: how much compute do we really need, and is this boom sustainable?

From Moore’s Law to a new compute S-Curve

The evolution of compute has long been defined by Moore’s Law, where transistor counts per computer chip have doubled roughly every two years while costs remain flat. That exponential progress powered faster, smaller and cheaper electronics for decades. But Moore’s Law has since run into physical and economic limits, making annual improvements in devices like smartphones and PCs less noticeable. Enter Nvidia, which has pioneered a new compute paradigm: parallel, accelerated computing¹. This novel approach to computing is reinvigorating an industry built around Moore's Law and helping disseminate AI across industries, ushering in a new “S-curve” of compute demand.  

The first wave of AI spending focused on training large language models. Training is significantly, and increasingly, compute-intensive, but early LLM demands were manageable². Today, compute needs are accelerating rapidly, particularly as more models move into production.

Inference is also evolving into a major driver of compute demand. Early inference was “single-shot” (quick responses based on pre-trained data) but is now shifting toward “reasoning inference,” which requires more compute but produces better outcomes and broadens AI use cases. Nvidia estimates that reasoning models answering challenging queries could require over 100 times more compute compared to single-shot inference³.

Business models are evolving around AI

Cloud providers now compete in large part on access to AI compute, making chips and infrastructure central to securing new business. Software firms are embedding AI into productivity, coding and customer support tools, aiming to monetize AI usage through subscriptions and enterprise contracts. Meanwhile, applications are broadening from digital domains into physical ones such as robotics and autonomous vehicles, further expanding demand.

Hyperscalers are also exploring ways to optimize their AI compute investments, including refining software algorithms and experimenting with specialized AI chips (ASICs) to make specific AI tasks more efficient. While debate continues regarding the long-term role of ASICs, the broader trend is clear: AI workloads keep proliferating, requiring ever more compute.

The road ahead

Skepticism about the sheer pace of AI investment and ROI is healthy and warranted. Foundational model builders like OpenAI and Anthropic represent roughly ~20% of AI capex⁴ by some estimates with still-nascent business models, while expanding players like Oracle are now tapping bond markets to finance AI infrastructure.

Still, beneath the near trillion-dollar headlines is a real computing platform shift decades in the making that is reshaping industries and business models. While this AI infrastructure buildout is unlikely to reverse, it foreseeably will not be a straight line, either. Not all participants will be winners, and the most revolutionary business models may not even exist yet. Active management, with an eye toward which companies can create enduring value, will be essential for investors as this story unfolds. 

¹ Parallel computing architectures, such as GPU-based and multi-core systems, handle complex, data-intensive workloads (like AI, big data analytics, and video processing) far more efficiently than traditional computing approaches underpinned by Moore’s Law. By processing vast amounts of data simultaneously rather than sequentially, parallel computing systems enable faster computation and greater scalability.

² The cost to train the most compute-intensive models has grown at a rate of 2.4x per year since 2016, with AI accelerator chips accounting for some of the most significant expenses. Source: Cottier et al., "The Rising Costs of Training Frontier AI Models," arXiv:2405.21015 (2024).

³ Source: Nvidia Glossary, AI Reasoning.

⁴ Source: NewStreet Research. 

c3d42a2a-9ec6-11f0-b59f-9f329f7c52a9