A single chatbot query hits a GPU, generates a response, and the computation ends. An AI agent doing the same job might run for hours, spawning sub-agents, calling APIs, querying databases, and looping back through the model dozens of times before producing a result.
A study published in April by researchers at Stanford's Digital Economy Lab found that agentic coding tasks consume 1,000 times more tokens than standard code reasoning and chat, a finding that puts hard numbers on a cost problem the AI industry is only beginning to confront.
Why every agent action costs more than the last
The mechanics are structural, not incidental. Unlike static reasoning models, AI agents continuously plan, invoke external tools, observe outcomes, and refine their reasoning, often performing dozens of inference calls to satisfy a single user request. Each call re-reads the entire conversation history accumulated so far, so an agent reads the task, gets a response, then has to re-read everything before the next action, and re-reads all of that plus the new response before the next action. The result is that context grows with every step.
In agentic workflows, key-value caches never reset but accumulate across multi-step reasoning chains, meaning what may be a 10 GB cache for a single inference becomes a moving bandwidth floor that compounds with every agent interaction, according to analysis by networking firm Ciena. This is not a marginal increase. Researchers at Korea Advanced Institute of Science and Technology found that AI agents "introduce substantial energy overheads that are orders of magnitude higher than conventional single-turn LLM inference," according to a paper published on arXiv that presented what the authors described as the first system-level characterization of agent infrastructure costs. Without system-level innovations, the authors warned, per-request computational costs could increase by orders of magnitude, making large-scale deployment "economically and environmentally prohibitive."
The hardware ratio is inverting
For the past several years, AI data centers were designed around a simple architecture: one CPU acting as a host node for four to eight GPUs. The GPU did the math. The CPU managed traffic. Agentic AI breaks that model.
Agentic AI workloads are reshaping data center compute requirements by shifting performance bottlenecks from GPU-centric inference to CPU-heavy orchestration, introducing substantial CPU demand and making CPU capacity a critical factor in system throughput, according to analysis published by SemiWiki. AMD $AMD now reports that, unlike the previous one-to-four or one-to-eight CPU-to-GPU ratio seen with chatbot AI, agentic AI is moving toward a one-to-one ratio, and in some cases toward higher ratios on the CPU side. The company doubled its forecast for the server CPU market in less than six months.
Where AMD previously projected server CPU market growth at 18% a year, the structural increase in compute driven by agents led the company to revise that expectation to greater than 35% annual growth, with total addressable market exceeding $120 billion by 2030, according to a blog post by Dan McNamara, senior vice president of compute and enterprise AI at AMD. Morgan Stanley $MS reached a similar conclusion in an April report, forecasting that agentic AI could add $32.5 billion to $60 billion in new demand for data center CPUs by 2030, arguing that "as AI transitions from generation to autonomous action, the computing bottleneck is shifting towards CPU and memory." At the AMD Advancing AI event in June 2025, CEO Lisa Su described agentic AI as "a new class of user," adding that "what we're actually seeing is we're adding the equivalent of billions of new virtual users to the global compute infrastructure."
The scale of what's coming
Goldman Sachs $GS put concrete projections on the demand trajectory in a report published this month. The bank expects token consumption to grow 24 times by 2030 compared to 2026 levels as AI queries jump from five billion to 23 billion per day, driven by non-human agents. Goldman Sachs projects that agentic AI will account for about 30% of those queries by 2030.
The infrastructure spending to support this is already in motion. The five largest U.S. cloud and AI infrastructure providers have collectively committed to spending between $660 billion and $690 billion on capital expenditure in 2026, close to double 2025 levels, according to Futurum Group. But spending alone does not guarantee that the infrastructure will match the workload. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value, or inadequate risk controls.
The cost surprises are real: the Stanford study found that token usage in agentic tasks is "highly variable and inherently stochastic," with runs on the same task differing by up to 30 times in total tokens consumed.
What this means for the energy equation
The compute trajectory has direct implications for the energy question hanging over the AI industry. As quantum computing moves into data center infrastructure alongside classical processors, the emerging hybrid model of CPUs, GPUs, and quantum processing units working together holds promise for specific scientific and financial workloads. But quantum machines draw about 30 kilowatts each, while a single rack of Nvidia $NVDA's AI training chips draws 120 to 140 kilowatts, with next-generation systems projected to exceed 200 kilowatts. Quantum is designed for a different category of problem. It is not going to absorb the load that agentic AI is creating.
The honest math is that agentic AI makes the data center power problem worse, not better. Every agent that runs for hours instead of seconds, every sub-agent it spawns, every loop it executes is compute that classical hardware must serve. Goldman Sachs Research projects that data center power consumption will jump 175% by 2030 from 2023 levels, and the rise of agents is a primary driver.
For enterprise buyers and data center planners, the practical takeaway is that infrastructure sized for chatbot-style AI will not support agentic deployments. The workloads are different in kind, not degree. Planning for agents means planning for more CPUs, more memory, more power, and architectures that most organizations have not yet built.