AI has a huge power problem. Solving it won't be easy

The computer chips powering your ChatGPT questions consume roughly six times more energy than the chips that dominated data centers just a few years ago. As AI pushes individual chips to consume dramatically more electricity, data centers are racing to squeeze more computing power from every watt — and the physics of keeping silicon cool may determine whether artificial intelligence becomes sustainable.

With grid constraints making new power connections increasingly difficult, data center operators and researchers are scrambling to find efficiency gains wherever they can. But small improvements — like liquid-cooled power systems, smarter maintenance schedules, and higher-voltage electricity distribution — are running up against fundamental physics problems and economic incentives that prioritize performance over sustainability. The question is whether these incremental wins can keep pace with AI's exponential appetite for electricity.

The stakes are getting higher by the server rack. Data centers already consume about 4% of the U.S. electrical grid, a figure expected to hit 9% within the next decade. In hot markets like Virginia and Texas, power companies are so overwhelmed with requests for new data center connections that they're charging millions of dollars just to study whether the grid can handle the load.

That has created a new urgency around an old metric: power usage effectiveness, or PUE, which measures how much electricity actually reaches the computers versus how much is wasted on cooling and other overhead. The math is simple, but the margins are tight. A data center running at 1.5 PUE can deliver only 67% of its incoming electricity to actual computing — the rest disappears into cooling systems and power conversion losses. Improving that even slightly can add up in energy and cost savings, according to Ryan Mallory, president and COO of data center operator Flexential.

"We're talking about tens and hundreds of a percentage point,” Mallory said. "But it's very, very impactful to the cost of operations. If you drop the PUE a tenth of a percentage point — say you're going from 1.4 to 1.3 — you could be gaining an efficiency of $50,000 per month for every megawatt of power consumption."

For a single large facility, that adds up fast. One of Mallory's clients operates a 27-megawatt AI facility, where a 0.1% PUE improvement saves $1.35 million per month, or more than $16 million annually. More importantly, that same efficiency gain means the facility can pack more computing power into the same grid connection — crucial when new power hookups can take years to approve and cost millions just to study.

Given the massive scale of construction underway, these gains become even more significant. Primary data center markets in North America now have nearly 7,000 megawatts of capacity, with more than 6,000 megawatts under construction, according to real estate firm CBRE. Across that footprint, even modest efficiency improvements can translate to significantly more AI computing capacity without requiring additional strain on an already overwhelmed electrical grid.

Cooling crisis

The path to those PUE gains often comes down to physics and planning. Hyperscale operators like Google and Meta can achieve PUE ratings as low as 1.1 or 1.2 because their server farms use identical equipment arranged in predictable patterns, creating consistent airflow. But most data centers house a mix of different customers with different hardware, creating what Mallory called "chaotic airflow patterns and thermal hot spots" that make efficient cooling much harder to achieve.

Yet no matter how perfectly arranged, all data centers are fighting the same battle against heat.

Operators are getting creative about managing that heat. Mallory's company schedules equipment maintenance during cooler morning hours to avoid the energy penalty of running tests during peak temperatures. In hot climates like Las Vegas and Phoenix, facilities use evaporative cooling systems that pre-cool outside air before it enters the main cooling system, similar to misters at outdoor restaurants. Some can even tap into "free air cooling" during winter months, opening vents to use cold outside air directly.

To handle the massive power loads more efficiently, data centers have had to upgrade their electrical systems. To handle the massive power loads more efficiently, data centers have had to upgrade their electrical systems. Traditional data centers used lower-voltage power distribution, but AI racks now require higher-voltage systems, with some operators planning jumps to 400 or even 800 volts.

The higher voltages allow for lower current at the same power, reducing the resistance losses that convert precious electricity into unwanted heat. It's a two-for-one efficiency gain that reduces both wasted energy and heat generation. But even these improvements can't solve the fundamental problem of racks that generate as much heat as space heaters crammed into a closet-sized footprint.

To really tackle the heat problem, data centers need more radical solutions. That's why TE Connectivity and other companies have developed liquid-cooled power distribution systems — essentially water-cooled electrical cables — that can handle more power in the same footprint as traditional systems while eliminating heat more effectively.

About 30% of new data centers are being built with liquid cooling systems, with that percentage expected to hit 50% within two to three years, according to Srinivasan. But liquid cooling creates its own sustainability challenge: Data centers can consume millions of gallons of water annually for cooling, straining local water supplies. Some facilities are experimenting with immersion cooling — literally dunking entire servers in mineral oil — which eliminates water use entirely, though the logistics make it impractical for most applications so far.

Efficiency's unintended consequences

Beyond infrastructure improvements, chip makers are pursuing their own efficiency gains. Companies such as AMD are betting on rack-scale architectures that can boost energy efficiency 20-fold by 2030, while newer chip designs support lower precision calculations that can dramatically reduce computational load. Nvidia's next-generation Blackwell GPUs — and the even newer Blackwell Ultra platform — promise their own efficiency improvements. Nvidia CEO Jensen Huang has stated that the company's GPUs are typically 20 times more energy-efficient for certain AI workloads than traditional CPUs.

But there's a fundamental paradox at work with newer chips. Energy bills have roughly doubled when upgrading to newer Nvidia chips, according to Dan Alistarh, a professor at the Institute of Science and Technology Austria who researches algorithm efficiency. "It's a weird trade-off because you're running things faster, but you're also using more energy," Alistarh said.

The algorithms powering AI show even less progress toward efficiency. Researchers like Alistarh are still working on techniques that could reduce the energy consumption of generative AI, such as using simpler math that requires less computing power. Other groups are exploring entirely different architectures that could replace transformers altogether. 

But these innovations have struggled to gain traction because AI companies are judged largely on how their models perform on standardized tests that measure capabilities like reasoning, math, and language comprehension, scores that directly influence funding and market perception.

Companies would rather build energy-hungry models that score higher on these tests than efficient ones that might lag behind competitors, even slightly. The result is an industry that optimizes for leaderboard rankings over sustainability, making efficiency improvements, regardless of the cost savings, a secondary concern at best.

"Anything that drops you lower on the rat race of benchmarks is a clear loss," Alistarh said. "No one can afford to do that." 

Some reporting for this article was conducted as part of a journalism residency funded by the Institute of Science and Technology Austria.