Why AI Is Changing the Design of Data Centers

For years, a data center was a fairly predictable building: rows of air-cooled servers, a steady power feed, and a network built around web traffic. AI has broken that template. Training and running large models packs enormous amounts of power into small spaces, and the buildings built for the cloud era are struggling to keep up. For ongoing coverage of the hardware driving this shift, see our AI infrastructure and semiconductor news.

Power density is exploding

The most basic change is how much power each rack draws. According to Data Center Dynamics, average rack density rose from about 16kW in 2025 to 27kW in 2026, yet only one in five operators say they are ready for the 50 to 70kW racks now common in AI deployments. The newest systems go much further: Nvidia’s Vera Rubin platform can push a single rack to as much as 246kW.

Even today’s mainstream AI hardware is far beyond older designs. A detailed liquid cooling guide notes that Nvidia specifies its GB200 NVL72 as a rack-scale, liquid-cooled design drawing roughly 120kW per rack. The reason is simple: AI chips are packed tightly together so they can share data quickly, which concentrates power and heat in fewer racks instead of spreading it across hundreds of lower-density servers.

Air cooling hits its limit

Heat is where that power ends up. Traditional air cooling works comfortably up to roughly 15kW per rack, according to CoreSite, while liquid systems can handle 200kW and beyond. One 2026 analysis argues that densities above 20kW call for at least direct-to-chip liquid cooling, and that above 50kW operators need immersion or purpose-built liquid infrastructure.

The economics are shifting too. A Lombard Odier review cites Schneider Electric modeling in which cooling takes about 21 percent of capital spending for a 40kW liquid-cooled rack, versus about 10 percent for a 10kW air-cooled one. Liquid cooling costs more up front but becomes cost-effective as racks pass roughly 40kW.

In practice, many operators are choosing hybrids. The same CoreSite piece describes a hybrid approach where air handles lower-density gear and liquid handles the hottest equipment. Even the chips themselves are being designed with this in mind: Google says its newest TPUs support fourth-generation liquid cooling, and Microsoft’s Maia 200 also uses integrated liquid cooling. Read more in our breakdown of why Google and Microsoft are designing their own AI chips.

Power is now the biggest constraint

Cooling is only half the story. Getting enough electricity to the site has become the defining challenge. Data Center Dynamics reports that global data center electricity demand is projected to reach about 132GW in 2026 and climb toward 290GW by 2030, driven largely by AI-optimized servers.

This is changing where data centers get built. CBRE reports that power availability now outweighs network connectivity in site selection, with operators prioritizing locations that can deliver 300MW or more within tight timelines. A site’s value used to depend on fiber access; it now depends on what the local grid can supply. Developers are also exploring on-site generation, with one power guide pointing to a shift away from pure grid dependence toward options like small modular reactors.

The network is part of the computer

AI training treats thousands of chips as one machine, so the network inside the data center matters as much as the servers. Google’s TPU 8t, for example, links 9,600 chips in a single pod, and a dedicated Virgo network is designed to keep scaling close to linear. Microsoft’s Maia 200 takes a different route, using standard Ethernet in a two-tier scale-up design for clusters of up to 6,144 accelerators.

The result is a different physical layout. Instead of independent servers, AI facilities are built as tightly coupled clusters, with short cable runs, high-bandwidth links, and the cooling and power systems designed around the cluster rather than the individual server.

Building faster, and building differently

AI is also changing how fast and how modularly facilities are built. Operators are working in phased campuses and prefabricated power and cooling modules, so capacity can come online in stages. Data Center Dynamics describes a phased campus planned to reach up to 750MW, with integrated power and liquid cooling infrastructure from Schneider Electric and Motivair. The goal is to get capacity running quickly, because every month of delay is a month of idle, expensive chips.

What this means going forward

The data center is turning from a general-purpose warehouse of servers into a purpose-built AI factory, designed around the chip, the rack, the cooling loop, and the grid connection as a single system. Forecasts suggest average rack densities heading toward 40kW within a few years, with some projections going even higher, so today’s high-density designs may soon look ordinary.

For operators, the lesson is that the old rules no longer hold. Cooling, power, and networking are no longer background utilities; they decide how much AI a facility can actually run. For more on the companies shaping this build-out, follow our data center and AI hardware coverage.