The Power Paradox in AI Datacenters
In the current landscape of AI datacenters, a surprising challenge has emerged: securing sufficient power supply is proving harder than acquiring cutting-edge hardware. The cost of 1,665 Nvidia ‘Blackwell’ B200 GPU compute engines and 10 megawatts of power for a four-year contract in the Northeast United States is roughly equivalent, around $66 million. However, the GPUs will consume approximately 13.4 megawatts in a GB200 NVL72 rackscale configuration, translating to 88.5 megawatts over four years.
Nvidia has begun requiring datacenters to prove they have allocated power before selling GPUs, especially for large-scale AI inference operations. Power companies are now scrutinizing not just the amount of power requested but also how it will be distributed and cooled within datacenters. This shift has made efficient cooling solutions a critical factor in securing power allocations.
The Rise of Direct Liquid Cooling
Direct liquid cooling (DLC) has become essential for AI systems running chain-of-thought models, which demand low-latency, coherent memory links between GPUs. Supermicro, a key player in this space, has developed successive generations of DLC technology. Their DLC-1 system, used in the xAI ‘Colossus’ system, could remove over 70% of the heat using water at 30°C. The latest DLC-2 system boasts improved efficiency, with coolant distribution units (CDUs) capable of delivering 250 kilowatts of cooling flow at 45°C, potentially eliminating the need for chillers.
The new 4U nodes, such as the SYS-422GS-NBRT-LCC, feature cold plates for CPUs, GPUs, main memory DIMMs, PCI-Express switches, and power supplies. Future HGX B300 systems will extend liquid cooling to network interface cards, aiming to remove 98% of generated heat through liquid cooling. This results in a significant reduction in power consumption for cooling – 40% less than air-cooled HGX H100 systems from just two years ago.
Efficiency and Implications
The DLC-2 setup targets a power usage effectiveness (PUE) of 1.02, a marked improvement over traditional datacenter racks (PUE of 1.6-2.0) and even the previous DLC-1 generation (PUE of 1.2). Noise levels also drop dramatically, from 75 dB to around 50 dB, comparable to normal conversation levels.
As datacenters continue to push the boundaries of AI processing, innovations in cooling technology will play a crucial role in meeting both performance and energy efficiency demands. The advancements in direct liquid cooling represent a significant step towards more sustainable and powerful AI infrastructure.
