THE IMPERATIVE OF LIQUID COOLING As noted above, the extreme power densities of modern AI racks have rendered traditional air cooling insufficient. The thermal loads from today's AI accelerators demand a fundamental transition to liquid cooling, which has thermal transfer efficiencies up to 3,000 times greater than air. The industry has coalesced around several key approaches, each with its own implications for physical infrastructure: • Direct-to-chip (DTC) cooling uses cold plates to precisely remove heat at the processor level, requiring intricate tubing within dense server chassis. • Immersion cooling, both single-phase and two-phase, submerges entire servers in dielectric fluid, dramatically redefining rack architecture and serviceability. • Rear-door heat exchangers (RDHx) act as a hybrid solution, capturing heat at the back of the rack before it enters the data hall.
(DCI) is growing rapidly. This is where coherent optics become indispensable. Unlike traditional intensity-modulated optics, coherent systems leverage sophisticated modulation techniques, combined with advanced digital signal processing (DSP), to vastly increase spectral efficiency and reach. This enables terabits of data to be pushed over thousands of kilometers, making geographically dispersed AI operations possible. Today, 400ZR and 400ZR+ transceivers are widely deployed, forming the backbone of metro and regional DCI. There are also trials and early deployments of 800ZR for next-gen inter-cluster traffic. What is particularly exciting is the innovation at the silicon level. The integration of coherent optics directly onto processing chips via co-packaged optics (CPO) and optical interposers reduces power and latency at the edge of the switch. Furthermore, the IEEE 802.3dj task force is driving the 1.6 Tb coherent Ethernet standard for AI fabrics, with ratification anticipated in 2026—further enabling this deep integration. THE RACK: A CRUCIBLE OF POWER AND HEAT After looking at the structured cabling issues, the next AI infrastructure challenge is the rack. This is where the convergence of extreme compute, power, and networking density creates an unprecedented thermal battleground.
A decade ago, a typical enterprise rack consumed 4 to 6 kW. In 2025, it is standard for AI racks to draw 100 to 150 kW, with roadmaps already pushing beyond 200 kW (Figure 6). This staggering increase is driven directly by the AI accelerators at the heart of the system. Next-generation GPUs feature thermal design powers (TDP) ranging from 1,000 to 2,000 watts per chip. When multiple of these processors are aggregated into a single server, and racks are filled with those servers, they produce a concentrated heat load that renders traditional air cooling obsolete. The physics are undeniable: air cannot transfer heat efficiently enough for these loads, making a wholesale shift to liquid cooling not just an option, but a necessity. The industry is seeing rapid adoption of direct-to-chip cold plates, rear-door heat exchangers, and full immersion cooling systems as the new standard for AI deployments. This thermal revolution has profound implications for physical layer design. High-density optical fiber and power cabling must now be meticulously co-designed and integrated with liquid cooling manifolds, coolant distribution units, and intricate piping. The performance and reliability of an AI cluster are no longer just a function of its processing speed and network bandwidth; they are now fundamentally dependent on a sophisticated and integrated thermal and physical infrastructure strategy.
facilities, built around continuous, redundant, 24/7 chilled water loops. This means that optical fiber and power infrastructure can no longer be designed in isolation. The layout of high-density optical fiber trays, the placement of power busways, and the routing of coolant manifolds are now deeply interdependent. POLICY & STANDARDS: THE NEW RULES As AI infrastructure scales globally, policy and standards are forcing design changes. The European Union has mandated climate-neutral data centers by 2030. In the United States, national initiatives acknowledge that AI infrastructure is strategic and have begun to address permitting and supply chain barriers. On the standards side: • The ANSI/TIA-942-C standard was updated in May 2024 to reflect AI-scale infrastructure needs. It includes support for higher power densities, liquid cooling, expanded optical fiber topologies, and sustainability guidance.
• The IEEE 802.3dj standard is advancing 1.6 Tb ethernet, critical for next-gen AI fabrics.
AI data centers are now being designed as "liquid-first"
FIGURE 6 : A timeline illustrating the increasing power density of server racks from 2015 to 2025, highlighting the challenges this poses for thermal management. Source: AFL
FIGURE 5 : HCF is a type of optical fiber that transmits light through an air-filled core instead of a solid glass core. Source: AFL
I
I
36
ICT TODAY
January/February/March 2026
37
Made with FlippingBook - Online catalogs