FIGURE 2: Migration to High-Speeds in AI Clusters (AI Back-End Networks) (Source: Dell'Oro Group AI Networks Report Dec 2023) 6
Power and Cooling Modern AI hardware has server rack power demands ranging from traditional 4-6kW up to an astonishing 50-100kW or more, with next-generation configurations expected to require between 60kW to 120kW per rack. 7 Managing heat dissipation in these high-density racks is a critical challenge. Traditional air-cooling methods are insufficient to handle the heat generated by AI-focused hardware. To address these cooling challenges, the industry is turning to innovative solutions like liquid cooling. Liquid cooling technologies offer several advantages, including improved energy efficiency, reduced water consumption, and the ability to handle higher power densities. Some liquid cooling solutions can remove nearly 100% of the heat generated across the entire IT stack while reducing energy use by up to 40% and water consumption by up to 100%. 8 DATA INFRASTRUCTURE EVOLUTION The widescale adoption of AI technologies is driving a fundamental transformation in data infrastructure. This transformation includes how data is collected,
stored, processed, and analyzed—creating a new paradigm in data center design and operation.
Network Architecture Changes The exponential growth in AI model complexity is pushing data infrastructure to its limits. AI models are growing by a factor of 1,000 every three years, requiring extreme scale and bandwidth for efficient operation. 9 This growth has fundamentally changed data center traffic patterns, driving a shift from traditional north-south traffic patterns (data moving between servers and the internet) to east-west traffic (extensive communication between servers and nodes) within data centers. This architectural shift has significant energy implications. The increased east-west traffic requires more internal networking infrastructure and creates additional heat loads that must be managed. Network delays and idle computing resources further contribute to increased energy consumption, making efficient network architecture imperative for both performance and energy optimization.
January/February/March 2025
I
9
Made with FlippingBook - Online catalogs