FROM THE INDUSTRY
processing is complete. Each new “hop” along that path – and there can be many – introduces lag that AI simply can’t tolerate. It also introduces issues around security, data sovereignty and multiplies potential points of failure. This is often referred to as the “tromboning” effect, and it doesn’t play well with AI. Training cycles slow, real-time inference becomes unusably unresponsive and operational costs rise as the volume of traffic increases. We’ve now reached the point where a fundamental rethink is required about where interconnection happens, so that data, compute and users are brought closer together rather than pushed ever further apart. Building “IP gravity” through interconnection An interconnection ecosystem made up of IXs doesn’t just benefit individual businesses and use-cases; it also starts to strengthen the digital economy in that region. After all, networks, cloud platforms, content providers and businesses are more likely to base their services nearby when they can exchange traffic efficiently and predictably. And over time, this creates a reinforcing effect where localised infrastructure attracts more participants, which in turn improves performance and resilience for everyone involved. We call this “IP gravity,” and it’s become increasingly relevant as AI workloads demand proximity between data, compute and users. We’re now seeing examples of this emerge in regions that have traditionally sat outside of the main technology corridors. For instance, Wichita State University (WSU) has now deployed a carrier-neutral IX on its campus to act as an anchor for regional digital ecosystem. By enabling local traffic exchange and attracting adjacent services, environments like these reduce dependence on distant hubs and create the conditions needed for advanced workloads, including AI. This is paving the way for similar initiatives nationwide, so instead of concentrating “intelligence” or compute resources in just a handful of locations, it can be spread more evenly to improve performance, resilience and access across entire regions. That’s the kind of decentralised thinking that will be needed if we want to realise the promise of AI.
When industries discuss AI, it’s often framed as a contest of scale – bigger models, denser GPUs, faster chips and larger data centres. That focus is understandable, as that’s where AI has seen the most impactful gains in recent years. The fact that we can all consult with a large language model (LLM) via a few taps of our phone, or that medical professionals can undertake AI-assisted surgical procedures without laying a hand on the patient, are largely due to the “bigger, better, faster” boom that AI has benefited from. But there’s another facilitator – or constraint – that isn’t often talked about. Every AI workload, whether it’s fine-tuning AI models or the more common use of real-time inference, depends on the speed of data. In other words, how fast data can move between various systems. As AI applications become more interactive, more distributed, and more deeply embedded in everyday processes, latency has become the determining factor. This is now exposing the limits of network architectures that were designed for a different era of applications. In 2012, the average latency for the top 20 applications was around 200 milliseconds. Today there’s virtually no application in the top 100 that would function effectively with that kind of latency. AI-driven applications, particularly those that are based on real-time input and output, become so unusable when latency is introduced that it’s completely changing the conversation around AI growth. Industry players are no longer thinking about bigger, better, faster models – but are instead considering how current and future generations of AI can be sustained globally on a network level. In other words, the question is no longer “How much compute can we deploy?” but “How can we connect it intelligently and how close can we get it to the end user?” How AI is exposing the limits of today’s networks AI workloads place demands on networks that are fundamentally different from those of earlier digital services. Model training involves moving vast data sets between clusters of GPUs that may be spread across multiple data centres and regions. Real-time inference, which is how most end users and applications engage and interact with AI, introduces a different challenge altogether.
AI systems depend on the reliable back- and-forth exchange of data between users, devices and applications. Because most AI use-cases are designed to deliver real-time experiences to users, even delays of a few milliseconds can render those use-cases virtually broken and unfit for purpose, effectively throttling AI innovation. While traditional networks were designed to handle big bursts of web traffic, they’re now being asked to support latency-sensitive, continuous traffic on an unprecedented scale – and most of them are struggling to keep up. The public Internet simply wasn’t designed for AI. It becomes easily congested and businesses are at the mercy of unpredictable routing paths with no control or visibility into how their data moves from A to B and back again. Internet Exchanges (IXs) remedy this by offering more control, reducing the number of “hops” data has to make to get to where it needs to be. This form of “interconnection” via IXs effectively brings networks, cloud platforms, and businesses closer together – even when geography is a constraining factor. For AI, the network needs to be more than just a delivery mechanism, because it directly affects whether real-time inference can deliver results fast enough to be useful and whether distributed training will be able to scale efficiently in the future. Why geography has become an AI bottleneck For decades now, centralisation has been the goal when it comes to digital infrastructure. Concentrating compute and storage in a small number of large hubs simplified operations and took advantage of economies of scale. For traditional applications, that model worked well enough, but AI is far more demanding. Workloads involving AI are becoming more distributed and latency-sensitive, to the point where the physical distance between where data is generated and where it needs to go starts to matter – a lot. The further data has to travel, the greater the latency, and the harder it becomes to deliver on the promise of any given AI application. Consider a region that sits far away from a traditional urban hub like New York. For businesses based in these remote regions, their data will have to travel hundreds of miles to reach the compute resources it needs to carry out calculations, then travel back along the same lengthy path when
www.de-cix.net
Volume 48 No.1 MARCH 2026
75
Made with FlippingBook - Online magazine maker