Why AI Inference Changes Infrastructure Geography

← Articles index · Home · Infrastructure · Nori · About

Topics: AI Inference · Geography

AI infrastructure debates often blur training and inference. They are related, but they stress geography differently. Training large models can tolerate batch transfer, scheduled jobs, and long-haul networking. Inference that powers interactive products, agents, and operational systems is judged by response time, consistency, and integration with nearby data sources.

As more production value moves to inference, infrastructure geography becomes a product decision, not only a real-estate decision.

Training follows power. Inference follows demand, data, and latency budgets.

Training vs inference

• Training: large, concentrated jobs; high bandwidth bursts; often more tolerant of distance.
• Inference: continuous or interactive calls; multi-step agent loops; sensitive to round-trip delay.
• Training clusters optimize accelerators and interconnect density.
• Inference regions optimize proximity, availability, and integration with applications.

Glossary: AI Inference and Metro AI Infrastructure.

Why latency compounds

A single chat reply may hide many model calls, retrieval steps, tool invocations, and policy checks. Each step can pay the network tax again. Moving compute 30 to 60 milliseconds farther away does not cost 30 to 60 milliseconds once. It can cost that amount on every hop in the chain.

Deep dive: The Latency Tax.

Why coastal and metro regions matter

Users, enterprises, cable landings, cloud on-ramps, and industrial systems concentrate in metros, many of them coastal. Placing persistent inference capacity in those regions reduces the mismatch between where GPUs are installed and where outputs are consumed.

Related: The Durable Geography of AI Demand and Persistent Regional AI.

What this means for ocean compute

Underwater and nearshore systems are interesting for inference when they can sit close to coastal demand and terrestrial fiber while easing land and cooling constraints. They are less compelling when they chase remote power and leave networking behind.

See What Is a Subsea Data Center? and An Electron Is an Electron.

Planning questions

• Which workloads are training-dominated versus inference-dominated?
• How many sequential model or tool calls occur per user action?
• Where do the users, data stores, and enterprise networks live?
• What latency budget remains after application logic?
• Is regional capacity available near that demand, or only in distant campuses?

Geography will not decide every AI workload. For interactive inference at scale, it decides more than the industry assumed a decade ago.