Data movement is the new performance battleground in semiconductor design — TechRadar

RESOLVING THE LEDGER

For much of the semiconductor industry’s history, performance debates have centered on compute throughput and memory capacity. Faster processors, wider vectors, and larger caches have been the primary levers for system architects, often treating the movement of data between them as a secondary concern.

Today, a different constraint is asserting itself as AI data center workloads proliferate, architectures diversify, and systems extend beyond traditional computing into the physical world. Data movement, rather than processing or storage , is increasingly defining the limits of performance, power efficiency, predictability, determinism, and scalability.

This shift around data movement is already visible. Across applications like advanced SoCs, AI accelerators, chiplet-based systems, and now in physical AI applications such as robotics, industrial automation , and intelligent vehicles, it has become clear that transporting the vast quantities of data required by such workloads is more demanding than processing them.

Physical AI does not introduce a new problem so much as it exposes an existing one: when systems must perceive, decide, and act in closed loops under real-time and safety constraints, inefficient or unpredictable data movement quickly becomes the dominant bottleneck.

The implications for both AI in the data center and physical AI extend far beyond any single market. As data center demand increases exponentially and the industry pushes toward increasingly complex, heterogeneous systems in anything from vehicles to industrial systems, data movement has become the primary performance battleground.

Performance limits are increasingly defined by data movement

Modern semiconductor systems integrate an unprecedented number of processing elements, including general-purpose CPUs , AI accelerators, GPUs , DSPs, sensor processors, and domain-specific engines. Still, simply adding compute resources rarely delivers proportional gains. In many advanced designs, performance plateaus long before compute capacity is exhausted.

Compute units are left sitting idle not because they lack capability, but because data cannot reach them efficiently or predictably. Late-stage analysis of high-performance SoCs often reveals that throughput shortfalls stem from contention, imbalanced bandwidth allocation, or inefficient routing within the communication fabric, rather than from compute limitations.

This is especially visible in physical AI systems, where delays introduced by data movement propagate directly into system behavior in closed-loop architectures. Latency or contention in the transport fabric can destabilize control algorithms, reduce accuracy, and even force over-provisioning of compute to compensate for the absence of bounded latency and guaranteed behavior.

While physical AI makes data movement constraints more immediately visible, the same forces are amplified dramatically in the data center. To push enormous volumes of data through extremely wide interfaces, processors increasingly contain hundreds of compute and accelerator instances, press against reticle-scale integration limits, use chiplet-based designs, and interface with multiple stacks of high-bandwidth memory (HBM).

As designs scale, the combined costs of moving data grow rapidly. In this environment, abundant compute and memory bandwidth offer little benefit without a data movement architecture capable of managing data at scale. What is a tight constraint in physical AI becomes an overwhelming one in the data center.

In both the AI data center and physical AI, performance is inseparable from the behavior of the data paths that connect perception, decision, and actuation. As a result, interconnect design can no longer be a back-end integration task. It has become a central architectural decision, on equal footing with compute and memory selection.

On-chip data movement is fundamentally different

Data movement inside a chip operates under constraints that differ radically from those of off-chip networking or board-level interconnect. On-chip transfers are extremely frequent, tightly synchronized with execution, and subject to stringent latency and power budgets.

Traffic is also inherently heterogeneous. A single system may need to carry high-bandwidth AI data streams, cache and coherency traffic, latency-critical control messages, and safety-related signaling, often simultaneously. These flows have very different requirements and cannot be treated uniformly.

Traditional best-effort arbitration models break down quickly under these conditions. Bursty accelerator traffic can interfere with control-plane and real-time data paths, leading to unpredictable system behavior. In mission and safety-critical or physical AI applications , such interference is unacceptable.

As a result, quality of service (QoS), traffic isolation, bounded latency, and determinism are now architectural requirements, not just optional optimizations. The interconnect fabric increasingly functions as an active system component, enforcing policy and guaranteeing behavior, rather than as a passive conduit for bits.

AI, heterogeneity, and physical intelligence magnify the challenge

AI workloads fundamentally change the character of data movement. They generate massive data volumes, irregular access patterns, and asymmetric traffic flows. At the same time, heterogeneous architectures distribute computation across many specialized engines, eliminating any single “center” of the system.

In both data center and edge contexts, this decentralization increases coordination costs. Data replication, synchronization overhead, and inefficient sharing can consume significant power and latency, eroding the benefits of specialized compute. Rather than compute placement, the challenge designers face now lies in how to efficiently orchestrate data movement between diverse compute elements.

Physical AI places additional pressure on these architectures. Unlike bat...

Data movement is the new performance battleground in semiconductor design — TechRadar | AIPROPX