The architecture of data-center networks dedicated to AI workloads is changing as clusters exceed the boundaries of a single rack, expanding the role of optical links and weakening the traditional rule that associates copper with scale-up operations and optical fiber with scale-out operations. According to an article published by Semiconductor Engineering on August 13, 2026, rising data-transfer rates within the rack are pushing copper toward limits in loss, distance, and power consumption, while light can provide a more suitable path for high data rates, provided that power and reliability requirements are controlled.
The importance of this shift stems from the effect of latency on GPU performance. Eric Aguilar, CEO and co-founder of Omnitron, said that numerous reports indicate GPU efficiency may reach approximately 25% because the units are waiting for data instead of executing work. Therefore, the choice of interconnect medium is not solely about connection distance, but also about the network’s ability to keep processors busy and reduce latency and wasted power.
From Scale-Up Within the Rack to a Broader Scope
Scale-up was traditionally defined as communication within a single rack using copper, with memory access enabled according to memory semantics. But this definition has become less precise as computing clusters extend to servers located in adjacent racks. In this case, the distance increases, and copper may no longer remain the best option.
Vishal Chandrasekar, head of product management at Ayar Labs, says copper is more likely to be used within the rack, while optical links are the expected choice when adjacent racks are exceeded. At a transmission rate of 200 gigabits, copper may be suitable for a distance of up to five meters, or seven meters when pushed to its limit, whereas optical links become necessary at ten meters or more, according to his estimate.
This does not mean that copper will disappear from all scale-up applications. Rather, the most stable element in the definition of scale-up may become the use of memory semantics, not the system’s location within a single rack or the type of physical medium. Priyank Shukla, director of product management for interface IP at Synopsys, explained that the software definition is associated with a single operating-system domain and a single memory space, allowing the processor to write to a memory location regardless of where that memory is located.
The Emergence of the Term Scale-In
Alongside scale-up, scale-out, and scale-across, a new term has emerged: scale-in. According to Chandrasekar, the term began to be used during the three months preceding the article and refers to the amount of bandwidth leaving a GPU that remains within a single server chassis.
- Scale-in: Communication between processors within a single server or on a single board.
- Scale-up: Communication within a shared memory domain, traditionally associated with the inside of the rack and copper.
- Scale-out: Communication between racks, using Ethernet RDMA semantics, with increasing reliance on optical links.
- Scale-across: Communication between different data centers or campuses over fiber.
The UALink standard maintains a basic expectation in scale-up environments: every accelerator should be one hop away from the other accelerators. According to Shukla, UALink networks are designed to preserve a connection between accelerators that passes through a single switch, even when the scale-up scope extends beyond one rack. This switch can be placed at different locations within the rack architecture, such as the top of the rack or its middle, but the principle is to keep data traffic between the two endpoints within a single hop.
Topology and Optical Switching
As the scale-up scope expands, the type of medium alone is no longer sufficient to determine performance; the topology connecting accelerators, switches, and servers becomes a decisive factor. One of the most common designs is the Clos, or leaf-and-spine, architecture, which provides a short path between servers. However, according to the article, Google uses a torus network in its AI network, in which TPU units are connected to neighboring units in three directions, giving the design a character resembling a systolic architecture.
A torus network typically requires a variable number of hops to reach between two points, and the path shown in the article may involve up to four hops, compared with one hop in a leaf-and-spine architecture. With packet switching, packet information must be examined at every switch or router. Because optical technology does not handle packets directly, the signal is converted at each node from optical to electrical to determine the next hop, and then converted back to optical. These repeated conversions increase latency and power consumption.
For this reason, circuit-switching technologies are returning to the discussion. Instead of processing packets at every node, circuit switching opens a complete path from source to destination and dedicates it to a single flow. Google has implemented an optical circuit-switched network in a torus configuration, while other entities are studying the use of OCS networks with different topologies. This indicates that scaling AI clusters depends not only on replacing copper with light, but also on reconsidering how paths are organized, the number of hops, and the mechanism used to transmit data.