As the number of components sharing data within artificial intelligence chips increases, managing that data is no longer solely a software responsibility. When a data accelerator writes to memory and a CPU then reads it, or when components exchange shared control information, software must know which copy of the data is the latest. The article explains that shifting more of this task to hardware, using memory coherency, can reduce software complexity, coordination errors, and the difficulty of developing systems.
The article was published as a sponsored blog post in Semiconductor Engineering and was written by Ashley Stevens, director of product management and marketing at Arteris, responsible for coherent interconnects and die-to-die communications. It therefore provides a useful technical explanation, but it also includes a presentation of Arteris product capabilities, which should be taken into account when evaluating the argument.
From Separate Copies to Shared Memory
In older systems, it was possible to keep the processor and accelerator relatively separate. The processor communicated with the accelerator over PCIe, copied data to the accelerator’s memory, then sent a message and waited for the results. This model was practical when the units operated more independently, but it becomes more complex as the volume of shared data and the number of processing stages increase.
The problem is more apparent in edge artificial intelligence systems, where a system may integrate a general-purpose processor and an artificial intelligence accelerator within a single SoC, or connect them in a design based on multiple dies. In this case, data passes through several stages before the result reaches the application, while coordination and control information needs to be available in the correct version to the relevant components.
Memory coherency is a chip-level technology that maintains a unified view of shared data, whether that data resides in main memory or in cache memory. Rather than having software track modifications and ensure that copies are updated, the coherent interconnect and memory system handle part of this task.
What Changes in Practice?
The main result is not the elimination of all copy operations or making all communications identical, but reducing the amount of software logic required to coordinate the operation of heterogeneous units. When hardware manages the state of shared data, software can focus more on application logic and work scheduling instead of tracking whether each component is reading an old or updated copy.
This shift can make software simpler and easier to develop, and may also reduce errors that are difficult to detect during operation. The article links this approach to software development speed and reliability, but it does not provide figures measuring the extent of the improvement in performance or development time. Therefore, the quantitative benefits cannot be considered confirmed based on the text alone.
Coherency Where It Is Needed
The article does not consider applying coherency to every data movement to be the best design. High-bandwidth streams do not always require memory coherency, whereas information that coordinates system operation among processing units does need this mechanism. For this reason, according to the argument presented, most artificial intelligence chips use a combination of coherent and non-coherent interconnects.
Coherent interconnects are used when several processing units share the same data or state, while non-coherent interconnects can carry data that does not require a shared view. This division gives designers an opportunity to optimize the remaining communications according to bandwidth, latency, and power-consumption requirements, rather than imposing the cost of coherency on every path.
The Importance of Multi-Die Design
The idea extends to multi-die systems, in which a larger chip is divided into several dies that exchange information. The article says that maintaining coherency among these dies can help scale heterogeneous systems without shifting full responsibility for coordinating shared data back to software.
In this context, Arteris mentions three components from its portfolio: FlexNoC for non-coherent communications, FlexGen for automating the design of high-performance NoC networks in advanced chips and multi-die systems, and Ncore for providing a coherent interconnect fabric. These references explain the technology’s position within the company’s products, but they do not represent a new product launch or an independent comparison of solutions in the market.
Why Does This News Matter?
The most prominent architectural value of the argument is that increasing the number of transistors dedicated to simplifying software is no longer necessarily considered wasteful. As artificial intelligence systems move toward designs that combine general-purpose processors, accelerators, and multi-die chips, the distribution of responsibilities between hardware and software becomes a decision that affects scalability and ease of development.
However, the practical question remains how much additional hardware is worth allocating to coherency, and where it should stop to avoid unnecessary costs in area, power, design, and verification. The article presents the principle of distinguishing between shared and non-shared data, but it does not specify quantitative criteria for making this decision, nor does it present test results or a documented use case comparing a coherent approach with one that relies more heavily on software management.