System optimization is no longer possible by considering hardware and software separately, according to an analysis published by Brian Bailey, the technology editor specializing in EDA at Semiconductor Engineering. Software workloads are changing faster than chip development cycles, while power, performance and heat have become interconnected factors requiring early cooperation between software and hardware teams, rather than handing the design from one team to another after each stage is completed.
The analysis examines the renewed interest in what is known as hardware-software co-design, but under conditions different from those that prevailed about 30 years ago. At that time, the available technologies were not capable of making a significant difference, so the industry turned to assembling and integrating pre-verified IP blocks. Some remnants of the previous approach remain, such as virtual prototyping and untimed or approximately timed SystemC models, but these tools alone are not considered sufficient to represent today’s real-world workloads.
From Separating Teams to Virtual Models
Steve Roddy, chief marketing officer at Quadric, says the era in which a hardware team built its design and then handed it over to the embedded software team ended more than two decades ago. Over the past decade, virtual modeling has expanded to include physical inputs and sensor interfaces, in what is known as virtual-twin modeling.
But the current challenge is more complex: systems have become much larger and are designed for software-defined workloads, while software iteration is faster than hardware development. This makes co-design a process in which hardware is constantly catching up with software. Arvind Srinivasan of Normal Computing believes that the abstraction layers that historically separated parts of the software stack from parts of the hardware stack helped manage complexity and improve development efficiency and reliability, but the need to extract the maximum optimization from every part of the toolchain, particularly for performance-sensitive AI workloads, is changing that balance.
Addressing the problem requires a broader understanding of the word “architecture.” It refers not only to the overall system structure or to the microarchitecture and instruction set, but also to software architecture and the workload that will actually be executed. Andy Meier of Siemens EDA says architecture selection should begin with the application or workload, and with how that workload affects the system structure, instructions and various elements within it.
The Specialization Wave Returns Driven by Power
The analysis draws on an observation by Tsugio Makimoto, who served as chief technology officer at Sony during the 1990s, about the industry’s movement in waves between specialization and generalization. After a period in which general-purpose processors dominated broad sectors, the computing requirements of the edge, followed by those of data centers, began driving the industry toward more specialized solutions.
In edge devices, the motivation is linked to extending battery life within available power limits. In data centers, power density and heat have begun limiting the available options for running AI workloads. As a result, general-purpose processing units alone are no longer the answer; instead, heterogeneous data centers containing multiple types of chips have become necessary, with workloads distributed or aggregated according to the characteristics of each chip.
Patrick Coles, chief scientist at Normal Computing, notes that graphics processing units may not represent the endpoint of computing’s evolution, as evidence grows of the expanding role of ASICs. Frank Schirrmeister of Synopsys also explains that hyperscale computing companies are developing their own chips and accelerators to achieve joint optimization of key performance indicators, such as performance and power consumption, rather than focusing on speed alone.
However, specialization is not solely a technical decision. It is necessary to determine how much customization is economically worth its cost and whether it will provide sufficient gains in performance and power. Purna Mohanty, CEO of SignatureIP, notes that the complexity of specifications may make the customer themselves unable to express precisely what they want. This requires continuous alignment between the IP provider and customer needs, while balancing the advantages of off-the-shelf solutions against the benefits of developing new features.
RISC-V as an Example of Workload-Related Customization
RISC-V stands out in the analysis as an example of the trend toward “workload-designed chips.” Companies are developing custom extensions for cores to improve performance or power, but these extensions also require parts of the ecosystem, such as compilers and software integration tools.
Andrea Gallo, CEO of RISC-V International, explains that using custom instructions makes the company bear the full cost of ownership, including the toolchain and software integration. This approach may be a rapid means of innovation, but the company may later seek to turn the extension into a standard so that the cost of maintaining it is distributed across the broader ecosystem. The analysis also notes that knowing the software workload in advance makes it possible to tailor processing elements to the application, while general-purpose, scalable compute engines remain important when running unknown or changing software.
Power and Heat Expand the Scope of Co-Design
The impact of co-design is not limited to performance. Software engineers may understand the importance of power, but they do not always have the tools to measure the impact of their decisions early. Therefore, teams are working to provide power-consumption figures during software development in parallel with hardware development, and to improve metrics such as TOPS per watt, rather than raw TOPS alone.
The difficulty increases in heterogeneous applications that include multiple levels of cache, as well as other functions such as design-for-test, or DFT, because power and performance must also be analyzed within a realistic workload that includes these functions. As analysis expands to heat, electromagnetic interference and multiphysics aspects, the union of hardware and software teams becomes a new unit to which multiphysics analyses are added.
Some participants in the analysis suggest that AI could be used to understand the complex relationships among hardware, software, power and performance. But this requires data describing how much power the software consumed at different levels, from the chip and server to the rack, vehicle and data center. Until mature AI-driven solutions become available, reliance may continue on integrating IPs and selecting varying degrees of specialization.
A discussion of array extensions in RISC-V illustrates that one solution does not suit every use case: a small extension may reduce power, while a larger set of registers, operations and accumulators provides higher throughput, but at the cost of die area and power consumption. The choice ultimately depends on how cores and accelerators are distributed and on the target workload. The analysis concludes that the path toward vertical integration is still far from complete, but it has become necessary to align design with software, power and performance in modern systems.