As the pace of AI model development accelerates, chip designers face increasing pressure to shift from general-purpose computing to highly specialized infrastructures. At the heart of this transformation, the RISC-V architecture emerges as a controversial choice; while some see it as a means to achieve unprecedented flexibility through custom instructions, others fear it may impose engineering complexities that undermine the efficiency of AI acceleration systems.
The Dilemma of Decoupling the Processor and Accelerator
Experts present a contrasting view on using RISC-V for heavy computational tasks (Inference). Steve Roddy, from Quadric, argues that starting with a CPU architecture to design an AI accelerator might be a "bad idea" from an engineering perspective. The reason lies in the problem of partitioning; designers are forced to split work between a programmable processor and an external accelerator.
This partitioning leads to several technical challenges:
- Data movement complexity: The need for highly sophisticated compilers to manage data traffic between the accelerator and the processor via a Network-on-Chip (NoC).
- Architectural inefficiency: Traditional processors are designed to track pointers and variable logic, whereas AI operations require predictable matrix-by-matrix calculations for millions of cycles, which units like Google's TPU excel at, unlike traditional RISC-V processors.
- Problem scaling: As the size of the accelerator increases, managing this partitioning and the cost of resolving data movement issues become significantly more difficult.
RISC-V as the "Brain" of Accelerators, Not the "Muscle"
Conversely, Dave Kelf, CEO of Breker, believes the true value of RISC-V lies not in performing heavy calculations, but in acting as a control function above the accelerators. In modern systems, complex state machines often fail to manage operations, and this is where RISC-V comes in to provide:
- Flexible control: The ability to manage divergence in SIMT (Single Instruction, Multiple Threads) computing operations efficiently.
- Custom load/store instructions: Reducing latency between the accelerator and the processor by adding special code instructions to optimize data movement.
- Avoiding royalties: Providing an economic alternative to proprietary processors (such as Arm or Tensilica) without paying high licensing fees, while maintaining the ability to customize.
Verification and Observability Challenges in Custom Systems
The challenges do not stop at instruction design but extend to the verification stage. Francisca Tan from Siemens EDA points out that high customization requires advanced observability capabilities to track custom instructions during operation. Furthermore, Ewald Liess from SignatureIP warns that multi-core systems in high-performance environments may lead to issues with cache coherency and deadlocks in the fabric, meaning instructions might work correctly in software, but the chip may fail due to data traffic bottlenecks.
What Changes Practically for Designers?
This discussion points to a fundamental shift in how RISC-V is used in the AI market. Instead of attempting to compete with specialized processors in matrix calculations, the trend is moving toward using RISC-V as an intelligent orchestrator. This means that companies adopting RISC-V must focus on:
- Building custom instructions aimed at reducing latency rather than increasing raw computational power.
- Investing heavily in verification and observability tools to ensure system stability during traffic bursts.
- Focusing on the integration between the processor and the Network-on-Chip (NoC) to ensure that flexibility does not become a burden in data management.