Follow the latest coverage, related explainers and connected technology stories.
Data centers are moving toward heterogeneous clusters that combine CPUs, GPUs, NPUs, custom accelerators, and optical interconnects, rather than relying on a single GPU for all tasks. This shift makes software, network management, and power key factors in reducing token costs and improving hardware utilization.
A Semiconductor Engineering analysis argues that the traditional separation between hardware and software teams is no longer suitable for modern systems, particularly as AI workloads accelerate and power and thermal constraints intensify. This is driving companies to design more specialized hardware in parallel with software, while challenges involving tools, cost and standardization remain.