Follow the latest coverage, related explainers and connected technology stories.
Open-weight AI models give organizations greater flexibility for local deployment and reduce reliance on service providers, but they do not automatically mean that training data, code, or usage rights are available. Semiconductor Engineering’s analysis confirms that their impact extends to memory design, interconnects, security, and chip-design-tool workflows.
A paper from UC Berkeley and FuriosaAI examines the use of High Bandwidth Flash to expand the memory of AI accelerators when running large language models. The simulations indicate a reduction in completion time of up to 87%, with potential improvements in energy consumption and write endurance through cache-aware scheduling.
Rani Borkar of Microsoft believes that the success of AI infrastructure should not be measured by the number of chips or tokens processed, but by the amount of useful intelligence produced from each unit of energy, memory, and investment. She calls for an integrated design extending from data centers and chips to models, software, and agents.
Sharad Chol of Expedera analyzes how the transition of edge devices from traditional vision networks to LLM and VLM models is shifting the nature of the bottleneck from computational capacity to memory traffic. He outlines the role of packet-based processing in reducing external data transfers and improving model execution inside vehicles and embedded devices.