High-bandwidth memory HBM is facing growing physical and manufacturing limits as system companies continue requesting higher bandwidth to process AI workloads. According to discussions at Hot Chips 2026, the most direct solution so far—adding more DRAM dies to the package—increases the difficulty of thinning dies, managing heat, achieving alignment, and maintaining acceptable yields.
HBM uses three-dimensional stacks of memory dies above a base logic die, with the layers connected through thousands or tens of thousands of through-silicon vias (TSVs). Increasing the number of these connections enables higher bandwidth, but consumes area that could otherwise be allocated to memory cells. Therefore, the performance increase does not produce a corresponding increase in capacity per wafer; instead, it raises silicon consumption and manufacturing capacity requirements per bit compared with conventional DRAM.
Every Additional Layer Raises Manufacturing Risks
DRAM dies must be thinned so that the height of the HBM unit remains within package limits. However, thinner dies are more difficult to handle during manufacturing and packaging and are more susceptible to warping. This affects wafer handling, bump alignment, and die-to-die bonding. A taller stack also increases power density and lengthens the thermal path within the unit.
The likelihood of defects increases with each new layer. Voids may appear in the underfill material, or structural defects may result from misalignment or process variation. Any one of these defects can turn the entire stack into high-cost scrap, putting pressure on yield and reducing the actual supply of DRAM.
SK hynix provided an example of these challenges in its work on 16-die HBM stacks. Jaesik Lee, vice president of package engineering at SK hynix America, said during his presentation at Hot Chips 2026 that increasing the cube thickness from 720 to 775 microns provides additional margin, but the dies still need to be reduced in thickness by 10% compared with a 12-die stack. Other challenges include reducing gap height, shrinking bump pitch, thinning die walls, and increasing power density and bandwidth.
Bandwidth Comes at the Expense of Area
Sangwook Han, a member of the Samsung Memory design team, said that HBM3E contains 128 banks in each DRAM die, while HBM4 reaches 256 banks per die. This architecture requires more data paths and peripheral circuits to connect the dies to the base die, and then to a GPU or XPU through microbumps and the interposer.
This architecture explains why it is difficult to improve performance simply by accelerating TSVs: increasing the speed of these connections may be difficult or risky, so the design has shifted toward increasing their number. However, a larger TSV array consumes die area, which requires the spacing between them to be continually reduced. Expanding the input/output PHY circuits in the base die is more complex, according to Han.
During his keynote, Jim Handy of Objective Analysis estimated that the amount of memory produced per wafer in HBM is roughly one-third that of conventional DDR. This means that HBM production consumes a large number of wafers, while different types of DRAM are manufactured on closely related process lines; consequently, increased HBM demand can put pressure on DRAM supplies as a whole.
Cooling Is Becoming an Architectural Constraint
Heat is not distributed evenly within an HBM stack. The base die is usually the hottest because it performs more complex work and contains high-speed connections, while the heat sink is located at the top of the stack. The height of the stack adds thermal resistance between the heat source and the cooling solution.
Raghu Sreeramaneni, a fellow of HBM design architecture engineering at Micron, said that rising temperatures could make DRAM cell refresh operations a reliability problem. As a result, thermal solutions have become a factor guiding the design of the system itself, rather than a subsequent step after the design is complete. This is an important point for system package designers who combine multiple GPUs and HBM units in a single space.
Why Does This News Matter?
The analysis links HBM constraints to a broader problem in the memory market. Handy identified three causes of the DRAM shortage: accelerating AI spending and demand for high-speed memory, reduced availability of DRAM wafers because HBM consumes more per bit, and insufficient manufacturing capacity to keep pace with demand.
The source notes that companies have not added new manufacturing capacity for more than ten years because market growth allowed them to increase the number of gigabytes per wafer instead of building new fabs. Now, building a new DRAM fab takes at least two years, and the process may take longer depending on the availability of power, water, and labor, as well as environmental restrictions. There is still no guarantee that the expansions currently under construction will be sufficient to restore balance to prices and supply.
Another gap is emerging between the growth of computing capacity and memory. According to Sreeramaneni, computing expands by approximately three times every two years, while HBM grows by about two times over the same period, meaning that the “memory wall” has not disappeared and may become more pronounced.
Future Development Paths
The industry is exploring improvements beyond adding new layers, including improved SerDes PHYs for memory, larger interposers or interposers made from different materials, higher-bandwidth channels, and the eventual introduction of optical technologies. Interconnect processes may also shift from thermocompression-based microbumps to fusion bonding and hybrid bonding, which support pitches down to the single-digit-micron range and may reduce thermal resistance by minimizing the insulating layers between dies.
The source does not offer a quick solution to the shortage or thermal constraints. However, it makes clear that expanding HBM has shifted from being a matter of increasing memory density to a challenge affecting the entire system architecture: from manufacturing processes and yield, to packaging and cooling, and then to where data is processed. As AI continues to spread, this could drive distributed processing that includes processing in memory or in edge sensors, reducing the amount of data sent to data centers.