Advanced memory technologies for AI

The rapid expansion of artificial intelligence, particularly large and generative models, has placed unprecedented demands on memory systems across cloud, edge, and embedded platforms. Memory capacity, bandwidth, energy efficiency, and scalability have emerged as primary constraints on AI performance and cost, often surpassing compute limitations. Conventional memory technologies, especially SRAM and DRAM, face growing challenges in advanced technology nodes due to power consumption, area scaling limits, and datamovement overheads.

 DRAM scaling highlights these challenges. Traditional 6F² DRAM cells have reached

fundamental limits in capacitor scaling, retention, and variability, driving transitions to more complex 4F² designs. As planar scaling stalls, further density improvements increasingly rely on 3D DRAM. However, Si-epitaxy–based 3D DRAM suffers from high cost and process complexity due to high-temperature steps, deep high–aspect-ratio structures, and yield challenges. IGZO-channel–based 3D DRAM offers a more scalable alternative, enabling lowtemperature processing, improved uniformity, lower leakage, and better cost efficiency for multi-layer stacking.

Beyond monolithic integration, customized DRAM combined with tight-pitch hybrid bondingprovides a promising high-density off-chip embedded memory solution. By tailoring DRAMarchitectures to AI workloads and using fine-pitch hybrid bonding to closely integrate memory with logic, this approach delivers near-on-chip bandwidth and energy efficiency while avoiding the scaling and cost penalties of deeply embedded SRAM. Tight-pitch bonding minimizes interconnect length and parasitics, enabling high bandwidth, low latency, and reduced power consumption, making it especially attractive for edge and AI accelerators.

To address the growing need for much higher memory capacity in AI workloads, highbandwidthmemory (HBM/HBF) can be combined with single-level-cell (SLC) 3D NAND. In this hybrid hierarchy, HBM provides ultra-high bandwidth and low latency for active model parameters, while SLC 3D NAND offers orders-of-magnitude higher capacity with improved endurance and latency compared to multi-level NAND. This complementary approach enables scalable, cost-effective memory systems that support large AI models without sacrificing performance.

Together with heterogeneous integration and emerging non-volatile and embedded memories(MRAM, FeRAM, FeFETs, and eDRAM), customized DRAM, tight-pitch hybrid bonding, and HBM–SLC 3D NAND integration highlight the shift toward memory-centric system design. These approaches underscore the importance of co-optimizing memory technology, packaging, and architecture to meet the performance, power, and capacity demands of the AI era.