### Summary
– NVIDIA’s new Inference Context Memory Storage (ICMS) technology could worsen the global NAND supply shortage.
– This innovation allows for significant improvements in inference speed and energy efficiency, potentially demanding more NAND resources.
– Major NAND suppliers like Samsung and SK Hynix are expected to benefit from this technological advancement.
—
In a recent analysis, Citibank highlighted that NVIDIA’s latest technology, Inference Context Memory Storage (ICMS), is set to exacerbate the ongoing global NAND supply shortage. This development arises from the increasing demands of artificial intelligence (AI) technologies, especially during the inference process.
### Understanding the ICMS Technology
The AI inference process relies on mechanisms such as KV caching, which function similarly to the “short-term memory” of AI models. Traditionally, these caches have relied heavily on expensive High Bandwidth Memory (HBM). However, with the significant expansion of context lengths required for modern applications, HBM has become a bottleneck, limiting performance.
To overcome this challenge, NVIDIA has introduced the ICMS platform, built upon the capabilities of the BlueField-4 Data Processing Unit (DPU). This platform establishes a more efficient storage layer designated as G3.5, which relocates the KV cache from HBM to a more scalable local solid-state drive (SSD). The result is a staggering increase in inference speed—up to five times quicker—along with a fivefold improvement in energy efficiency and a notable reduction in latency.
### Implications of the Vera Rubin System
Citi’s report predicts that to effectively operate ICMS technology, NVIDIA may configure up to 16TB of additional storage per rack within the Vera Rubin NVL72 AI setup, culminating in a total SSD capacity of around 1,152TB. Market forecasts indicate a surge in Vera Rubin server shipments, projecting 30,000 units by 2026 and escalating to 100,000 units by 2027.
This anticipated growth translates to a significant uptick in NAND demand, with estimates showing an additional need of 34.56 million TB in 2026—accounting for approximately 2.8% of the total global demand—and skyrocketing to 115 million TB in 2027, equaling 9.3% of total NAND requirements.
### Extended Demand Beyond Vera Rubin
It’s noteworthy that Citi’s projections only consider the Vera Rubin platform’s specifications. The analysis does not account for the even higher specifications of the Rubin Ultra system or potential demand driven by the forthcoming Feynman architecture. Thus, the actual growth in NAND demand may surpass these conservative estimates.
### Opportunities for NAND Suppliers
Citigroup’s analysis indicates that NVIDIA’s integration of ICMS technology may serve as a strong catalyst for NAND suppliers. Industry giants such as Samsung, SK Hynix, Kioxia, Western Digital, and Micron are positioned to be the primary beneficiaries of this shift. As demand grows, these companies stand to gain substantially from the evolving landscape of memory technology.
### Future Considerations
As we advance into a more AI-driven future, the implications of NVIDIA’s ICMS technology will likely reverberate throughout the semiconductor industry. The ensuing demand for NAND resources will not only impact the manufacturers but also influence market dynamics, pricing, and technological progression across various sectors reliant on memory solutions.
In conclusion, NVIDIA’s introduction of ICMS technology signifies a major leap forward for artificial intelligence applications, but it also presents challenges regarding NAND supply. For suppliers and manufacturers, adapting to these rapid changes will be crucial in maintaining competitive advantages in a fast-evolving market.
