Endless Memory with CXL®: A Recap
The AI Leadership Forum, managed by IT Brand Pulse, hosted the first episode of the Endless Memory with CXL® series on September 30, 2026. The speakers came from an analyst firm and companies actively involved in CXL implementations, and this first episode set the stage for the sessions to follow. This episode examined why AI needs so much memory, the supply and cost pressures around DRAM, and practical ways of closing the gap. Here’s the plan for the sessions in this series.
Episode 2 –KV Cache
Episode 3 – Reusing DDR4
Episode 4 – Memory Pooling
The presentations and videos for the Episode 1 are available at https://www.linkedin.com/pulse/tv-series-your-next-ai-tech-binge-frank-berry-8hgpc/
In this article I review some of the key points delivered in these presentations. Figures are quoted as presented by each speaker.
Why CXL Matters by IT Brand Pulse
In this session, Frank Berry from IT Brand Pulse started the discussion by stating that high memory capacity and lower memory cost are two of the biggest needs in the AI Infrastructure world and the ability of CXL to address both the concerns. He illustrated the growing appetite for memory: HBM per GPU has grown 3.6X, from 80 GB on the H100 to 288 GB on the GB300, and a single GB300 NVL72 rack holds about 20 TB of HBM plus 17 TB of DRAM, a combined 37 TB of GPU and CPU memory per rack. He then highlighted the sharp rise in DRAM prices, headlined as 5.67X for high-capacity RDIMMs between 3Q24 and 2Q26. Memory is also absorbing a growing share of spending: DRAM and NAND are projected to rise from 47% of major CSP CapEx in 2026 to 68% in 2027. This presents an opportunity to reuse DDR4 modules from decommissioned servers. An estimated 511 million DDR4 modules (24.5 EB) in total, of which about 179 million (8.6 EB) are reusable, against $86 to $258 billion of new DDR5 purchases at $10 to $30 per GB if DDR4 is not reused. DDR4 reuse might not be as straightforward as it sounds.
View the recording of this presentation here.
CXL Market Update by Objective Analysis
I have known about Jim Handy since Objective Analysis first presented its reports on the CXL market a couple of years ago. This was an interesting session with the agenda organized around the popular CXL features, the ones that set CXL apart from other memory technologies. He discussed CXL in the context of the following CXL features:
Large memories
High bandwidth
Support for any memory type or speed
Memory pooling
Shared memory
Fabrics
Coherency
As in several other sessions of this episode, Jim Handy opened by pointing to the scarcity of DRAM. Contract prices are about 3X their 2024 level, OEMs are signing multi-year purchase agreements, and some are being told to look elsewhere for supply. His chart of lowest spot prices showed branded DRAM up 7.1X and NAND flash up 6.9X from mid-2025 levels. What makes the situation more critical is that memory is a key performance bottleneck.
One of the important features of CXL is that it enables memory expansion using any DDR Generation. (DDR4, DDR5, and so on)
He also mentioned the use of caching software to manage hot and cold data, with DDR4 on CXL holding the cold data while the host’s DDR5 holds the hot data and noted that hyperscalers are already salvaging DDR4 from decommissioned servers and fitting it into DDR5-based servers with CXL cards. He cited Meta's Vistara DDR4 paper, whose figure shows improvements in throughput and compute capacity requirements with the Vistara CXL device as the model size is scaled from about 5 TB to 20 TB.
Also presented was the relative performance versus price per gigabyte chart that reminds one of the memory pyramid that we are so used to seeing in the memory world.
Here is another illustration on the CXL feature of memory expansion using any kind of memory and a switch from Jim’s slides.
Memory pools offer some relief from the problem of stranded memory, which is the memory that lies underutilized in a server because of a lighter workload while other servers starve for memory to serve their workloads. Memory pools aggregate the memory modules using CXL and enable more efficient utilization of the scarce memory resources.
Similarly, the shared memory technique enables more efficient memory utilization.
Jim also touched on the remaining features: fast DMA transfers between server and GPU memory over PCIe, fabrics that boost data transfer between CPUs, GPUs, and memory, and coherency, the mechanism that keeps every processor working on the same data when shared memory cached in one processor is modified.
In his closing remarks, Jim forecast slow and steady growth of the CXL market, from roughly a few billions in 2026 to about $16 billion by 2031. He also reminded the audience that DRAM is a commodity: shortages and oversupplies are inevitable, and hyperscaler AI demand cannot grow forever.
View the recording of this presentation here.
CXL: From Standard to Industry Transformation by Intel/CXL Consortium
Anil Godbole, Chair of the CXL Consortium’s Marketing Work Group, and Xeon CXL Strategy and Marketing Manager at Intel presented this session. The session started by tracking the progression of the CXL specification from CXL 1.0 (March 2019) to CXL 4.0 (November 2025). CXL 4.0 doubles the data rate to 128 GT/s with zero added latency and adds CXL bundled ports and memory RAS enhancements. Anil then presented a slide on the value proposition of CXL memory, listing higher memory capacity, higher memory bandwidth, and lower memory TCO as the key benefits.
There was a mention of Microsoft Azure’s announcement of the first cloud VM with Intel Xeon 6 and CXL memory, which was in private preview at the time. Here’s a link to the blog post. Also worth a mention are the slides on Intel Flat Memory Mode, a hardware-based CXL memory expansion technique in which both DRAM and far memory are exposed to the OS as a combined physical memory, or one memory tier. Its advantages are good performance at a 1:1 near/far memory ratio and no need for dedicated CPU cores for hot/cold page telemetry; hot data is swapped into DRAM one cacheline at a time rather than a whole 4 KB page. Intel reported up to 10% better query throughput and up to 10% lower latency than software-controlled tiering on MongoDB, and 40%+ TCO savings with DDR4 RDIMM reuse on SAP HANA.
Anil then moved to memory pooling, introduced in CXL 2.0 and CXL 3.0, which allows data sharing by reference rather than shuffling data over networks. He cited three recent demonstrations: Alibaba on distributed database acceleration, Micron on a vectorized database for RAG (Retrieval Augmented Generation), and a KV cache repository with Nvidia Dynamo. Before summarizing his presentation and sharing the call to action, he presented a slide on how CXL is gaining traction as a performance enabler for AI, with agentic AI as the next big use case: CXL memory pools, as an alternative to SSD pools, offer big data footprints, naturally co-located data, and better latency and power consumption. He also stated that Nvidia acknowledged at FMS 2026 that CXL memory pooling will be the best solution for the agentic AI era.
View the recording of this presentation here.
CXL and the New AI Memory Hierarchy by Liqid
This presentation began with Steve Scargall, Chief Architect at Liqid, mentioning the growing use of AI and the shift of AI compute from training to inference. Liqid’s slide showed inference rising from about a third of AI compute in 2023 to an estimated two-thirds in 2026. The rapid growth of inference is driving demand for larger memory footprints across the AI stack, in particular for KV cache, which scales with context length. The speaker then made the case for moving latency-sensitive applications from NVMe to CXL, using the memory pyramid as the supporting figure and citing 100x to 200x lower latency than NVMe.
The speaker presented Liqid’s CXL-based hardware products and Matrix orchestration software for managing memory pools, including the EX-5410C memory expansion chassis, specified at up to 20 TB of DRAM per chassis, scalable to 100 TB per pod, at roughly 500 ns latency. He also discussed results for CXL memory pooling with Nvidia Dynamo KV cache offload, including a claim that pooling delivers 99% of native DRAM performance, and industry results, cited from published papers, with noteworthy throughput gains depending on the study and workload.
View the recording of this presentation here.
The Memory Advantage in the AI Era by Penguin Solutions
This presentation by Arthur Sainio, Director, DRAM, and David Selle, Manager, Business Development, from Penguin Solutions made the case for matching the memory type to the workload. It opened with the pressures the memory world is grappling with: compute has outpaced memory, memory demand keeps growing in the agentic AI era, HBM supply is constrained, and high memory cost has become the norm. On pricing, they showed 64 GB DDR5 RDIMM contract prices rising from $873 in 1Q26 to a forecast $1,586 by 4Q26, an 82% rise, and projected no return to 2023 pricing through 2030 (a projection built from third-party analyst sources). They also called on organizations to take a multi-faceted approach, given that no single memory solution addresses all four challenges.
The speakers stressed the importance of workload-optimized memory architectures, namely DDR5 RDIMM for mainstream capacity, MRDIMM for bandwidth, LPDDR5x SOCAMM2 for power efficiency, and CXL memory for capacity expansion, as a shift away from one-size-fits-all DRAM.
The following slide summarizes the purpose of each memory architecture, the best-fit workloads, and key architectural attributes of each.
This was followed by detailed descriptions of each memory architecture. For CXL, citing TechInsights, they showed CXL-attached DRAM per server growing from 402 GB in 2025 to 1,741 GB by 2031, 49% of servers using memory pooling by 2031, and reused DDR4 meeting 74% to 86% of early CXL expansion demand through 2027.
View the recording of this presentation here.
Big Data Feeds Big Compute through Memory by Samsung
Siamak Tavallaei, Sr Principal Engineer, Samsung Semiconductor Inc. (SSI), presented this session by first introducing a three-point premise as follows:
The future of AI isn't just about faster compute; it’s about faster data movement enabled by smarter memory to accelerate the next generation of autonomous agents
Standardization is key to help suppliers benefit from their investments: Win/Win
Standard hardware will encourage broader software development: a positive feedback loop to garner higher RoI
The advent of Big Data accelerated storage technology, including NAND and SSDs. That era was marked by huge amounts of labeled, human-generated data kept on permanent storage. The Big Compute era of today is marked by AI/ML and machine-generated data, with large scratchpads for working sets and space for long-term memory. Unlike the Big Data era, the need now is for faster, temporary space like DRAM, with interconnects helping data move fast. The speaker then described solutions and experiments from the Samsung Research team to explore this landscape: the latency-focused S-CHMU (Samsung CXL Hot-page Monitoring Unit), a device-level page migration technology; the bandwidth-focused CXL-PNM (Processing Near Memory), which pre- and post-processes data on the CXL memory side; and the capacity-focused Pangaea (CXL pooling software), which increases effective memory capacity by accessing stranded memory.
The speaker’s point was that, with the transition to agentic AI, the bottleneck has shifted from compute to data movement, and that memory is where data is kept in transit and staged before execution. He illustrated this with a RAG pipeline: different pipeline phases need different amounts of memory, so instead of overprovisioning for peak demand, memory pooling lets compute nodes inflate or deflate their memory footprint.
View the recording of this presentation here.
Conclusion
Across the six sessions, the common thread was that memory capacity and cost, and not compute alone, now constrain AI infrastructure. CXL’s nearest-term value appears to be memory expansion and DDR4 reuse, with pooling and sharing for KV cache and agentic workloads as the next wave.
If you attended Episode 1, I would like to hear which session or claim you found most compelling, or most debatable. Please share your thoughts in the comments.
