NVHBM: NVIDIA's Custom High Bandwidth Memory Architecture for Semi-Custom AI Silicon
HBM has always had two owners. The DRAM vendor builds the memory stack and the base die beneath it; the accelerator designer supplies the controller and PHY that talk to it. JEDEC standardizes the seam between the two, and every qualified HBM stack is supposed to work with every compliant controller because of it.
On August 26, 2026, NVIDIA said it wants to own that seam. The result is NVHBM, a custom high bandwidth memory architecture that moves the memory controller off the XPU compute die and into the HBM base die itself.
Drivers for NVHBM
Every square millimeter of an XPU's compute die is expensive, and increasingly, hyperscalers are the ones paying for it. Custom silicon programs like AWS's Trainium want that area spent on matrix engines, not interface circuitry.
In a conventional HBM implementation, the memory controller and PHY sit on the XPU die itself, substantial silicon area and I/O real estate spent just talking to memory, before a single AI computation happens.
Agentic AI, trillion-parameter models, and KV cache-heavy inference are pushing memory bandwidth demand up sharply. Reading model weights, activations, and KV cache entries fast enough to keep compute fed is increasingly the bottleneck, not raw FLOPs.
Hyperscalers building their own silicon still want to plug into NVIDIA's broader rack-scale ecosystem, NVLink, NVLink Fusion, MGX, without separately designing, integrating, and qualifying a memory controller with each of the “Big Three” DRAM makers.
What Is NVHBM?
NVHBM is an extension of NVLink Fusion, NVIDIA's platform (introduced in 2025 and expanded through GTC 2026) for connecting third-party CPUs and XPUs into NVIDIA's scale-up and rack-scale architecture. NVLink Fusion's existing partner list includes MediaTek, Marvell, Alchip, Astera Labs, Synopsys, and Cadence on the custom-compute side, and Fujitsu and Qualcomm Technologies on the CPU side.
Where NVLink Fusion addresses the system-level, rack-scale interconnect, NVHBM addresses the package-level memory interface. In a standard HBM4E implementation, the memory controller lives on the XPU compute die, and the bus between XPU and HBM stack must be JEDEC-compliant so that any qualified stack works with it. NVHBM instead moves that controller into the HBM stack's own base die, a custom base die co-designed and validated by NVIDIA together with memory vendors.
The DRAM core layers inside the stack are untouched; it's the base die and its interface to the XPU that becomes custom. Because that interface no longer has to be a JEDEC-standard bus, NVIDIA can design it purely for speed rather than for cross-vendor interoperability. NVIDIA has said it intends to establish a standard NVHBM implementation available from multiple memory suppliers, so this isn't meant to be single-sourced, but the specification itself is authored by NVIDIA rather than by a consortium.
The USP
NVIDIA's published figures for NVHBM versus standard HBM4E:
Up to 30% greater memory bandwidth
Up to 15% lower HBM power consumption
Up to 25% more usable XPU compute die area
Up to 67% less PHY and supporting interface area
Up to 80% more usable silicon across the overall package layout
The rationale: HBM4 already runs a roughly 2,048-wide parallel data bus, and packaging constraints make it impractical to widen that further. The alternative lever is pushing per-pin speed, which is exactly where HBM4E variants have been headed, but pushing per-pin speed demands a PHY design effort that has historically tripped up memory vendors more used to optimizing for density and yield than raw interface speed.
There's also a value-chain angle I’ll mention: industry reporting has pointed out that the HBM base die can cost several times as much as the DRAM core die itself. Whoever designs the base die captures a disproportionate share of a stack's value, and NVHBM puts that design pen in NVIDIA's hand.
Ecosystem Players
NVIDIA is the architecture's originator and sets the base-die interface. Amazon's Annapurna Labs is the first named NVHBM collaborator, working on it alongside NVLink Fusion adoption for Trainium4. In NVIDIA's announcement, Annapurna Labs VP Nafea Bshara called NVHBM “a new architectural approach to advancing high-bandwidth memory performance and efficiency.”
On the supply side, Korean trade press has reported that Samsung is developing a custom 8-layer HBM4E part targeting 17–18 Gbps per pin specifically for NVHBM, trading stack height for pin speed and manufacturability. That's separate from Samsung's standard 16 Gbps, 4.0 TB/s HBM4E part destined for NVIDIA's Vera Rubin Ultra platform. SK hynix and Micron, the other two of the “Big Three” HBM suppliers, haven't been named as NVHBM collaborators yet, though the expectation across the industry is that comparable custom base-die parts from both will follow.
Honest Caveats
NVHBM departs from the JEDEC-standardized interface that has historically let any qualified HBM stack from any vendor work with any compliant controller. NVIDIA proposing its own base-die design as “the standard” is a different kind of standardization, one NVIDIA authors, not an open consortium. Whether memory vendors converge on identical NVIDIA-defined base dies, or each negotiate bespoke variants, isn't yet clear.
Concentrating base-die design with NVIDIA plausibly shifts value capture in NVIDIA's direction. Some industry commentary, not NVIDIA's own framing has read NVHBM as pushing Samsung, SK hynix, and Micron closer to commodity DRAM suppliers, given how much of a stack's cost sits in the base die. That's one interpretation circulating, not a confirmed strategic goal.
The bandwidth, power, and area figures are all “up to” numbers from NVIDIA's own announcement. No NVHBM-based product has taped out publicly yet, so none of this has been independently verified in shipping silicon.
Samsung's reported 8-layer, 17–18 Gbps HBM4E part for NVHBM comes from Korean trade press (Seoul Economic Daily) as of this writing, not an official Samsung confirmation. As of now, it is more of a report, not a spec sheet.
Timelines are less immediate than the announcement's tone suggests. AWS has indicated Trainium4 deliveries are expected to begin in 2027, and AWS's own materials describe Trainium4 as scaling with both NVLink Fusion and UALink, this looks like coexistence with AWS's existing interconnect roadmap, not a wholesale replacement of it.
JEDEC, separately, is developing SPHBM4, an open standard for reduced-pin-count, HBM4-equivalent-throughput memory on organic substrates. NVIDIA's proprietary route isn't the only one being pursued to solve similar pin-count and interface problems.
Outlook
NVHBM reads as NVIDIA extending its NVLink Fusion strategy one layer deeper, from the rack-scale interconnect down into the memory package itself. For now it's aimed squarely at hyperscalers building semi-custom XPUs, with Trainium4 as the named example, rather than at NVIDIA's own GPU line. But NVIDIA has said NVHBM uses the same memory technology it plans to use in future GPUs, so the line between “custom silicon feature” and “NVIDIA's own roadmap” may blur over time.
Whether Samsung, SK hynix, and Micron treat this as an acceptable trade, ceding base-die design control in exchange for guaranteed volume, or push back with differentiated base-die offerings of their own, is worth watching over the next few quarters, particularly as Samsung's reported custom part for NVHBM firms up.
As always, if you've come across other NVHBM developments, or read this differently, I'd love to hear it in the comments.
