NVHBM: NVIDIA's Custom High Bandwidth Memory Architecture for Semi-Custom AI Silicon

Recently, NVIDIA introduced a new architecture, called NVHBM, which pulls the memory controller out of the XPU chip and puts it inside the HBM stack's own base die. Amazon's Annapurna Labs is already on board for Trainium4.