ROCm: AMD's Open-Source Path Through the AI Software Stack

gpusoftware
Written By Dhupinder Malhotra

I have spent a lot of time on writing about hardware topics, related to memory, interconnects, networking. But an AI accelerator is only as useful as the software that lets it run a model, and for AMD's Instinct GPUs, ROCm is that software. If CUDA is the reason NVIDIA's hardware advantage has proven so durable, ROCm is AMD's answer to the same problem: how do you get PyTorch, vLLM, and a few hundred thousand developers to treat your silicon as a first-class citizen rather than an afterthought.

ROCm has had a long, occasionally rocky decade getting there. It's now at an interesting point, AMD just marked ROCm's tenth anniversary with a major version bump to ROCm 10.0 and a new agentic-AI layer called ROCm.AI, on the back of billions of dollars in AMD-OpenAI and AMD-hyperscaler compute commitments that make the software stack's readiness a very real commercial question, not just a technical one. This article looks at what ROCm actually is, how it got here, and how close the long-discussed "CUDA gap" really is to closing.

From Close to Metal to ROCm: A Decade in Brief

AMD's GPU compute software has gone through several names before ROCm. The lineage runs from Close to Metal, AMD/ATI's first GPGPU stack, through Stream, and drew on ideas from the Heterogeneous System Architecture (HSA) initiative, which aimed at a hardware-agnostic intermediate representation for GPU and accelerator code. ROCm itself launched on November 14, 2016, under the name Radeon Open Compute Platform, as part of what AMD called the Boltzmann Initiative, the company's push to build an open, Linux-first GPU compute stack rather than a proprietary one modeled purely on CUDA.

"Radeon Open Compute" stopped functioning as a literal acronym once Open Compute turned out to be someone else's registered trademark, so today it's simply ROCm, free and open-source software (aside from GPU firmware blobs) distributed under the MIT license, with its meta-repository on GitHub. That open-source posture has remained ROCm's defining philosophical difference from CUDA throughout the past decade: where NVIDIA's stack is closed and tightly controlled, ROCm's source is inspectable, forkable, and, at least in principle, extensible by anyone.

At the center of ROCm sits HIP (Heterogeneous-compute Interface for Portability), a C++ API AMD introduced in 2016 that deliberately mirrors CUDA's programming model closely enough that a lot of CUDA code can be ported to HIP through fairly mechanical, largely automatable changes rather than a full rewrite. That resemblance is ROCm's central strategy for lowering the switching cost for developers who've spent a decade building muscle memory around CUDA.

What's Actually in the ROCm Stack

ROCm is not a single program; it's a layered collection of drivers, runtimes, compilers, and libraries. At the bottom sits the AMDGPU kernel driver, which talks directly to Instinct and Radeon hardware. Above that is the ROCm runtime and the LLVM-based HIP compiler toolchain; AMD has contributed substantially to upstream LLVM to keep ROCm's compiler technology aligned with the broader open-source compiler ecosystem, and has been migrating away from the older HIPCC compiler driver toward more direct Clang-based workflows.

Above the compiler sits a set of math, AI, and HPC libraries roughly analogous to CUDA's cuBLAS/cuDNN/NCCL family: hipBLASLt for matrix math, MIOpen for deep learning primitives, and RCCL (ROCm Communication Collectives Library) for the multi-GPU collective operations, allreduce, all-to-all, that large-scale training depends on. RCCL in particular received what AMD has described as the largest single investment it's ever made in the library, reflecting how central efficient multi-GPU communication has become to large training runs.

On top of all of that sit the frameworks developers actually touch day to day. PyTorch has shipped official ROCm support since PyTorch 1.8 in 2021, and by 2026 ROCm is a first-class PyTorch backend, not an experimental one, alongside multiple CUDA versions. vLLM and SGLang, the two dominant open-source LLM inference engines, both ship official ROCm support as well, and AMD has partnered directly with Hugging Face to keep model checkpoints and tooling working smoothly on Instinct hardware.

TheRock and the Six-Week Release Cadence

A quieter but consequential change has been how ROCm itself gets built and shipped. TheRock, AMD's automated open-source build and release system, reached production status with ROCm 7.14 in mid-2026, replacing a more fragmented, harder-to-reproduce build process. On the back of that, AMD announced at its Advancing AI event in July 2026 that ROCm would move from a roughly quarterly release cadence to a rigid six-week release cycle, a deliberate effort to get new features, performance optimizations, and hardware support into developers' hands faster, at the cost of a faster-moving target that some developers have flagged as its own kind of friction (more on that in the caveats below).

ROCm 10.0 and ROCm.AI

In August 2026, timed deliberately to coincide with ROCm's tenth anniversary, AMD made a major version jump straight to ROCm 10.0, skipping past incremental version numbers to signal an inflection point rather than another routine release. The headline feature is ROCm.AI, an "AI-native" developer experience layer that sits on top of the existing ROCm Core SDK and focuses less on raw primitives and more on the workflow around them: installing, validating, serving, and optimizing AI workloads with less manual toil.

ROCm.AI ships as three components in ROCm 10.0. The ROCm CLI is a version-agnostic command-line tool intended to give developers a consistent experience across ROCm releases even as the underlying stack changes every six weeks; it's been available as a technology preview since ROCm 7.13, with full ROCm 10 support arriving alongside this release. AMD Skills and Hyperloom round out the layer. Hyperloom, in particular, is described as a workflow tool, part of AMD's broader bet that AI development is moving toward agentic, workflow-driven tooling rather than developers hand-tuning individual kernels. AMD is explicit that this is a technology preview: expect the interface to keep evolving rather than treating it as a finished product.

Beyond ROCm.AI, ROCm 10 broadens hardware and deployment reach: expanded virtualization support, validated inference containers spanning Instinct, Radeon, and Ryzen products, and continued work on Windows compatibility alongside ROCm's traditional Linux-first focus, the upcoming ROCm 7.2 release earlier in the year had already extended Windows/Linux compatibility and made new PyTorch builds directly accessible through AMD software on Windows, part of what AMD says drove roughly a tenfold year-over-year increase in ROCm downloads.

Ecosystem and Adoption

ROCm's commercial relevance has grown in direct proportion to AMD's Instinct GPU roadmap. Microsoft Azure, Meta, Oracle, Dell, HPE, Lenovo, and Supermicro have all adopted Instinct MI300X-generation hardware running on ROCm, going back to AMD's December 2023 Advancing AI event, where ROCm 6 launched alongside the original MI300 series. Microsoft built its Azure ND MI300X v5 virtual machine series around it for Azure OpenAI services; Meta has described MI300X as broadly deployed for Llama 3 and Llama 4 inference, citing compute, performance-per-TCO, and the accelerator's large HBM3 memory capacity as reasons for continuing on to the MI350 generation.

The step change came in October 2025, when AMD and OpenAI announced a six-gigawatt, multi-year, multi-generation agreement, starting with a 1-gigawatt deployment of AMD Instinct MI450 Series GPUs in the second half of 2026 and scaling from there, with a warrant for up to 160 million AMD shares structured to vest as deployment milestones are hit. AMD has said seven of the ten largest AI customers now deploy Instinct accelerators, and named Meta, OpenAI, Microsoft, Oracle, Cohere, HUMAIN, Red Hat, Astera Labs, and Marvell as active collaborators at its most recent Advancing AI event. None of that adoption happens without ROCm; the hardware and software commitments are, at this point, inseparable.

AMD's memory-capacity advantage has become a differentiator alongside the software story: the MI300X ships with up to 192GB of HBM3, and the MI350X moves to up to 288GB of HBM3E on a 3nm process with FP4/FP6 datatype support, both comfortably ahead of competing NVIDIA accelerators on per-GPU memory capacity, which matters directly for how large a model or KV cache fits on a single GPU before requiring sharding.

The CUDA Gap: How Much Is Left?

For most of ROCm's history, the answer to "how does it compare to CUDA" has been: the hardware is competitive, the software isn't quite there. That gap has narrowed substantially through 2025 and 2026, but it hasn't closed, and it's worth being specific about where it stands rather than taking either AMD's or NVIDIA's marketing at face value.

On raw inference throughput, independently reported figures have moved a lot. Earlier MI300X benchmarks showed roughly 37-66% of comparable H100/H200 performance on LLM inference, a gap attributed by most analysts to software immaturity rather than the underlying silicon. By 2026, informal benchmarking from developers running local inference commonly cites ROCm trailing CUDA by something more like 15-25% on comparable consumer and workstation hardware, a meaningfully narrower gap than two years prior. One market-analysis firm has gone as far as quantifying the software disadvantage as a "CUDA Gap Score" ranging from roughly 29 to 99 across its benchmark suite, a wide range that itself signals how workload-dependent the comparison is.

Where the gap persists most visibly is training rather than inference: CUDA's multi-GPU tooling, NCCL, Transformer Engine, FP8 training recipes remains, by most accounts, smoother and better pre-tuned than ROCm's equivalents (RCCL, hipBLASLt) for large-scale training runs, even after RCCL's recent investment. CUDA-specific libraries like TensorRT-LLM and FlashAttention 3 don't yet have full ROCm equivalents, meaning workloads built around them pay a real porting cost on AMD hardware. Inference is the workload where the gap has closed furthest: PyTorch, vLLM, and SGLang all ship official, non-experimental ROCm support in 2026, and for many inference-only deployments, ROCm is now described by practitioners as "good enough for daily use" rather than a compromise.

Caveats and Open Questions

A few things to flag.

First, the performance comparisons above come almost entirely from independent blogs, dev-community write-ups, and one market-research firm's proprietary scoring methodology, not from a peer-reviewed or standardized benchmark suite that both AMD and NVIDIA have agreed represents real-world workloads fairly. The specific percentage figures (37-66%, 15-25%, a 29-99 "Gap Score") should be read as directional signals from different methodologies and different hardware generations, not as a single consistent number that's tracking downward over time in a clean line.

Second, ROCm's own pace of change is, by some accounts, becoming part of the friction rather than purely a solution to it. Moving to a six-week release cycle gets new hardware support and optimizations out faster, but several practitioner write-ups in 2026 explicitly flag AMD's version-numbering churn, including jumping from ROCm 7.x straight to 10.0, and technology previews that arrive ahead of full support in a later release, as its own ongoing cost for teams trying to standardize on a stable platform for production deployments.

Third, the setup experience remains meaningfully rougher than CUDA's for developers outside of validated, vendor-supported configurations: ROCm is still fundamentally Linux-first, with Windows support real but newer and less complete; consumer and workstation GPU support is narrower than Instinct's data-center support; and multiple independent write-ups describe ROCm setup as requiring more Linux comfort and more willingness to troubleshoot than CUDA's comparatively turnkey driver-and-toolkit installation.

Fourth, ROCm.AI, the centerpiece of the ROCm 10.0 announcement, is explicitly a technology preview as of this writing. AMD Skills and Hyperloom in particular are too new to have an independent track record; it's reasonable to treat the "AI-native developer experience" framing as a roadmap and a stated direction rather than a mature, battle-tested product today.

Finally, adoption numbers like "seven of the ten largest AI customers" and OpenAI's six-gigawatt commitment are genuinely significant, but they're also multi-year commitments still largely ahead of us, the first 1-gigawatt MI450 deployment for OpenAI doesn't begin until the second half of 2026. How ROCm performs under that scale of real, sustained production load, versus how it performs in benchmarks and pilot deployments today, is something this series will be able to speak to with more confidence a year or two from now than it can today.

Final Words

ROCm's ten-year arc is a reasonable case study in what it actually takes to build an alternative to an entrenched, deeply-moated software platform: not a single breakthrough release, but a decade of compounding investment, HIP's CUDA-compatible programming model, upstream LLVM contributions, official PyTorch and vLLM support, an automated build system mature enough to support a six-week cadence, and now an agentic-AI layer aimed at where AMD believes AI development is heading next. Whether that's enough to meaningfully dent CUDA's dominance depends less on any single ROCm release than on whether AMD's hyperscaler and OpenAI commitments translate into the kind of sustained, high-volume production use that closes the remaining gaps faster than marketing claims currently suggest.

I might have missed some recent ROCm developments. Please comment to mention any that I've missed; I'll add on to the growing list in this article.

References

Dhupinder Malhotra
Next

Co-Packaged Optics: Moving Light Closer to Silicon