AMD Instinct

The idea in one minute#

AMD builds its data-center GPUs from chiplets stacked in 3D: small compute dies (XCDs) bonded on top of larger I/O dies that hold the cache and memory controllers, with HBM stacks alongside. That lets it mix process nodes and put more HBM on a package than NVIDIA in the same generation. Memory per GPU has been AMD’s lever for three generations: 192 GB on MI300X, 288 GB on MI355X, 432 GB on MI455X.

In 2026 AMD also became a rack vendor. Helios puts 72 MI455X GPUs in one scale-up domain over UALink, with Ultra Ethernet for scale-out: AMD’s answer to NVL72, built on open standards. Facts checked on 16 October 2026.

A picture#

How it really works#

Chiplets in three dimensions#

Where NVIDIA builds two huge compute dies, AMD builds many small ones:

• XCD (accelerator complex die): the compute. On MI355X, 8 XCDs on TSMC N3, each with 36 compute units (CUs) of which 32 are enabled: 256 CUs. A CU is AMD’s SM: it has matrix cores (AMD’s tensor cores), vector units and its own local memory.
• I/O die: the base layer, on the older and cheaper N6. Memory controllers, the 256 MB Infinity Cache and the Infinity Fabric that joins everything. MI300X used four; MI355X uses two larger ones, which cuts the hops between XCDs.
• 3D hybrid bonding (TSMC SoIC) joins XCDs to I/O dies with copper-to-copper bonds at a very fine pitch; the I/O dies and HBM then sit on a silicon interposer (CoWoS-S). Small dies yield well, and each layer uses the cheapest process that suits it.

The cost is locality. Each XCD has its own L2 cache, and by default the hardware deals workgroups to XCDs in round-robin. Kernels that share data between neighbouring workgroups can lose cache hits that a single big die would keep, which is one reason AMD-tuned kernels (its AITER library, tuned Triton) matter. MI300-series GPUs can also be partitioned into separate logical GPUs per XCD (CPX mode), much like NVIDIA’s MIG.

The generations#

AMD quotes MI455X bandwidth as 19.6 TB/s in early material and up to 23.3 TB/s on its current product page; treat both as claims. The MI400 family also includes the MI430X for scientific computing (strong FP64) and the MI440X for eight-GPU servers; MI500 is announced for 2027.

Helios#

• 72 MI455X GPUs, 31 TB of HBM4, about 2.9 EF of FP4 (AMD).
• Scale-up over UALink, an open standard backed by AMD, Intel, Google, Meta, Microsoft and others, carried over Ethernet-style switch trays: 260 TB/s across the rack, the same figure as Vera Rubin NVL72.
• Scale-out over Ultra Ethernet with AMD Pensando “Vulcano” 800G NICs.
• EPYC “Venice” CPUs, and a double-wide rack built to Meta’s Open Rack Wide design.
• Announced customers include OpenAI (a multi-gigawatt agreement whose first deployment uses MI450-series GPUs in the second half of 2026) and Oracle.

Software: ROCm#

ROCm 7 is a credible stack for mainstream work: PyTorch and JAX run upstream, vLLM and SGLang support Instinct GPUs as first-class targets, Triton compiles for them, and HIP lets most CUDA C++ be translated mechanically. The gaps are in the long tail: new attention variants, custom fused kernels and profiling tools usually arrive on CUDA first. For a team serving popular open models, AMD is a real option; for a team writing novel kernels every week, it costs more effort.

Where AMD wins#

Capacity per GPU decides how many GPUs one copy of a model needs. Fewer GPUs per replica means less communication, more replicas per rack and more KV cache per user. The code below counts them.

Code#

How many GPUs does one replica of a large model need on each part, once you reserve memory for the KV cache?

// replica.go — memory per GPU decides GPUs per model copy.
package main

import (
	"fmt"
	"math"
)

type GPU struct {
	Name         string
	MemGB, BWTBs float64
}

func main() {
	gpus := []GPU{
		{"H200", 141, 4.8},
		{"MI355X", 288, 8},
		{"B300", 288, 8},
		{"Rubin", 288, 22},
		{"MI455X", 432, 19.6},
	}
	models := []struct {
		Name    string
		WeightG float64 // GB of weights
		KVGB    float64 // KV cache budget for the target concurrency
	}{
		{"405B dense, FP8", 405, 300},
		{"1T MoE, FP8", 1000, 500},
		{"1T MoE, FP4", 560, 500},
	}
	usable := 0.9 // leave 10% for activations and the runtime
	for _, m := range models {
		fmt.Printf("\n%s + %.0f GB KV cache\n", m.Name, m.KVGB)
		for _, g := range gpus {
			n := math.Ceil((m.WeightG + m.KVGB) / (g.MemGB * usable))
			// round up to a power of two: parallel groups are usually 1, 2, 4, 8...
			p := math.Pow(2, math.Ceil(math.Log2(n)))
			fmt.Printf("  %-8s %3.0f GPUs (group of %2.0f)  aggregate bandwidth %5.0f TB/s\n",
				g.Name, n, p, p*g.BWTBs)
		}
	}
}

At 432 GB per GPU, the MI455X fits a 1T-parameter MoE at FP4 plus a large KV cache in a group of 4 where 288 GB parts need 8. Whether that wins depends on aggregate bandwidth and price: the group of 8 Rubins has more than twice the bandwidth of the 4 MI455X, so it decodes faster but splits the model more ways. Memory per GPU is a real lever, not the only one.

Remember this#

• AMD Instinct = small compute dies (XCDs) 3D-bonded onto I/O dies, beside HBM on an interposer.
• Its lever is memory per GPU: 192 → 288 → 432 GB.
• MI355X: 256 CUs, 288 GB HBM3E, 8 TB/s, FP4/FP6, 1,400 W.
• Helios: 72 MI455X, 31 TB HBM4, UALink scale-up, Ultra Ethernet scale-out, shipping from H2 2026.
• ROCm handles mainstream PyTorch and serving well; custom kernels cost more effort than on CUDA.

Try it#

1. Run replica.go. Add the MI325X (256 GB, 6 TB/s). Where does it beat the H200?
2. Change the usable fraction to 0.75 for a long-context workload. Which GPU’s group size changes first?
3. Find the current ROCm support matrix for vLLM. Is your favourite model’s attention variant supported on MI355X?

Check yourself#

1. What is an XCD, and what sits underneath it?
2. Why does chiplet compute make cache locality a kernel-tuning concern?
3. What two open standards does Helios use, and for which networks?
4. Why does more memory per GPU reduce communication?

Sources#

• AMD CDNA 4 architecture whitepaper
• AMD reveals MI355X core specs (Tom’s Hardware)
• AMD’s MI355X is a 1.4 kW liquid-cooled monster (The Register)
• AMD Instinct MI400 series
• AMD launches Instinct MI455X and Helios (Phoronix)
• UALink Consortium