The AI Accelerator Landscape in 2026: Every Chip Family Compared

The idea in one minute#

Every AI chip multiplies matrices. They differ in two decisions:

Where the model’s weights live. In large off-chip DRAM (HBM or LPDDR), which holds big models but must be streamed in for every token; or in on-chip SRAM, which is tens of times faster but holds far less.
How programmable the chip is. A general GPU runs any model and any new operator; a specialised chip trades that flexibility for efficiency on one class of workload.

Place a chip on those two axes and you know most of what it is good at. Then check the third factor, which decides more purchases than either: the software stack. This lesson maps the field as it stood on 16 October 2026.

A picture#

Positions are qualitative. Bottom-left is the safe default; top-right is fastest per user and holds the least.

What you program each chip with. Three dots: used in production by many teams. One dot: narrow or vendor-only.

How it really works#

The families#

Family	Who	The bet	The cost
Merchant GPUs	NVIDIA; AMD Instinct; Intel; Huawei Ascend (an NPU, used like a GPU)	One flexible part for training and serving any model	Never the most efficient for any one workload
Hyperscaler ASICs	Google TPU; AWS Trainium and Inferentia; Microsoft Maia; Meta MTIA	Own the design, own the margin, tune for known workloads at huge volume	Mostly available only in that company’s cloud; software tied to it
SRAM-first specialists	Cerebras (wafer-scale), Groq (now also NVIDIA’s LPX), SambaNova, d-Matrix	Keep weights on chip and remove the memory wall for decode	Little memory per chip, so many chips per model; narrower model support
Fixed-architecture ASICs	Etched and similar transformer-only designs	Hard-wire one architecture for maximum efficiency	Useless if the architecture changes
Client and edge	Apple M-series, phone and PC NPUs (Qualcomm, Intel, AMD), Jetson	Low power, large unified memory, private	Low bandwidth and compute

The field in October 2026#

AMD ships MI355X (288 GB HBM3E) and, from the second half of 2026, its first rack-scale system, Helios, with 72 MI455X GPUs of 432 GB HBM4 each (lesson 02).
Google has Ironwood (TPU7x) generally available and announced its eighth generation in April 2026 as two chips: TPU 8t for training and TPU 8i for inference (lesson 03).
AWS has Trainium3 generally available since December 2025 and has previewed Trainium4; Microsoft launched Maia 200 for inference in January 2026, and a successor is reported; Meta showed its MTIA roadmap at Hot Chips 2026 (lesson 04).
NVIDIA licensed Groq’s technology in December 2025 and now sells the Groq 3 LPX rack beside its GPUs; Cerebras laid out CS-5 (2027) and a stacked-DRAM CS-6 at Hot Chips 2026 (lesson 05).
Intel has branded its Xe3P inference card, codenamed Crescent Island, as the GPU D-series: 160 GB of LPDDR5X (up to 480 GB in partner variants) on a 350 W air-cooled card. It bets on capacity per dollar, not bandwidth. Gaudi 3 (128 GB HBM2e) remains its current data-center accelerator.
Huawei began mass production of the Ascend 950PR in 2026 with its own HBM, and plans the 950DT (144 GB, 4 TB/s) and Atlas 950 SuperPoDs of 8,192 chips for the fourth quarter. Export controls make it the main accelerator inside China.
Qualcomm has announced AI200 and AI250 inference cards with large LPDDR memory for 2026-27; Apple’s M3 Ultra with 512 GB of unified memory runs very large models on a desk.

Why software decides#

The second figure is the honest summary. NVIDIA is mature at every layer; AMD is close for mainstream PyTorch and serving work but thinner for custom kernels; Google’s stack is excellent if you adopt JAX and XLA; AWS’s Neuron and Intel’s Gaudi stacks work for supported models with gaps elsewhere; Groq and Cerebras are mostly consumed as hosted APIs.

Questions to ask before any switch:

Does my exact model, at my precision, run on it today in my serving engine?
Are the kernels I depend on (attention variant, MoE routing, quantization format) there and fast?
How do I profile and debug it when something is 3x slower than expected?
Can I get capacity, and from more than one supplier?
How many engineer-weeks will the port take, and who maintains it?

For a large, stable workload, the answers can justify the switch; that is why the biggest operators run several kinds of chip. For a small team moving fast, software maturity usually outweighs a better price per FLOP.

Code#

A two-stage evaluation: hard gates first (does it run, can I buy it), then cost per unit of work, with the porting effort amortised over the deployment.

// evaluate.go — gates first, then cost per million tokens including the port.
package main

import "fmt"

type Option struct {
	Name           string
	RunsMyModel    bool    // gate: exact model, precision and serving engine work today
	Available      bool    // gate: capacity can be reserved this quarter
	TokensPerSec   float64 // measured throughput per accelerator on YOUR workload
	DollarsPerHour float64 // per accelerator
	PortWeeks      float64 // engineer-weeks to reach that throughput
}

func main() {
	const (
		accelerators = 64
		months       = 24.0
		weekCost     = 6000.0 // fully loaded engineer-week, dollars
	)
	opts := []Option{
		{"Incumbent GPU", true, true, 2400, 6.0, 0},
		{"Alternative GPU", true, true, 2600, 4.5, 6},
		{"Cloud ASIC", true, true, 2000, 3.0, 12},
		{"Specialist inference", false, true, 9000, 20.0, 20},
		{"Next-gen GPU", true, false, 6000, 9.0, 2},
	}
	hours := months * 730
	fmt.Println("option                 gates          $ per M tokens   of which port")
	for _, o := range opts {
		if !o.RunsMyModel || !o.Available {
			why := "does not run my model"
			if o.RunsMyModel {
				why = "no capacity"
			}
			fmt.Printf("%-22s %-14s\n", o.Name, "FAIL: "+why)
			continue
		}
		compute := o.DollarsPerHour * accelerators * hours
		port := o.PortWeeks * weekCost
		tokens := o.TokensPerSec * accelerators * hours * 3600
		total := (compute + port) / tokens * 1e6
		fmt.Printf("%-22s %-14s %16.3f %14.4f\n", o.Name, "pass", total, port/tokens*1e6)
	}
}

All numbers are placeholders; measure your own. Two lessons survive any numbers: a gate failure ends the comparison however good the price, and at fleet scale the porting cost is small next to compute, which is why large operators port and small ones often should not.

Remember this#

Two axes explain the field: where the weights live (DRAM vs SRAM) and how programmable the chip is.
Families: merchant GPUs, hyperscaler ASICs, SRAM-first specialists, fixed-architecture ASICs, client and edge.
Software maturity is usually the deciding factor; gate on “runs my exact model today”.
Porting cost matters for small fleets and almost disappears for large ones.

Try it#

Run evaluate.go. At what fleet size does the Cloud ASIC’s port cost exceed 5% of its total?
Pick one non-NVIDIA chip and find out whether your serving engine supports your model on it. Record how long finding out took.
Place two chips not on the map (say, d-Matrix Corsair and Tenstorrent Blackhole) on its axes. Justify each position.

Check yourself#

What are the two axes of the map, and what does each predict?
Name one chip in each family and its bet.
Why can a chip with better price per FLOP lose a real evaluation?
Why do the largest operators run several kinds of accelerator while small teams usually run one?

The Accelerator Landscape