AWS Trainium3, Microsoft Maia 200 and Meta MTIA Explained

The idea in one minute#

The companies that buy the most AI hardware also design their own. AWS has Trainium (and the older inference-only Inferentia), Microsoft has Maia, Meta has MTIA, and Google has the TPU. Each one is a bet that, at their volume, a chip designed for their own workloads and sold at cost to themselves beats paying a merchant vendor’s margin.

Trainium3, generally available since December 2025, is the most widely rented of these after the TPU: 144 GB of HBM3e per chip, 144 chips in one scale-up “UltraServer”, and a software stack, Neuron, that exposes the chip’s engines directly to kernel writers. Facts checked on 16 October 2026.

A picture#

Trainium3 and its UltraServer. The NeuronCore count is drawn as eight, the Trainium2 figure.

How it really works#

Inside a NeuronCore#

A NeuronCore is a small dataflow machine with four specialised engines that run in parallel, each fed from a large software-managed SRAM (SBUF):

Engine	Does	GPU analogue
Tensor	matrix multiply on a systolic array	tensor cores
Vector	elementwise and reduction work over vectors	CUDA cores on vectors
Scalar	per-element non-linear functions (exp, GELU)	special-function units
GpSimd	general-purpose C++ code for anything else	ordinary CUDA threads

Nothing is cached automatically: the compiler, or a kernel written in NKI (Neuron Kernel Interface, a Python-embedded kernel language), decides which tiles move from HBM into SBUF and when, and schedules the four engines to overlap. That is more work than CUDA for the programmer, and more predictable once done. Dedicated collective engines run all-reduce and all-to-all over NeuronLink without taking engine time.

The generations#

	Trainium2 (GA Dec 2024)	Trainium3 (GA Dec 2025)	Trainium4 (previewed)
Process	5 nm	TSMC 3 nm	not disclosed
FP8 dense	~1.3 PF	2.52 PF (MXFP8)	~3x Trn3 (target)
FP4	n/a	MXFP4 supported	~6x Trn3 (target)
Memory	96 GB HBM3, 2.9 TB/s	144 GB HBM3e, 4.9 TB/s	~4x Trn3 bandwidth (target)
Scale-up unit	64-chip Trn2 UltraServer	144-chip Trn3 UltraServer, 362 PF MXFP8, 20.7 TB HBM	to support NVLink Fusion

Trainium4’s support for NVIDIA’s NVLink Fusion is notable: AWS plans to put its own chips and NVIDIA GPUs in the same scale-up fabric. Beyond an UltraServer, chips scale out over AWS’s EFA network into “UltraClusters”; Project Rainier, built for Anthropic in 2025, was hundreds of thousands of Trainium2 chips. Inferentia2 remains available for small, cheap inference, but Trainium now handles both training and serving.

Microsoft Maia#

Maia 200, announced in January 2026, is an inference chip: TSMC 3 nm, 216 GB of HBM3e at about 7 TB/s, 272 MB of on-chip SRAM, native FP8 and FP4, about 10 PF of FP4 at 750 W according to Microsoft’s paper. The large SRAM is the interesting choice: like Google’s TPU 8i, it keeps more of the hot working set on chip for decode. Microsoft claims about 30% better performance per dollar than its previous hardware. A successor, Maia 300 (multi-chiplet, 2 nm, eight HBM4 stacks), was reported in October 2026 but not confirmed by Microsoft at the time of writing.

Meta MTIA#

Meta’s MTIA chips started with the workload that dominates Meta’s data centers: recommendation and ranking. MTIA v2 (2024) used LPDDR5 rather than HBM and ran at about 90 W: ranking models are dominated by embedding lookups and small matrices, where capacity and cost matter more than bandwidth. At Hot Chips 2026 Meta presented the next MTIA generations, aimed increasingly at generative AI; Meta also remains one of the largest buyers of NVIDIA and AMD GPUs.

Why build your own chip#

Reason	What it means
Margin	A merchant GPU’s price includes a large gross margin; an in-house chip is bought at cost
Supply	A second source of compute when GPUs are scarce, and leverage in negotiations
Fit	Size the memory, SRAM and interconnect for your own models and your own data centers
Power	Better performance per watt on known workloads, in power-limited buildings

The costs are real: a leading-edge chip program costs hundreds of millions of dollars per generation, needs a design partner (Broadcom, Marvell, Alchip, MediaTek and others do this work), takes two to three years, and its software must be good enough that internal teams choose it. It only pays at very large volume, as the code shows.

Code#

A break-even model for an in-house accelerator program against buying merchant GPUs.

// custom.go — at what fleet size does designing your own chip pay off?
package main

import "fmt"

func main() {
	const (
		programCost = 800e6 // design, masks, software team, per generation (dollars)
		years       = 3.0   // useful life of one generation
		gpuPrice    = 35000.0
		gpuWatts    = 1400.0
		asicPrice   = 12000.0 // at cost, but you need more of them for the same work
		asicWatts   = 900.0
		asicPerGPU  = 1.4 // ASICs needed to match one GPU on your workload
		dollarsKWh  = 0.08
		pue         = 1.25
	)
	energy := func(watts, n float64) float64 {
		return watts / 1000 * n * 24 * 365 * years * pue * dollarsKWh
	}
	fmt.Println("GPU-equivalents   merchant GPUs     in-house ASICs    winner")
	for _, n := range []float64{10e3, 50e3, 100e3, 250e3, 1e6} {
		buy := n*gpuPrice + energy(gpuWatts, n)
		a := n * asicPerGPU
		build := programCost + a*asicPrice + energy(asicWatts, a)
		w := "buy"
		if build < buy {
			w = "build"
		}
		fmt.Printf("%15.0f   $%12.2f B   $%12.2f B    %s\n", n, buy/1e9, build/1e9, w)
	}
}

With these placeholder numbers the program breaks even at about 43,000 GPU-equivalents and wins comfortably above that: that is why only the very largest buyers do this. Change asicPerGPU to reflect a weaker chip or immature software and the break-even moves out quickly.

Remember this#

AWS Trainium3: 2.52 PF MXFP8, 144 GB HBM3e at 4.9 TB/s, 144-chip UltraServers; Trainium4 will join NVLink Fusion fabrics.
A NeuronCore is four engines (tensor, vector, scalar, GpSimd) fed by software-managed SRAM; NKI is its kernel language.
Microsoft Maia 200 is an inference chip with 216 GB HBM3e and 272 MB SRAM; Meta’s MTIA started with ranking and recommendation.
Hyperscalers build chips for margin, supply, fit and power; it pays only at very large volume.

Try it#

Run custom.go. Find the fleet size at which build and buy cost the same. Then set asicPerGPU to 2.0 and find it again.
Read the NKI documentation and write, in pseudo-code, how you would tile a 4096 × 4096 matrix multiply into SBUF.
Compare Maia 200’s SRAM-to-HBM ratio with Ironwood’s and Rubin’s. What does a larger ratio suggest about the workload it was built for?

Check yourself#

Name the four engines of a NeuronCore and the job of each.
What does “software-managed SRAM” mean, and what does it cost the programmer?
Why did Meta’s first MTIA chips use LPDDR instead of HBM?
Give three reasons a hyperscaler builds its own accelerator, and the main reason a smaller company should not.

AWS Trainium and the Hyperscaler Chips