AMD / discrete

AMD Instinct MI350X

288 GB decides what fits. 8000 GB/s decides how fast it runs once it does.

Quick answer

The AMD Instinct MI350X fits 286 of 320 sized models entirely in its 288 GB; the largest widely used one is MiniMax-M3 (264.0 GB at Q4_K_M). Memory decides what fits; its 8000 GB/s of bandwidth decides how fast it answers.

Largest popular model that fitsMiniMax-M3427B params · Q4_K_M · needs 264.0 GB~386 tok/s Faster than you readOpen in the calculator →
Best fast pickMiniMax-Text-01456B params · Q4_K_M · needs 283.5 GB~245 tok/s Faster than you readOpen in the calculator →
Most downloaded that fitsGLM-5.3-Flash321B params · Q4_K_M · needs 198.3 GB~660 tok/s Faster than you readOpen in the calculator →

At 8,192 tokens of context with the whole model in device memory. Speeds are estimates from memory bandwidth, not benchmarks run on this card.

AMD Instinct MI350X · 288 GB · 8000 GB/s · Q4_K_M where published · 8,192 tokensSpecificationpublished dataDecode speedestimate

286 of 320 fit entirely

286 of the 320 models the engine can size fit entirely in device memory at Q4_K_M where it is published, otherwise the nearest published format, and 8,192 tokens. 1 run with some layers on system memory (32 GB assumed), and 33 do not run at all.

89%of the models the engine can size fit entirely

288GB of device memory

8000GB/s memory bandwidth

1run with system memory

33do not run at 8,192 tokens

Featured models · Q4_K_M at 8,192 tokens, including offload

Model	Needs	Verdict	Decode	Calculator
Qwen3.5-2B2.3B parameters	2.32 GB	fits	~5053 tok/s	Open →
NVIDIA-Nemotron-3-Nano-4B-BF164.0B parameters	3.46 GB	fits	~2341 tok/s	Open →
Qwen3-VL-8B-Instruct8.8B parameters	7.04 GB	fits	~1059 tok/s	Open →
Qwen3.5-9B9.7B parameters	7.05 GB	fits	~1282 tok/s	Open →
gemma-4-12B-it12B parameters	9.54 GB	fits	~763 tok/s	Open →
gpt-oss-20b21B parameters	13.8 GB	fits	~2492 tok/s	Open →
Qwen3.8-27B28B parameters	18.6 GB	fits	~421 tok/s	Open →
Qwen3.6-27B28B parameters	18.6 GB	fits	~421 tok/s	Open →
Kimi-Linear-48B-A3B-Instruct49B parameters	30.4 GB	fits	~3334 tok/s	Open →
Qwen3.8-Flash-Next180B parameters	111.6 GB	fits	~1732 tok/s	Open →
DeepSeek-V4-Flash-Vision-Exp305B parameters	188.1 GB	fits	~554 tok/s	Open →
GLM-5.3-Flash321B parameters	198.3 GB	fits	~660 tok/s	Open →

These examples are selected from prominent labs using the catalogue’s latest Hugging Face 30-day downloads and repository-creation freshness signal, with newer releases guaranteed a place. Offloaded rows assume 32 GB of system RAM, and a speed is only shown for a row that runs. Fitting in memory is not the same as loading: whether the runtime and version you have supports each architecture and format on this machine has not been tested here. Each “Open” link carries the same model, format, context and RAM into the calculator.

Fits entirely in AMD Instinct MI350X memory — 286 of 320 sized models

Every model that fitswhole model resident · offload off

The most demanding model that fits is MiniMax-Text-01 at Q4_K_M: 283.5 GB of the 288 GB, leaving 4.53 GB spare.

Fully resident at 8,192 tokens (or the model’s own maximum, where that is shorter), offload off, at 8000 GB/s. Each row is a run of the engine for this configuration; the rows start with current, prominent releases and “fits” is memory, not a tested runtime. Older or less prominent models remain available through this search and “Show all”.

Find a model in this list

Showing 40 of 286 models that fit.

Model	Format	Needs	Spare	Decode	Downloads / 30d	Calculator
GLM-5.3-FlashNEWzai-org · 321B params	Q4_K_M	198.3 GB	89.7 GB	~660 tok/s	5.4M	Open →
Qwen3.8-27BQwen · 28B params	Q4_K_M	18.6 GB	269.4 GB	~421 tok/s	6.9M	Open →
DeepSeek-V4-Flash-0731DeepSeek · 304B params	Q4_K_M	187.5 GB	100.5 GB	~554 tok/s	4.5M	Open →
Qwen3.8-Flash-NextNEWQwen · 180B params	Q4_K_M	111.6 GB	176.4 GB	~1732 tok/s	1.4M	Open →
gemma-4-26B-A4B-itGoogle · 26B params	Q4_K_M	16.8 GB	271.2 GB	~2341 tok/s	13M	Open →
DeepSeek-V4-Flash-Vision-ExpNEWDeepSeek · 305B params	Q4_K_M	188.1 GB	99.9 GB	~554 tok/s	915.3K	Open →
gemma-4-31B-itGoogle · 31B params	Q4_K_M	22.2 GB	265.8 GB	~303 tok/s	9.9M	Open →
Qwen3.5-9BQwen · 9.7B params	Q4_K_M	7.05 GB	281.0 GB	~1282 tok/s	9M	Open →
Qwen3.5-4BQwen · 4.7B params	Q4_K_M	3.98 GB	284.0 GB	~2308 tok/s	7.8M	Open →
Qwen3.6-35B-A3BQwen · 36B params	Q4_K_M	23.1 GB	264.9 GB	~3362 tok/s	3.3M	Open →
gemma-4-12B-itGoogle · 12B params	Q4_K_M	9.54 GB	278.5 GB	~763 tok/s	1.9M	Open →
Inkling-Smallthinkingmachines · 266B params	Q4_K_M	165.5 GB	122.5 GB	~875 tok/s	657.4K	Open →
Qwen3.6-27BQwen · 28B params	Q4_K_M	18.6 GB	269.4 GB	~421 tok/s	2.5M	Open →
Qwen3.5-2BQwen · 2.3B params	Q4_K_M	2.32 GB	285.7 GB	~5053 tok/s	4.9M	Open →
gemma-4-E4B-itGoogle · 8.0B params	Q4_K_M	5.86 GB	282.1 GB	~2060 tok/s	4.4M	Open →
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16NVIDIA · 32B params	Q4_K_M	20.3 GB	267.7 GB	~320 tok/s	530K	Open →
NVIDIA-Nemotron-3-Nano-4B-BF16NVIDIA · 4.0B params	Q4_K_M	3.46 GB	284.5 GB	~2341 tok/s	3.4M	Open →
gemma-4-E2B-itGoogle · 5.1B params	Q4_K_M	4.02 GB	284.0 GB	~5443 tok/s	3M	Open →
Qwen3.5-0.8BQwen · 873M params	Q4_K_M	1.46 GB	286.5 GB	~11070 tok/s	2.6M	Open →
DeepSeek-V4-FlashDeepSeek · 284B params	Q4_K_M	175.9 GB	112.1 GB	~554 tok/s	1.1M	Open →
Qwen3-VL-8B-InstructQwen · 8.8B params	Q4_K_M	7.04 GB	281.0 GB	~1059 tok/s	14.6M	Open →
MiniMax-M2.7MiniMaxAI · 229B params	Q4_K_M	141.2 GB	146.8 GB	~725 tok/s	1.1M	Open →
Qwen3.5-27BQwen · 28B params	Q4_K_M	18.6 GB	269.4 GB	~421 tok/s	1.9M	Open →
North-Micro-Vision-InstructCohereLabs · 2.5B params	Q4_K_M	2.99 GB	285.0 GB	~3323 tok/s	180.7K	Open →
Hy3Tencent · 299B params	Q4_K_M	181.2 GB	106.8 GB	~441 tok/s	304.9K	Open →
Qwen3.5-35B-A3BQwen · 36B params	Q4_K_M	23.1 GB	264.9 GB	~3362 tok/s	1.6M	Open →
NVIDIA-Nemotron-3-Super-120B-A12B-BF16NVIDIA · 124B params	Q4_K_M	76.9 GB	211.1 GB	~82 tok/s	1.2M	Open →
granite-4.2-8bIBM · 8.8B params	Q4_K_M	7.49 GB	280.5 GB	~965 tok/s	128.4K	Open →
GLM-4.7-Flashzai-org · 31B params	Q4_K_M	20.4 GB	267.6 GB	~318 tok/s	1.8M	Open →
granite-4.1-3bIBM · 3.4B params	Q4_K_M	3.72 GB	284.3 GB	~2250 tok/s	521.8K	Open →
LFM2.5-2.6BLiquidAI · 2.7B params	Q4_K_M	2.61 GB	285.4 GB	~3609 tok/s	108.1K	Open →
MiMo-V2.6-Flash-RLNEWXiaomiMiMo · 309B params	Q4_K_M	192.2 GB	95.8 GB	~643 tok/s	48.6K	Open →
MiniMax-M3MiniMaxAI · 427B params	Q4_K_M	264.0 GB	24.0 GB	~386 tok/s	177.2K	Open →
MiMo-V2.5XiaomiMiMo · 311B params	Q4_K_M	192.2 GB	95.8 GB	~643 tok/s	245.2K	Open →
Qwen3-VL-4B-InstructQwen · 4.4B params	Q4_K_M	4.72 GB	283.3 GB	~1683 tok/s	3.5M	Open →
granite-4.1-30bIBM · 29B params	Q4_K_M	20.4 GB	267.6 GB	~317 tok/s	301.5K	Open →
Qwen3.5-122B-A10BQwen · 125B params	Q4_K_M	77.9 GB	210.1 GB	~1231 tok/s	512.6K	Open →
Qwen3-VL-2B-InstructQwen · 2.1B params	Q4_K_M	3.05 GB	285.0 GB	~3051 tok/s	2.8M	Open →
granite-4.2-3bIBM · 3.7B params	Q4_K_M	3.72 GB	284.3 GB	~2250 tok/s	50.6K	Open →
LFM2.5-230MLiquidAI · 230M params	Q4_K_M	1.05 GB	286.9 GB	~25157 tok/s	88.6K	Open →

Sized at the format most people actually download, not at FP16. The catalogue ordered by downloads →

Under the hood

The specification behind every figure

What the manufacturer publishes for this device, and the pages it was read from.

Manufacturer specification

Memory scope	dedicated
Capacity	288 GB
Published options	288 GB
Bandwidth	8000 GB/s
Memory type	HBM3E
Bus width	8192-bit
FP32 peak	144.2 TFLOPS
Dense matrix peak	2300 TFLOPS (FP16 Matrix, dense)
Typical Board Power (TBP)	1000 W

Source ledger

AMD Instinct MI350X GPUs ↗

Retrieved 2026-09-13

Caveats

Per-OAM figures. AMD also publishes an eight-accelerator MI350X platform, whose totals are not used as single-device capacity.
The FP16 matrix figure is the dense one; AMD's separate "with Structured Sparsity" row (4.6 PFLOPs) is not used.

Published capacity is a hardware ceiling, not guaranteed free runtime memory. The calculator shows the runtime reserve separately rather than folding it into a single number.

Run the diagnostic on the AMD Instinct MI350X →