AMD / discrete

AMD Instinct MI325X Accelerator

256 GB decides what fits. 6000 GB/s decides how fast it runs once it does.

Quick answer

The AMD Instinct MI325X Accelerator fits 283 of 320 sized models entirely in its 256 GB; the largest widely used one is Qwen3.5-397B-A17B (248.9 GB at Q4_K_M). Memory decides what fits; its 6000 GB/s of bandwidth decides how fast it answers.

Largest popular model that fitsQwen3.5-397B-A17B403B params · Q4_K_M · needs 248.9 GB~505 tok/s Faster than you readOpen in the calculator →
Most downloaded that fitsGLM-5.3-Flash321B params · Q4_K_M · needs 198.3 GB~495 tok/s Faster than you readOpen in the calculator →

At 8,192 tokens of context with the whole model in device memory. Speeds are estimates from memory bandwidth, not benchmarks run on this card.

AMD Instinct MI325X Accelerator · 256 GB · 6000 GB/s · Q4_K_M where published · 8,192 tokensSpecificationpublished dataDecode speedestimate

283 of 320 fit entirely

283 of the 320 models the engine can size fit entirely in device memory at Q4_K_M where it is published, otherwise the nearest published format, and 8,192 tokens. 3 run with some layers on system memory (32 GB assumed), and 34 do not run at all.

88%of the models the engine can size fit entirely

256GB of device memory

6000GB/s memory bandwidth

3run with system memory

34do not run at 8,192 tokens

Featured models · Q4_K_M at 8,192 tokens, including offload

Model	Needs	Verdict	Decode	Calculator
Qwen3.5-2B2.3B parameters	2.32 GB	fits	~3790 tok/s	Open →
NVIDIA-Nemotron-3-Nano-4B-BF164.0B parameters	3.46 GB	fits	~1756 tok/s	Open →
Qwen3-VL-8B-Instruct8.8B parameters	7.04 GB	fits	~794 tok/s	Open →
Qwen3.5-9B9.7B parameters	7.05 GB	fits	~962 tok/s	Open →
gemma-4-12B-it12B parameters	9.54 GB	fits	~572 tok/s	Open →
gpt-oss-20b21B parameters	13.8 GB	fits	~1869 tok/s	Open →
Qwen3.8-27B28B parameters	18.6 GB	fits	~316 tok/s	Open →
Qwen3.6-27B28B parameters	18.6 GB	fits	~316 tok/s	Open →
Kimi-Linear-48B-A3B-Instruct49B parameters	30.4 GB	fits	~2501 tok/s	Open →
Qwen3.8-Flash-Next180B parameters	111.6 GB	fits	~1299 tok/s	Open →
DeepSeek-V4-Flash-Vision-Exp305B parameters	188.1 GB	fits	~415 tok/s	Open →
GLM-5.3-Flash321B parameters	198.3 GB	fits	~495 tok/s	Open →

These examples are selected from prominent labs using the catalogue’s latest Hugging Face 30-day downloads and repository-creation freshness signal, with newer releases guaranteed a place. Offloaded rows assume 32 GB of system RAM, and a speed is only shown for a row that runs. Fitting in memory is not the same as loading: whether the runtime and version you have supports each architecture and format on this machine has not been tested here. Each “Open” link carries the same model, format, context and RAM into the calculator.

Fits entirely in AMD Instinct MI325X Accelerator memory — 283 of 320 sized models

Every model that fitswhole model resident · offload off

The most demanding model that fits is Llama-4-Maverick-17B-128E-Instruct at Q4_K_M: 248.9 GB of the 256 GB, leaving 7.07 GB spare.

Fully resident at 8,192 tokens (or the model’s own maximum, where that is shorter), offload off, at 6000 GB/s. Each row is a run of the engine for this configuration; the rows start with current, prominent releases and “fits” is memory, not a tested runtime. Older or less prominent models remain available through this search and “Show all”.

Find a model in this list

Showing 40 of 283 models that fit.

Model	Format	Needs	Spare	Decode	Downloads / 30d	Calculator
GLM-5.3-FlashNEWzai-org · 321B params	Q4_K_M	198.3 GB	57.7 GB	~495 tok/s	5.4M	Open →
Qwen3.8-27BQwen · 28B params	Q4_K_M	18.6 GB	237.4 GB	~316 tok/s	6.9M	Open →
DeepSeek-V4-Flash-0731DeepSeek · 304B params	Q4_K_M	187.5 GB	68.5 GB	~415 tok/s	4.5M	Open →
Qwen3.8-Flash-NextNEWQwen · 180B params	Q4_K_M	111.6 GB	144.4 GB	~1299 tok/s	1.4M	Open →
gemma-4-26B-A4B-itGoogle · 26B params	Q4_K_M	16.8 GB	239.2 GB	~1756 tok/s	13M	Open →
DeepSeek-V4-Flash-Vision-ExpNEWDeepSeek · 305B params	Q4_K_M	188.1 GB	67.9 GB	~415 tok/s	915.3K	Open →
gemma-4-31B-itGoogle · 31B params	Q4_K_M	22.2 GB	233.8 GB	~227 tok/s	9.9M	Open →
Qwen3.5-9BQwen · 9.7B params	Q4_K_M	7.05 GB	249.0 GB	~962 tok/s	9M	Open →
Qwen3.5-4BQwen · 4.7B params	Q4_K_M	3.98 GB	252.0 GB	~1731 tok/s	7.8M	Open →
Qwen3.6-35B-A3BQwen · 36B params	Q4_K_M	23.1 GB	232.9 GB	~2521 tok/s	3.3M	Open →
gemma-4-12B-itGoogle · 12B params	Q4_K_M	9.54 GB	246.5 GB	~572 tok/s	1.9M	Open →
Inkling-Smallthinkingmachines · 266B params	Q4_K_M	165.5 GB	90.5 GB	~656 tok/s	657.4K	Open →
Qwen3.6-27BQwen · 28B params	Q4_K_M	18.6 GB	237.4 GB	~316 tok/s	2.5M	Open →
Qwen3.5-2BQwen · 2.3B params	Q4_K_M	2.32 GB	253.7 GB	~3790 tok/s	4.9M	Open →
gemma-4-E4B-itGoogle · 8.0B params	Q4_K_M	5.86 GB	250.1 GB	~1545 tok/s	4.4M	Open →
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16NVIDIA · 32B params	Q4_K_M	20.3 GB	235.7 GB	~240 tok/s	530K	Open →
NVIDIA-Nemotron-3-Nano-4B-BF16NVIDIA · 4.0B params	Q4_K_M	3.46 GB	252.5 GB	~1756 tok/s	3.4M	Open →
gemma-4-E2B-itGoogle · 5.1B params	Q4_K_M	4.02 GB	252.0 GB	~4082 tok/s	3M	Open →
Qwen3.5-0.8BQwen · 873M params	Q4_K_M	1.46 GB	254.5 GB	~8302 tok/s	2.6M	Open →
DeepSeek-V4-FlashDeepSeek · 284B params	Q4_K_M	175.9 GB	80.1 GB	~415 tok/s	1.1M	Open →
Qwen3-VL-8B-InstructQwen · 8.8B params	Q4_K_M	7.04 GB	249.0 GB	~794 tok/s	14.6M	Open →
MiniMax-M2.7MiniMaxAI · 229B params	Q4_K_M	141.2 GB	114.8 GB	~544 tok/s	1.1M	Open →
Qwen3.5-27BQwen · 28B params	Q4_K_M	18.6 GB	237.4 GB	~316 tok/s	1.9M	Open →
North-Micro-Vision-InstructCohereLabs · 2.5B params	Q4_K_M	2.99 GB	253.0 GB	~2492 tok/s	180.7K	Open →
Hy3Tencent · 299B params	Q4_K_M	181.2 GB	74.8 GB	~331 tok/s	304.9K	Open →
Qwen3.5-35B-A3BQwen · 36B params	Q4_K_M	23.1 GB	232.9 GB	~2521 tok/s	1.6M	Open →
NVIDIA-Nemotron-3-Super-120B-A12B-BF16NVIDIA · 124B params	Q4_K_M	76.9 GB	179.1 GB	~61 tok/s	1.2M	Open →
granite-4.2-8bIBM · 8.8B params	Q4_K_M	7.49 GB	248.5 GB	~723 tok/s	128.4K	Open →
GLM-4.7-Flashzai-org · 31B params	Q4_K_M	20.4 GB	235.6 GB	~238 tok/s	1.8M	Open →
granite-4.1-3bIBM · 3.4B params	Q4_K_M	3.72 GB	252.3 GB	~1688 tok/s	521.8K	Open →
LFM2.5-2.6BLiquidAI · 2.7B params	Q4_K_M	2.61 GB	253.4 GB	~2707 tok/s	108.1K	Open →
MiMo-V2.6-Flash-RLNEWXiaomiMiMo · 309B params	Q4_K_M	192.2 GB	63.8 GB	~482 tok/s	48.6K	Open →
MiMo-V2.5XiaomiMiMo · 311B params	Q4_K_M	192.2 GB	63.8 GB	~482 tok/s	245.2K	Open →
Qwen3-VL-4B-InstructQwen · 4.4B params	Q4_K_M	4.72 GB	251.3 GB	~1262 tok/s	3.5M	Open →
granite-4.1-30bIBM · 29B params	Q4_K_M	20.4 GB	235.6 GB	~238 tok/s	301.5K	Open →
Qwen3.5-122B-A10BQwen · 125B params	Q4_K_M	77.9 GB	178.1 GB	~923 tok/s	512.6K	Open →
Qwen3-VL-2B-InstructQwen · 2.1B params	Q4_K_M	3.05 GB	253.0 GB	~2288 tok/s	2.8M	Open →
granite-4.2-3bIBM · 3.7B params	Q4_K_M	3.72 GB	252.3 GB	~1688 tok/s	50.6K	Open →
LFM2.5-230MLiquidAI · 230M params	Q4_K_M	1.05 GB	254.9 GB	~18868 tok/s	88.6K	Open →
Qwen3-0.6BQwen · 752M params	Q4_K_M	2.22 GB	253.8 GB	~3510 tok/s	29.7M	Open →

Sized at the format most people actually download, not at FP16. The catalogue ordered by downloads →

Under the hood

The specification behind every figure

What the manufacturer publishes for this device, and the pages it was read from.

Manufacturer specification

Memory scope	dedicated
Capacity	256 GB
Published options	256 GB
Bandwidth	6000 GB/s
Memory type	HBM3E
Bus width	8192-bit
FP32 peak	163.4 TFLOPS
Dense matrix peak	1300 TFLOPS (FP16, dense)
Typical Board Power (TBP), peak	1000 W

Source ledger

AMD Instinct MI325X Accelerators ↗

Retrieved 2026-08-30

AMD Instinct MI325X Platform Datasheet ↗

Retrieved 2026-08-30

AMD Instinct MI325X Platform ↗

Retrieved 2026-08-30

Caveats

Platform-level 2.048TB is not used as single-GPU capacity; the candidate retains AMD's exact per-OAM 256GB value.

Published capacity is a hardware ceiling, not guaranteed free runtime memory. The calculator shows the runtime reserve separately rather than folding it into a single number.

Run the diagnostic on the AMD Instinct MI325X Accelerator →