Best local LLMs for 192 GB VRAM
Every catalogued model that fits entirely in 192 GB, with no layers moved to system RAM. Speeds are estimated on the AMD Instinct MI300X Accelerator.
- Largest popular model that fitsDeepSeek-V4-Flash-Vision-Exp305B params · Q4_K_M · needs 188.1 GB~367 tok/s Faster than you readOpen in the calculator →
- Most downloaded that fitsQwen3.8-27B28B params · Q4_K_M · needs 18.6 GB~279 tok/s Faster than you readOpen in the calculator →
“Best” is not a quality ranking. The picks are the largest widely downloaded model that fits, the largest that still answers at 30 tokens a second or more, and the most downloaded; the list puts current, widely downloaded releases first. Speeds are estimates from memory bandwidth, not benchmarks.
Is 192 GB of VRAM enough for a local LLM?
Yes: every model in the catalogue fits entirely at Q4_K_M, the format most people download. All 265 that fit answer at 30 tokens a second or more on the AMD Instinct MI300X Accelerator, which is faster than most people read. A longer conversation needs more memory than the 8,192 tokens counted here, and a smaller format than Q4_K_M needs less; the calculator sizes either.
| Model size | Fit in 192 GB | For example |
|---|---|---|
| Under 4B parameters | 66 of 66 | Qwen3.5-2B · needs 2.32 GB |
| 4B to 9B parameters | 58 of 58 | Qwen3.5-4B · needs 3.98 GB |
| 9B to 16B parameters | 21 of 21 | Qwen3.5-9B · needs 7.05 GB |
| 16B to 36B parameters | 66 of 66 | Qwen3.8-27B · needs 18.6 GB |
| 36B and larger parameters | 51 of 51 | DeepSeek-V4-Flash-0731 · needs 187.5 GB |
Models that fit in 192 GB
| Model | Parameters | Needs | Spare | Decode | Answer |
|---|---|---|---|---|---|
| Qwen3.8-27B Qwen · Q4_K_M | 28B | 18.6 GB | 173.4 GB | ~279 tok/s | Details |
| DeepSeek-V4-Flash-0731 DeepSeek · Q4_K_M | 304B | 187.5 GB | 4.48 GB | ~367 tok/s | Details |
| Qwen3.8-Flash-Next Qwen · Q4_K_M | 180B | 111.6 GB | 80.4 GB | ~1148 tok/s | Details |
| gemma-4-26B-A4B-it Google · Q4_K_M | 26B | 16.8 GB | 175.2 GB | ~1551 tok/s | Details |
| DeepSeek-V4-Flash-Vision-Exp DeepSeek · Q4_K_M | 305B | 188.1 GB | 3.87 GB | ~367 tok/s | Details |
| gemma-4-31B-it Google · Q4_K_M | 31B | 22.2 GB | 169.8 GB | ~201 tok/s | Details |
| Qwen3.5-9B Qwen · Q4_K_M | 9.7B | 7.05 GB | 185.0 GB | ~850 tok/s | Details |
| Qwen3.5-4B Qwen · Q4_K_M | 4.7B | 3.98 GB | 188.0 GB | ~1529 tok/s | Details |
| Qwen3.6-35B-A3B Qwen · Q4_K_M | 36B | 23.1 GB | 168.9 GB | ~2227 tok/s | Details |
| gemma-4-12B-it Google · Q4_K_M | 12B | 9.54 GB | 182.5 GB | ~505 tok/s | Details |
| Inkling-Small thinkingmachines · Q4_K_M | 266B | 165.5 GB | 26.5 GB | ~580 tok/s | Details |
| Qwen3.6-27B Qwen · Q4_K_M | 28B | 18.6 GB | 173.4 GB | ~279 tok/s | Details |
| Qwen3.5-2B Qwen · Q4_K_M | 2.3B | 2.32 GB | 189.7 GB | ~3348 tok/s | Details |
| gemma-4-E4B-it Google · Q4_K_M | 8.0B | 5.86 GB | 186.1 GB | ~1365 tok/s | Details |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 NVIDIA · Q4_K_M | 32B | 20.3 GB | 171.7 GB | ~212 tok/s | Details |
| NVIDIA-Nemotron-3-Nano-4B-BF16 NVIDIA · Q4_K_M | 4.0B | 3.46 GB | 188.5 GB | ~1551 tok/s | Details |
| gemma-4-E2B-it Google · Q4_K_M | 5.1B | 4.02 GB | 188.0 GB | ~3606 tok/s | Details |
| Qwen3.5-0.8B Qwen · Q4_K_M | 873M | 1.46 GB | 190.5 GB | ~7334 tok/s | Details |
| DeepSeek-V4-Flash DeepSeek · Q4_K_M | 284B | 175.9 GB | 16.1 GB | ~367 tok/s | Details |
| Qwen3-VL-8B-Instruct Qwen · Q4_K_M | 8.8B | 7.04 GB | 185.0 GB | ~702 tok/s | Details |
| MiniMax-M2.7 MiniMaxAI · Q4_K_M | 229B | 141.2 GB | 50.8 GB | ~481 tok/s | Details |
| Qwen3.5-27B Qwen · Q4_K_M | 28B | 18.6 GB | 173.4 GB | ~279 tok/s | Details |
| North-Micro-Vision-Instruct CohereLabs · Q4_K_M | 2.5B | 2.99 GB | 189.0 GB | ~2201 tok/s | Details |
| Hy3 Tencent · Q4_K_M | 299B | 181.2 GB | 10.8 GB | ~292 tok/s | Details |
| Qwen3.5-35B-A3B Qwen · Q4_K_M | 36B | 23.1 GB | 168.9 GB | ~2227 tok/s | Details |
| NVIDIA-Nemotron-3-Super-120B-A12B-BF16 NVIDIA · Q4_K_M | 124B | 76.9 GB | 115.1 GB | ~54 tok/s | Details |
| granite-4.2-8b IBM · Q4_K_M | 8.8B | 7.49 GB | 184.5 GB | ~639 tok/s | Details |
| GLM-4.7-Flash zai-org · Q4_K_M | 31B | 20.4 GB | 171.6 GB | ~210 tok/s | Details |
| granite-4.1-3b IBM · Q4_K_M | 3.4B | 3.72 GB | 188.3 GB | ~1491 tok/s | Details |
| LFM2.5-2.6B LiquidAI · Q4_K_M | 2.7B | 2.61 GB | 189.4 GB | ~2391 tok/s | Details |
| Qwen3-VL-4B-Instruct Qwen · Q4_K_M | 4.4B | 4.72 GB | 187.3 GB | ~1115 tok/s | Details |
| granite-4.1-30b IBM · Q4_K_M | 29B | 20.4 GB | 171.6 GB | ~210 tok/s | Details |
| Qwen3.5-122B-A10B Qwen · Q4_K_M | 125B | 77.9 GB | 114.1 GB | ~815 tok/s | Details |
| Qwen3-VL-2B-Instruct Qwen · Q4_K_M | 2.1B | 3.05 GB | 189.0 GB | ~2021 tok/s | Details |
| granite-4.2-3b IBM · Q4_K_M | 3.7B | 3.72 GB | 188.3 GB | ~1491 tok/s | Details |
| LFM2.5-230M LiquidAI · Q4_K_M | 230M | 1.05 GB | 190.9 GB | ~16667 tok/s | Details |
| Qwen3-0.6B Qwen · Q4_K_M | 752M | 2.22 GB | 189.8 GB | ~3101 tok/s | Details |
| Qwen3-Coder-Next Qwen · Q4_K_M | 80B | 50.0 GB | 142.0 GB | ~1481 tok/s | Details |
| gpt-oss-20b OpenAI · Q4_K_M | 21B | 13.8 GB | 178.2 GB | ~1651 tok/s | Details |
| MiniMax-M2.5 MiniMaxAI · Q4_K_M | 229B | 141.2 GB | 50.8 GB | ~481 tok/s | Details |
2 devices with 192 GB
The same models fit on every one of them. What changes is speed, which follows memory bandwidth: a card with twice the bandwidth decodes roughly twice as fast.
| Device | Bandwidth | Memory | Where to find one |
|---|---|---|---|
| AMD Instinct MI300X Accelerator speeds on this page | 5300 GB/s | 192 GB, dedicated | — |
| Apple M2 Ultra | 800 GB/s | 192 GB, unified | Amazon ↗ · eBay (new and used) ↗ |
Which card to buy, at every memory size →
Store links may pay us a commission. They never decide which models are listed — the memory arithmetic does.