LLM//BOTTLENECK
Sized at 8,192 tokens of context with the whole model in device memory, across 18 common devices. Speeds are estimates, not benchmarks.
Find the NVIDIA GeForce RTX 5070, the smallest card listed here that holds it: Amazon ↗ · eBay (new and used) ↗Store links may pay us a commission. They never decide which card is suggested — the memory arithmetic does.
| Format | Size | Range | Basis |
|---|---|---|---|
| FP16 | 3.63 GB | 3.42 GB – 3.64 GB | reconstructed size |
| Q8_0 | 1.93 GB | 1.82 GB – 1.94 GB | reconstructed size |
| Q6_K | 1.49 GB | 1.40 GB – 1.50 GB | reconstructed size |
| Q5_K_M | 1.30 GB | 1.21 GB – 1.31 GB | reconstructed size |
| Q5_0 | 1.26 GB | 1.18 GB – 1.28 GB | reconstructed size |
| Q4_K_M | 1.11 GB | 1.03 GB – 1.14 GB | reconstructed size |
| Q4_0 | 1.05 GB | 0.96 GB – 1.07 GB | reconstructed size |
| Q3_K_M | 0.92 GB | 0.83 GB – 0.98 GB | reconstructed size |
| Q2_K | 0.70 GB | 0.61 GB – 0.75 GB | reconstructed size |
A file size is not the memory a run needs: the KV cache and the runtime reserve come on top, and the calculator adds both.
| Device | Memory | Verdict | Decode | Calculator |
|---|---|---|---|---|
| NVIDIA GeForce RTX 5070 | 3.52 GB of 12 GB | fits | ~171 tok/s | Open → |
| AMD Radeon™ RX 9070 XT | 3.52 GB of 16 GB | fits | ~187 tok/s | Open → |
| NVIDIA GeForce RTX 4080 | 3.52 GB of 16 GB | fits | ~183 tok/s | Open → |
| NVIDIA GeForce RTX 5080 | 3.52 GB of 16 GB | fits | ~245 tok/s | Open → |
| NVIDIA GeForce RTX 5070 Ti | 3.52 GB of 16 GB | fits | ~228 tok/s | Open → |
| NVIDIA GeForce RTX 5060 Ti 16GB | 3.52 GB of 16 GB | fits | ~114 tok/s | Open → |
| NVIDIA GeForce RTX 4090 | 3.52 GB of 24 GB | fits | ~257 tok/s | Open → |
| AMD Radeon™ RX 7900 XTX | 3.52 GB of 24 GB | fits | ~280 tok/s | Open → |
| NVIDIA GeForce RTX 3090 | 3.52 GB of 24 GB | fits | ~239 tok/s | Open → |
| NVIDIA GeForce RTX 5090 | 3.52 GB of 32 GB | fits | ~457 tok/s | Open → |
| NVIDIA RTX 6000 Ada Generation | 3.52 GB of 48 GB | fits | ~245 tok/s | Open → |
| Apple M4 Pro | 3.52 GB of 64 GB | fits | ~62 tok/s | Open → |
| Apple M5 Pro | 3.52 GB of 64 GB | fits | ~67 tok/s | Open → |
| AMD Ryzen AI Max+ 395 with Radeon 8060S | 3.52 GB of 128 GB | fits | ~75 tok/s | Open → |
| Apple M3 Max | 3.52 GB of 128 GB | fits | ~78 tok/s | Open → |
| Apple M4 Max | 3.52 GB of 128 GB | fits | ~92 tok/s | Open → |
| NVIDIA DGX Spark | 3.52 GB of 128 GB | fits | ~70 tok/s | Open → |
| Apple M2 Ultra | 3.52 GB of 192 GB | fits | ~108 tok/s | Open → |
Decode is a calibrated estimate from the published calibration; capacity is the manufacturer's published ceiling, not guaranteed free memory. Fitting in memory is not the same as loading: whether the runtime and version you have supports this architecture and format on that machine has not been tested here. Offloaded rows assume there is enough system RAM for the overflow — the calculator checks that against the RAM you declare. Each “Open” link keeps this model, format and context.
Every rentable machine that holds the whole model, from Vast.ai and RunPod, with this site's speed estimate for each and the provider's own price, read live. Cheapest first; sort by cost per token or speed instead, or switch the format.
| Machine | Speed | Per hour | Per M tokens | Where | Rent |
|---|---|---|---|---|---|
| 1× RTX 306012 GB | ~91 tok/sFaster than you read | $0.036 | $0.11 | Vast.aiMarketplace · 99.8% reliable | Rent on Vast.ai |
| 1× RTX 308010 GB | ~193 tok/sFaster than you read | $0.082 | $0.12 | Vast.aiMarketplace · 98.3% reliable | Rent on Vast.ai |
| 1× RTX 30708 GB | ~114 tok/sFaster than you read | $0.082 | $0.20 | Vast.aiMarketplace · 99.1% reliableRunPod $0.13/h | Rent on Vast.ai |
| 1× RTX 4060 Ti 16GB16 GB | ~73 tok/sFaster than you read | $0.090 | $0.34 | Vast.aiMarketplace · 99.4% reliable | Rent on Vast.ai |
| 1× RTX 407012 GB | ~128 tok/sFaster than you read | $0.096 | $0.21 | Vast.aiMarketplace · 99.8% reliable | Rent on Vast.ai |
| 1× RTX 309024 GB | ~238 tok/sFaster than you read | $0.12 | $0.14 | Vast.aiMarketplace · 99.5% reliableRunPod $0.50/h | Rent on Vast.ai |
| 1× RTX 5060 Ti 16GB16 GB | ~114 tok/sFaster than you read | $0.12 | $0.30 | Vast.aiMarketplace · 99.1% reliable | Rent on Vast.ai |
| 1× RTX 3080 Ti12 GB | ~232 tok/sFaster than you read | $0.14 | $0.16 | Vast.aiMarketplace · 99.9% reliable | Rent on Vast.ai |
No Q4_K_M file of SmolLM2-1.7B-Instruct is catalogued here, so there is no exact command to vouch for. Start the machine from the llama.cpp server image ghcr.io/ggml-org/llama.cpp:server-cuda and point -hf at a Q4_K_M build from the publisher or a quantizer you trust — search Hugging Face for one. Check its file size against the memory figure on this page before you rent.
Every machine holds the whole model at Q4_K_M and 8,192 tokens of context, no offload. Speeds are this site’s single-stream decode estimates; prices are what each provider’s own API quoted, on-demand, for the whole machine. Vast hosts below 98% measured reliability are left out.
Referral links Vast.ai, RunPod and Novita pay us a share of what you spend if you sign up through these buttons. It costs you nothing, and it never decides an order or a recommendation: both are computed from the live price and the speed, and options that pay us nothing are listed and recommended on the same terms. How we rank
Buying a card instead, or paying by the token? Compare a year of SmolLM2-1.7B-Instruct three ways →
The numbers every figure above is computed from, with the file they came from.
Retrieved 2026-09-13 at pinned commit 31b70e2e869a; parameter count from the safetensors index at the same revision.
Every attention head caches its own keys and values, so the KV cache grows at the full multi-head rate with context.
curl -s https://llmbottleneck.com/v1/analyze \
-H "Authorization: Bearer $LLMB_KEY" -H "content-type: application/json" \
-d '{"model":"huggingfacetb-smollm2-1-7b-instruct","quantization":"Q4_K_M","context":8192,"hardware":"nvidia-geforce-rtx-4090"}'Same engine, same evidence, every field sourced. Free key in one step →
<script src="https://llmbottleneck.com/widget.js" data-model="huggingfacetb-smollm2-1-7b-instruct" data-quantization="Q4_K_M" data-hardware="nvidia-geforce-rtx-4090" data-context="8192"></script>
No key needed. Unbranded, with your own buy button, on Pro and Business →
LLM Bottleneck. “SmolLM2-1.7B-Instruct VRAM and hardware requirements.” Architecture from HuggingFaceTB/SmolLM2-1.7B-Instruct at revision 31b70e2e869a, retrieved 2026-09-13. https://llmbottleneck.com/models/huggingfacetb-smollm2-1-7b-instruct
Every figure above is either the published value or a reconstruction whose measured error is on the accuracy page.