LLM//BOTTLENECK
Sized at 2,048 tokens of context with the whole model in device memory, across 18 common devices. Speeds are estimates, not benchmarks.
Find the NVIDIA GeForce RTX 5070, the smallest card listed here that holds it: Amazon ↗ · eBay (new and used) ↗Store links may pay us a commission. They never decide which card is suggested — the memory arithmetic does.
| Format | Size | Range | Basis |
|---|---|---|---|
| FP16 | 1.01 GB | 0.91 GB – 1.02 GB | reconstructed size |
| Q8_0 | 0.54 GB | 0.48 GB – 0.54 GB | reconstructed size |
| Q6_K | 0.42 GB | 0.37 GB – 0.42 GB | reconstructed size |
| Q5_K_M | 0.37 GB | 0.32 GB – 0.37 GB | reconstructed size |
| Q5_0 | 0.36 GB | 0.31 GB – 0.36 GB | reconstructed size |
| Q4_K_M | 0.32 GB | 0.27 GB – 0.33 GB | reconstructed size |
| Q4_0 | 0.30 GB | 0.26 GB – 0.31 GB | reconstructed size |
| Q3_K_M | 0.26 GB | 0.22 GB – 0.28 GB | reconstructed size |
| Q2_K | 0.21 GB | 0.16 GB – 0.23 GB | reconstructed size |
A file size is not the memory a run needs: the KV cache and the runtime reserve come on top, and the calculator adds both.
| Device | Memory | Verdict | Decode | Calculator |
|---|---|---|---|---|
| NVIDIA GeForce RTX 5070 | 1.32 GB of 12 GB | fits | ~938 tok/s | Open → |
| AMD Radeon™ RX 9070 XT | 1.32 GB of 16 GB | fits | ~1024 tok/s | Open → |
| NVIDIA GeForce RTX 4080 | 1.32 GB of 16 GB | fits | ~1001 tok/s | Open → |
| NVIDIA GeForce RTX 5080 | 1.32 GB of 16 GB | fits | ~1340 tok/s | Open → |
| NVIDIA GeForce RTX 5070 Ti | 1.32 GB of 16 GB | fits | ~1251 tok/s | Open → |
| NVIDIA GeForce RTX 5060 Ti 16GB | 1.32 GB of 16 GB | fits | ~626 tok/s | Open → |
| NVIDIA GeForce RTX 4090 | 1.32 GB of 24 GB | fits | ~1407 tok/s | Open → |
| AMD Radeon™ RX 7900 XTX | 1.32 GB of 24 GB | fits | ~1536 tok/s | Open → |
| NVIDIA GeForce RTX 3090 | 1.32 GB of 24 GB | fits | ~1307 tok/s | Open → |
| NVIDIA GeForce RTX 5090 | 1.32 GB of 32 GB | fits | ~2502 tok/s | Open → |
| NVIDIA RTX 6000 Ada Generation | 1.32 GB of 48 GB | fits | ~1340 tok/s | Open → |
| Apple M4 Pro | 1.32 GB of 64 GB | fits | ~132 tok/s | Open → |
| Apple M5 Pro | 1.32 GB of 64 GB | fits | ~136 tok/s | Open → |
| AMD Ryzen AI Max+ 395 with Radeon 8060S | 1.32 GB of 128 GB | fits | ~410 tok/s | Open → |
| Apple M3 Max | 1.32 GB of 128 GB | fits | ~143 tok/s | Open → |
| Apple M4 Max | 1.32 GB of 128 GB | fits | ~151 tok/s | Open → |
| NVIDIA DGX Spark | 1.32 GB of 128 GB | fits | ~381 tok/s | Open → |
| Apple M2 Ultra | 1.32 GB of 192 GB | fits | ~158 tok/s | Open → |
Decode is a calibrated estimate from the published calibration; capacity is the manufacturer's published ceiling, not guaranteed free memory. Fitting in memory is not the same as loading: whether the runtime and version you have supports this architecture and format on that machine has not been tested here. Offloaded rows assume there is enough system RAM for the overflow — the calculator checks that against the RAM you declare. Each “Open” link keeps this model, format and context.
Every rentable machine that holds the whole model, from Vast.ai and RunPod, with this site's speed estimate for each and the provider's own price, read live. Cheapest first; sort by cost per token or speed instead, or switch the format.
| Machine | Speed | Per hour | Per M tokens | Where | Rent |
|---|---|---|---|---|---|
| 1× RTX 306012 GB | ~502 tok/sFaster than you read | $0.036 | $0.020 | Vast.aiMarketplace · 99.8% reliable | Rent on Vast.ai |
| 1× RTX 308010 GB | ~1061 tok/sFaster than you read | $0.082 | $0.022 | Vast.aiMarketplace · 98.3% reliable | Rent on Vast.ai |
| 1× RTX 30708 GB | ~625 tok/sFaster than you read | $0.082 | $0.037 | Vast.aiMarketplace · 99.1% reliableRunPod $0.13/h | Rent on Vast.ai |
| 1× RTX 4060 Ti 16GB16 GB | ~402 tok/sFaster than you read | $0.090 | $0.062 | Vast.aiMarketplace · 99.4% reliable | Rent on Vast.ai |
| 1× RTX 407012 GB | ~703 tok/sFaster than you read | $0.096 | $0.038 | Vast.aiMarketplace · 99.8% reliable | Rent on Vast.ai |
| 1× RTX 309024 GB | ~1306 tok/sFaster than you read | $0.12 | $0.026 | Vast.aiMarketplace · 99.5% reliableRunPod $0.50/h | Rent on Vast.ai |
| 1× RTX 5060 Ti 16GB16 GB | ~625 tok/sFaster than you read | $0.12 | $0.054 | Vast.aiMarketplace · 99.1% reliable | Rent on Vast.ai |
| 1× RTX 3080 Ti12 GB | ~1273 tok/sFaster than you read | $0.14 | $0.030 | Vast.aiMarketplace · 99.9% reliable | Rent on Vast.ai |
No Q4_K_M file of pythia-410m is catalogued here, so there is no exact command to vouch for. Start the machine from the llama.cpp server image ghcr.io/ggml-org/llama.cpp:server-cuda and point -hf at a Q4_K_M build from the publisher or a quantizer you trust — search Hugging Face for one. Check its file size against the memory figure on this page before you rent.
Every machine holds the whole model at Q4_K_M and 2,048 tokens of context, no offload. Speeds are this site’s single-stream decode estimates; prices are what each provider’s own API quoted, on-demand, for the whole machine. Vast hosts below 98% measured reliability are left out.
Referral links Vast.ai, RunPod and Novita pay us a share of what you spend if you sign up through these buttons. It costs you nothing, and it never decides an order or a recommendation: both are computed from the live price and the speed, and options that pay us nothing are listed and recommended on the same terms. How we rank
Buying a card instead, or paying by the token? Compare a year of pythia-410m three ways →
The numbers every figure above is computed from, with the file they came from.
Retrieved 2026-09-01 at pinned commit 9879c9b5f8be; parameter count from the safetensors index at the same revision.
Every attention head caches its own keys and values, so the KV cache grows at the full multi-head rate with context.
curl -s https://llmbottleneck.com/v1/analyze \
-H "Authorization: Bearer $LLMB_KEY" -H "content-type: application/json" \
-d '{"model":"eleutherai-pythia-410m","quantization":"Q4_K_M","context":2048,"hardware":"nvidia-geforce-rtx-4090"}'Same engine, same evidence, every field sourced. Free key in one step →
<script src="https://llmbottleneck.com/widget.js" data-model="eleutherai-pythia-410m" data-quantization="Q4_K_M" data-hardware="nvidia-geforce-rtx-4090" data-context="2048"></script>
No key needed. Unbranded, with your own buy button, on Pro and Business →
LLM Bottleneck. “pythia-410m VRAM and hardware requirements.” Architecture from EleutherAI/pythia-410m at revision 9879c9b5f8be, retrieved 2026-09-01. https://llmbottleneck.com/models/eleutherai-pythia-410m
Every figure above is either the published value or a reconstruction whose measured error is on the accuracy page.