LLM//BOTTLENECK
Sized at 2,048 tokens of context with the whole model in device memory, across 18 common devices. Speeds are estimates, not benchmarks.
Find the NVIDIA GeForce RTX 5070, the smallest card listed here that holds it: Amazon ↗ · eBay (new and used) ↗Store links may pay us a commission. They never decide which card is suggested — the memory arithmetic does.
| Format | Size | Range | Basis |
|---|---|---|---|
| FP16 | 5.56 GB | 3.89 GB – 5.61 GB | size range |
| Q8_0 | 2.96 GB | 2.07 GB – 3.23 GB | size range |
| Q6_K | 2.28 GB | 1.60 GB – 2.62 GB | size range |
| Q5_K_M | 1.99 GB | 1.34 GB – 2.62 GB | size range |
| Q5_0 | 1.94 GB | 1.34 GB – 2.62 GB | size range |
| Q4_K_M | 1.71 GB | 1.09 GB – 2.62 GB | size range |
| Q4_0 | 1.62 GB | 1.09 GB – 2.62 GB | size range |
| Q3_K_M | 1.39 GB | 0.84 GB – 2.62 GB | size range |
| Q2_K | 1.10 GB | 0.64 GB – 2.62 GB | size range |
A file size is not the memory a run needs: the KV cache and the runtime reserve come on top, and the calculator adds both.
| Device | Memory | Verdict | Decode | Calculator |
|---|---|---|---|---|
| NVIDIA GeForce RTX 5070 | 3.18 GB of 12 GB | fits | ~163 tok/s | Open → |
| AMD Radeon™ RX 9070 XT | 3.18 GB of 16 GB | fits | ~178 tok/s | Open → |
| NVIDIA GeForce RTX 4080 | 3.18 GB of 16 GB | fits | ~174 tok/s | Open → |
| NVIDIA GeForce RTX 5080 | 3.18 GB of 16 GB | fits | ~233 tok/s | Open → |
| NVIDIA GeForce RTX 5070 Ti | 3.18 GB of 16 GB | fits | ~217 tok/s | Open → |
| NVIDIA GeForce RTX 5060 Ti 16GB | 3.18 GB of 16 GB | fits | ~109 tok/s | Open → |
| NVIDIA GeForce RTX 4090 | 3.18 GB of 24 GB | fits | ~244 tok/s | Open → |
| AMD Radeon™ RX 7900 XTX | 3.18 GB of 24 GB | fits | ~267 tok/s | Open → |
| NVIDIA GeForce RTX 3090 | 3.18 GB of 24 GB | fits | ~227 tok/s | Open → |
| NVIDIA GeForce RTX 5090 | 3.18 GB of 32 GB | fits | ~434 tok/s | Open → |
| NVIDIA RTX 6000 Ada Generation | 3.18 GB of 48 GB | fits | ~233 tok/s | Open → |
| Apple M4 Pro | 3.18 GB of 64 GB | fits | ~60 tok/s | Open → |
| Apple M5 Pro | 3.18 GB of 64 GB | fits | ~65 tok/s | Open → |
| AMD Ryzen AI Max+ 395 with Radeon 8060S | 3.18 GB of 128 GB | fits | ~71 tok/s | Open → |
| Apple M3 Max | 3.18 GB of 128 GB | fits | ~76 tok/s | Open → |
| Apple M4 Max | 3.18 GB of 128 GB | fits | ~90 tok/s | Open → |
| NVIDIA DGX Spark | 3.18 GB of 128 GB | fits | ~66 tok/s | Open → |
| Apple M2 Ultra | 3.18 GB of 192 GB | fits | ~106 tok/s | Open → |
Decode is a calibrated estimate from the published calibration; capacity is the manufacturer's published ceiling, not guaranteed free memory. Fitting in memory is not the same as loading: whether the runtime and version you have supports this architecture and format on that machine has not been tested here. Offloaded rows assume there is enough system RAM for the overflow — the calculator checks that against the RAM you declare. Each “Open” link keeps this model, format and context.
Every rentable machine that holds the whole model, from Vast.ai and RunPod, with this site's speed estimate for each and the provider's own price, read live. Cheapest first; sort by cost per token or speed instead, or switch the format.
| Machine | Speed | Per hour | Per M tokens | Where | Rent |
|---|---|---|---|---|---|
| 1× RTX 306012 GB | ~87 tok/sFaster than you read | $0.036 | $0.11 | Vast.aiMarketplace · 99.8% reliable | Rent on Vast.ai |
| 1× RTX 308010 GB | ~184 tok/sFaster than you read | $0.082 | $0.12 | Vast.aiMarketplace · 98.3% reliable | Rent on Vast.ai |
| 1× RTX 30708 GB | ~108 tok/sFaster than you read | $0.082 | $0.21 | Vast.aiMarketplace · 99.1% reliableRunPod $0.13/h | Rent on Vast.ai |
| 1× RTX 4060 Ti 16GB16 GB | ~69 tok/sFaster than you read | $0.090 | $0.36 | Vast.aiMarketplace · 99.4% reliable | Rent on Vast.ai |
| 1× RTX 407012 GB | ~122 tok/sFaster than you read | $0.096 | $0.22 | Vast.aiMarketplace · 99.8% reliable | Rent on Vast.ai |
| 1× RTX 309024 GB | ~226 tok/sFaster than you read | $0.12 | $0.15 | Vast.aiMarketplace · 99.5% reliableRunPod $0.50/h | Rent on Vast.ai |
| 1× RTX 5060 Ti 16GB16 GB | ~108 tok/sFaster than you read | $0.12 | $0.31 | Vast.aiMarketplace · 99.1% reliable | Rent on Vast.ai |
| 1× RTX 3080 Ti12 GB | ~221 tok/sFaster than you read | $0.14 | $0.17 | Vast.aiMarketplace · 99.9% reliable | Rent on Vast.ai |
No Q4_K_M file of phi-2 is catalogued here, so there is no exact command to vouch for. Start the machine from the llama.cpp server image ghcr.io/ggml-org/llama.cpp:server-cuda and point -hf at a Q4_K_M build from the publisher or a quantizer you trust — search Hugging Face for one. Check its file size against the memory figure on this page before you rent.
Every machine holds the whole model at Q4_K_M and 2,048 tokens of context, no offload. Speeds are this site’s single-stream decode estimates; prices are what each provider’s own API quoted, on-demand, for the whole machine. Vast hosts below 98% measured reliability are left out.
Referral links Vast.ai, RunPod and Novita pay us a share of what you spend if you sign up through these buttons. It costs you nothing, and it never decides an order or a recommendation: both are computed from the live price and the speed, and options that pay us nothing are listed and recommended on the same terms. How we rank
Buying a card instead, or paying by the token? Compare a year of phi-2 three ways →
The numbers every figure above is computed from, with the file they came from.
Retrieved 2026-09-01 at pinned commit 810d367871c1; parameter count from the safetensors index at the same revision.
Every attention head caches its own keys and values, so the KV cache grows at the full multi-head rate with context.
curl -s https://llmbottleneck.com/v1/analyze \
-H "Authorization: Bearer $LLMB_KEY" -H "content-type: application/json" \
-d '{"model":"microsoft-phi-2","quantization":"Q4_K_M","context":2048,"hardware":"nvidia-geforce-rtx-4090"}'Same engine, same evidence, every field sourced. Free key in one step →
<script src="https://llmbottleneck.com/widget.js" data-model="microsoft-phi-2" data-quantization="Q4_K_M" data-hardware="nvidia-geforce-rtx-4090" data-context="2048"></script>
No key needed. Unbranded, with your own buy button, on Pro and Business →
LLM Bottleneck. “phi-2 VRAM and hardware requirements.” Architecture from microsoft/phi-2 at revision 810d367871c1, retrieved 2026-09-01. https://llmbottleneck.com/models/microsoft-phi-2
Every figure above is either the published value or a reconstruction whose measured error is on the accuracy page.