Best AI models for an 8 GB GPU (RTX 4060, RTX 5060)
Eight gigabytes is the most common amount of video memory, and it's enough for a real local assistant: models of up to about 8 billion parameters fit once they're compressed to 4 or 5 bits, with room for a useful amount of context.
The table lists the most downloaded models that fit on an 8 GB card with at least 8K tokens of context. It's a popularity ranking of what fits, not a quality ranking. Each name opens the full answer for an RTX 4060.
| Model | Params | File that fits | Context |
|---|---|---|---|
| Qwen/Qwen3-0.6B | 752M | As published | full 40K |
| Qwen/Qwen3-VL-8B-Instruct | 8.77B | Q4_K_M | ≈ 14K |
| Qwen/Qwen3-8B | 8.19B | Q5_K_M | ≈ 9K |
| Qwen/Qwen2.5-7B-Instruct | 7.62B | Q6_K | ≈ 16K |
| Qwen/Qwen3.5-9B | 9.65B | UD-Q5_K_XL | ≈ 12K |
| Qwen/Qwen2.5-0.5B-Instruct | 494M | As published | full 32K |
| Qwen/Qwen2.5-1.5B-Instruct | 1.54B | As published | full 32K |
| Qwen/Qwen3-4B | 4.02B | Q8_0 | ≈ 19K |
| Qwen/Qwen3.5-4B | 4.66B | UD-Q8_K_XL | ≈ 36K |
| Qwen/Qwen2.5-VL-7B-Instruct | 8.29B | 4-bit (estimate) | ≈ 43K |
| ornith-ai/Ornith-1.5-9B-GGUF | 9.20B | Q5_K_M | ≈ 15K |
| Qwen/Qwen2.5-3B-Instruct | 3.09B | As published | ≈ 27K |
| Qwen/Qwen3.5-2B | 2.27B | As published | ≈ 209K |
| Qwen/Qwen3-4B-Instruct-2507 | 4.02B | 4-bit (estimate) | ≈ 33K |
| Qwen/Qwen3-1.7B | 2.03B | As published | ≈ 27K |
| Qwen/Qwen3-VL-4B-Instruct | 4.44B | Q8_0 | ≈ 19K |
| RadixArk/Kimi-K3-DSpark | 2.25B | As published | ≈ 129K |
| Qwen/Qwen2.5-7B-Instruct-AWQ | 7.62B | As published | ≈ 28K |
| ornith-ai/Ornith-1.0-9B-GGUF | 8.95B | Q5_K_M | ≈ 20K |
| Qwen/Qwen3-VL-2B-Instruct | 2.13B | As published | ≈ 25K |
298 models the site has analyzed fit on an RTX 4060 with at least 8K tokens of context; the 20 most downloaded are shown. See them all in the hardware calculator.
What fits in 8 GB
About 90% of the card is usable, so roughly 7.2 GB. A 4-bit file takes a little over half a gigabyte per billion parameters, so:
- Up to about 4B parameters fits with plenty of room, even at 8-bit.
- 7–9B parameters fits at 4 or 5 bits (Q4_K_M, Q5_K_M), with a few thousand to tens of thousands of tokens of context depending on the model.
- Bigger models only fit below 3 bits per weight, where quality drops noticeably.
Tips
- Context costs memory. If a model loads but runs out of memory in long chats, lower the context length or switch the context cache to 8-bit in your app's settings.
- Smaller is often better here. A 4B model at 8-bit can be a better experience than an 8B model squeezed to 3 bits.
- For a different card, try the hardware calculator: pick your GPU and it lists everything that fits.