Can I run Qwythos-9B-Claude-Mythos-5-1M-GGUF on an RTX 4060?
Yes, just
Its Q5_K_M file (6.73 GB) fits in the 7.20 GB an RTX 4060 can use, with room for ≈ 13K tokens of context.
Full analysis of Qwythos-9B-Claude-Mythos-5-1M-GGUF
Everything that runs on RTX 4060
Get the Q5_K_M file on Hugging Face
ollama run hf.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q5_K_MFiles and context on RTX 4060
Each file of Qwythos-9B-Claude-Mythos-5-1M-GGUF (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.
| File | Size | On RTX 4060 | Context (16-bit) | Context (8-bit) |
|---|---|---|---|---|
| Qwythos-9B-Claude-Mythos-5-1M-MTP-BF16 | 18.4 GB | Doesn't fit | — | — |
| BF16 | 17.9 GB | Doesn't fit | — | — |
| Q8_0 | 9.79 GB | Doesn't fit | — | — |
| Qwythos-9B-Claude-Mythos-5-1M-Q8_0 | 9.53 GB | Doesn't fit | — | — |
| Q6_K | 7.62 GB | Doesn't fit | — | — |
| Qwythos-9B-Claude-Mythos-5-1M-Q6_K | 7.36 GB | Doesn't fit | — | — |
| Q5_K_M | 6.73 GB | Tight | 13K | 25K |
| Qwythos-9B-Claude-Mythos-5-1M-Q5_K_M | 6.47 GB | Tight | 20K | 41K |
| Q4_K_M | 5.89 GB | Tight | 38K | 75K |
| Qwythos-9B-Claude-Mythos-5-1M-Q4_K_M | 5.63 GB | Fits | 45K | 91K |