Can I run clef on a Radeon RX 7900 XTX?
Yes, with a GGUF version
The published weights are too big, but the Q5_K_M file from bartowski/Cloudflare_clef-GGUF (20.6 GB) fits, with room for ≈ 12K tokens of context.
Full analysis of clef
Everything that runs on Radeon RX 7900 XTX
Get the Q5_K_M file on Hugging Face
ollama run hf.co/bartowski/Cloudflare_clef-GGUF:Q5_K_MFiles and context on Radeon RX 7900 XTX
Each file of clef (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.
| File | Size | On Radeon RX 7900 XTX | Context (16-bit) | Context (8-bit) |
|---|---|---|---|---|
| As published | 54.7 GB | Doesn't fit | — | — |
| BF16 | 54.1 GB | Doesn't fit | — | — |
| Q8_0 | 28.8 GB | Doesn't fit | — | — |
| Q6_K_L | 24.0 GB | Doesn't fit | — | — |
| Q6_K | 23.3 GB | Doesn't fit | — | — |
| Q5_K_M | 20.6 GB | Tight | 12K | 24K |
| Q5_K_S | 19.6 GB | Tight | 28K | 56K |
| Q4_K_L | 18.6 GB | Tight | 42K | 85K |
| Q4_1 | 17.7 GB | Tight | 56K | 111K |
| Q4_K_M | 17.7 GB | Tight | 57K | 113K |
| Q4_K_S | 16.6 GB | Fits | 72K | 145K |
| IQ4_NL | 16.3 GB | Fits | 77K | 153K |
| Q4_0 | 16.2 GB | Fits | 78K | 157K |
| IQ4_XS | 15.6 GB | Fits | 88K | 175K |
| Q3_K_L | 15.2 GB | Fits | 94K | 188K |
| Q3_K_M | 14.5 GB | Fits | 104K | 208K |
| IQ3_M | 13.8 GB | Fits | 114K | 229K |
| Q3_K_S | 13.6 GB | Fits | 117K | 234K |
| Q2_K | 11.7 GB | Fits | 145K | 256K |
| BF16 | 54.1 GB | Doesn't fit | — | — |
| Q8_0 | 28.7 GB | Doesn't fit | — | — |
| Q4_K_M | 19.2 GB | Tight | 33K | 66K |
clef on other hardware
- NVIDIA H100Yes
- RTX 5090Yes, with a GGUF version
- RTX 4090Yes, with a GGUF version
- RTX 3090Yes, with a GGUF version
- RTX 5060 Ti 16 GBYes, with a GGUF version
- RTX 3060 12 GBNo
- RTX 4060No
- Radeon RX 7900 XTXYes, with a GGUF version
- Mac with 32 GBYes, with a GGUF version
- Mac with 64 GBYes, with a GGUF version
- Mac with 128 GBYes