Can I run clef on a Mac with 24 GB?
Yes, with a GGUF version
The published weights are too big, but the IQ4_XS file from bartowski/Cloudflare_clef-GGUF (15.6 GB) fits, with room for ≈ 4K tokens of context.
ollama run hf.co/bartowski/Cloudflare_clef-GGUF:IQ4_XSFiles and context on Mac with 24 GB
Each file of clef (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.
| File | Size | On Mac with 24 GB | Context (16-bit) | Context (8-bit) |
|---|---|---|---|---|
| As published | 54.7 GB | Doesn't fit | — | — |
| BF16 | 54.1 GB | Doesn't fit | — | — |
| Q8_0 | 28.8 GB | Doesn't fit | — | — |
| Q6_K_L | 24.0 GB | Doesn't fit | — | — |
| Q6_K | 23.3 GB | Doesn't fit | — | — |
| Q5_K_M | 20.6 GB | Doesn't fit | — | — |
| Q5_K_S | 19.6 GB | Doesn't fit | — | — |
| Q4_K_L | 18.6 GB | Doesn't fit | — | — |
| Q4_1 | 17.7 GB | Doesn't fit | — | — |
| Q4_K_M | 17.7 GB | Doesn't fit | — | — |
| Q4_K_S | 16.6 GB | Doesn't fit | — | — |
| IQ4_NL | 16.3 GB | Doesn't fit | — | — |
| Q4_0 | 16.2 GB | Doesn't fit | — | — |
| IQ4_XS | 15.6 GB | Tight | 4K | 9K |
| Q3_K_L | 15.2 GB | Tight | 10K | 21K |
| Q3_K_M | 14.5 GB | Tight | 20K | 41K |
| IQ3_M | 13.8 GB | Tight | 31K | 62K |
| Q3_K_S | 13.6 GB | Tight | 34K | 67K |
| Q2_K | 11.7 GB | Fits | 62K | 123K |
| BF16 | 54.1 GB | Doesn't fit | — | — |
| Q8_0 | 28.7 GB | Doesn't fit | — | — |
| Q4_K_M | 19.2 GB | Doesn't fit | — | — |
clef on other hardware
- NVIDIA H100Yes
- RTX 5090Yes, with a GGUF version
- RTX 4090Yes, with a GGUF version
- RTX 3090Yes, with a GGUF version
- RTX 5060 Ti 16 GBYes, with a GGUF version
- RTX 3060 12 GBNo
- RTX 4060No
- Mac with 24 GBYes, with a GGUF version
- Mac with 32 GBYes, with a GGUF version
- Mac with 64 GBYes, with a GGUF version
- Mac with 128 GBYes