Can I run flint on a Mac with 36 GB?
Yes, with a GGUF version
The published weights are too big, but the UD-Q3_K_XL file from unsloth/flint-GGUF (18.5 GB) fits, with room for ≈ 186K tokens of context.
Full analysis of flint
Everything that runs on Mac with 36 GB
Get the UD-Q3_K_XL file on Hugging Face
ollama run hf.co/unsloth/flint-GGUF:UD-Q3_K_XLFiles and context on Mac with 36 GB
Each file of flint (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.
| File | Size | On Mac with 36 GB | Context (16-bit) | Context (8-bit) |
|---|---|---|---|---|
| As published | 79.6 GB | Doesn't fit | — | — |
| BF16 | 79.6 GB | Doesn't fit | — | — |
| Q8_0 | 42.3 GB | Doesn't fit | — | — |
| UD-Q6_K_XL | 36.3 GB | Doesn't fit | — | — |
| UD-Q5_K_XL | 29.9 GB | Doesn't fit | — | — |
| UD-Q4_K_XL | 25.1 GB | Doesn't fit | — | — |
| UD-Q3_K_XL | 18.5 GB | Fits | 186K | 256K |
| UD-IQ3_XXS | 14.5 GB | Fits | 256K | 256K |
| UD-Q2_K_XL | 13.6 GB | Fits | 256K | 256K |
| UD-IQ2_XXS | 11.9 GB | Fits | 256K | 256K |
| UD-IQ1_M | 11.4 GB | Fits | 256K | 256K |
flint on other hardware
- NVIDIA H100Yes, with a GGUF version
- RTX 5090Yes, with a GGUF version
- RTX 4090Yes, with a GGUF version
- RTX 3090Yes, with a GGUF version
- RTX 5060 Ti 16 GBYes, with a GGUF version
- RTX 3060 12 GBNo
- RTX 4060No
- Mac with 32 GBYes, with a GGUF version
- Mac with 36 GBYes, with a GGUF version
- Mac with 64 GBYes, with a GGUF version
- Mac with 128 GBYes, just