Can I run Muse-Glimmer-30B-GGUF on an NVIDIA B300?
Yes
Its BF16 file (55.7 GB) fits in the 259 GB an NVIDIA B300 can use, with room for an unknown amount of context.
Full analysis of Muse-Glimmer-30B-GGUF
Everything that runs on NVIDIA B300
Get the BF16 file on Hugging Face
llama-server -hf unsloth/Muse-Glimmer-30B-GGUF:BF16Files and context on NVIDIA B300
Each file of Muse-Glimmer-30B-GGUF (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.
| File | Size | On NVIDIA B300 | Context (16-bit) | Context (8-bit) |
|---|---|---|---|---|
| BF16 | 55.7 GB | Fits | not known | not known |
| UD-Q8_K_XL | 32.3 GB | Fits | not known | not known |
| Q8_0 | 29.6 GB | Fits | not known | not known |
| UD-Q6_K_XL | 26.3 GB | Fits | not known | not known |
| UD-Q5_K_XL | 21.8 GB | Fits | not known | not known |
| UD-Q5_K_L | 19.8 GB | Fits | not known | not known |
| UD-Q5_K_M | 19.2 GB | Fits | not known | not known |
| UD-Q4_K_XL | 15.9 GB | Fits | not known | not known |
| UD-IQ3_M | 14.1 GB | Fits | not known | not known |
| UD-Q3_K_XL | 13.4 GB | Fits | not known | not known |
| UD-IQ3_XXS | 13.1 GB | Fits | not known | not known |
| UD-Q2_K_XL | 12.4 GB | Fits | not known | not known |
| UD-IQ2_M | 12.3 GB | Fits | not known | not known |
| UD-IQ2_XS | 11.5 GB | Fits | not known | not known |
| UD-IQ2_XXS | 10.7 GB | Fits | not known | not known |