Can I run Muse-Glimmer-30B-GGUF on an NVIDIA H200?
Yes
Its BF16 file (55.7 GB) fits in the 127 GB an NVIDIA H200 can use, with room for an unknown amount of context.
Full analysis of Muse-Glimmer-30B-GGUF
Everything that runs on NVIDIA H200
Get the BF16 file on Hugging Face
llama-server -hf unsloth/Muse-Glimmer-30B-GGUF:BF16Files and context on NVIDIA H200
Each file of Muse-Glimmer-30B-GGUF (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.
| File | Size | On NVIDIA H200 | Context (16-bit) | Context (8-bit) |
|---|---|---|---|---|
| BF16 | 55.7 GB | Fits | not known | not known |
| UD-Q8_K_XL | 32.3 GB | Fits | not known | not known |
| Q8_0 | 29.6 GB | Fits | not known | not known |
| UD-Q6_K_XL | 26.3 GB | Fits | not known | not known |
| UD-Q5_K_XL | 21.8 GB | Fits | not known | not known |
| UD-Q5_K_L | 19.8 GB | Fits | not known | not known |
| UD-Q5_K_M | 19.2 GB | Fits | not known | not known |
| UD-Q4_K_XL | 15.9 GB | Fits | not known | not known |
| UD-IQ3_M | 14.1 GB | Fits | not known | not known |
| UD-Q3_K_XL | 13.4 GB | Fits | not known | not known |
| UD-IQ3_XXS | 13.1 GB | Fits | not known | not known |
| UD-Q2_K_XL | 12.4 GB | Fits | not known | not known |
| UD-IQ2_M | 12.3 GB | Fits | not known | not known |
| UD-IQ2_XS | 11.5 GB | Fits | not known | not known |
| UD-IQ2_XXS | 10.7 GB | Fits | not known | not known |