Model Anatomy See what's inside open-weight AI models

Can I run glm-4-9b-chat-IMat-GGUF on an RTX 5090?

Yes

Its BF16 file (18.8 GB) fits in the 28.8 GB an RTX 5090 can use, with room for ≈ 128K tokens of context.

Full analysis of glm-4-9b-chat-IMat-GGUF Everything that runs on RTX 5090 Get the BF16 file on Hugging Face
ollama run hf.co/legraphista/glm-4-9b-chat-IMat-GGUF:BF16

Files and context on RTX 5090

Each file of glm-4-9b-chat-IMat-GGUF (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.

FileSizeOn RTX 5090 Context (16-bit)Context (8-bit)
BF1618.8 GBFits128K128K
glm-4-9b-chat.FP1618.8 GBFits128K128K
Q8_09.99 GBFits128K128K
Q6_K8.26 GBFits128K128K
Q5_K7.14 GBFits128K128K
Q5_K_S6.69 GBFits128K128K
Q4_K6.25 GBFits128K128K
Q4_K_S5.75 GBFits128K128K
IQ4_NL5.46 GBFits128K128K
Q3_K_L5.28 GBFits128K128K
IQ4_XS5.25 GBFits128K128K
Q3_K5.06 GBFits128K128K
IQ3_M4.81 GBFits128K128K
IQ3_S4.59 GBFits128K128K
Q3_K_S4.59 GBFits128K128K
IQ3_XS4.43 GBFits128K128K
IQ3_XXS4.26 GBFits128K128K
Q2_K3.99 GBFits128K128K
Q2_K_S3.96 GBFits128K128K
IQ2_M3.93 GBFits128K128K
IQ2_S3.77 GBFits128K128K
IQ2_XS3.61 GBFits128K128K
IQ2_XXS3.43 GBFits128K128K
IQ1_M3.22 GBFits128K128K
IQ1_S3.10 GBFits128K128K

glm-4-9b-chat-IMat-GGUF on other hardware

Popular models that run on RTX 5090

Assumes 90% of the card's memory is usable and at least 4K tokens of context for one conversation, with the context cache in 16-bit unless noted. Fits leaves at least 20% of that memory for context; Tight leaves less.