Guides · September 19, 2026

Best AI models for a 24 GB GPU (RTX 4090, RTX 3090)

A 24 GB card like the RTX 4090, RTX 3090 or Radeon RX 7900 XTX is the sweet spot for running AI models at home. It holds most models up to about 30 billion parameters once they're compressed to 4 or 5 bits, with room left over for a long conversation.

The table below lists the most downloaded models that fit, with at least 16K tokens of context. "Best" here means popular and fits, not a quality ranking: the site measures what a model needs, not how well it answers. Each name opens a page with the full answer for an RTX 4090: which files fit, and how much context each leaves room for.

How to read the table

• File that fits is the largest file of that model that still leaves room for 16K tokens. For GGUF repos that's a specific quantization, like Q5_K_M; bigger files are closer to the original's quality. When a repo only has full-size weights, the table names a GGUF version of it made by someone else ("from …"), or shows a 4-bit estimate if the site hasn't analyzed one yet.
• Context is how much conversation fits in the memory left over. Every token of context takes memory (the "KV cache"), so a file that only just fits leaves little room.
• The page assumes about 90% of the card's 24 GB is usable. The rest goes to the driver and the program running the model.

Getting more out of 24 GB

• Pick a smaller file. Dropping from Q6_K to Q4_K_M frees several GB for context, usually at a small cost in quality. Files under 3 bits per weight (Q2_K, IQ2) fit bigger models but lose noticeably more.
• Use an 8-bit context cache. llama.cpp and LM Studio can store the context in 8-bit, which halves its memory. The hardware calculator shows what that makes possible.
• Mixture-of-experts models only use a fraction of their parameters for each token, so they run fast, but all of their experts still have to fit in memory.

To run a file, the simplest options are Ollama and LM Studio; each model's page has a copy-ready command.