Guides · September 21, 2026 · Data updated October 27, 2026

The best local LLMs in 2026, by what fits your GPU

A local LLM is a language model you run on your own machine, from a file you downloaded, with nothing sent to anyone's server. Every model here was checked and measured against each card's memory from its actual weight files.

The short answer

The pattern to notice: from 16 GB up, every tier's largest model is the same 35B mixture-of-experts family. A bigger card does not buy you a bigger model. It buys you the same model with less of it rounded away.

How "fits" is decided

A model fits a card when a real file of it leaves room for 8K tokens of context, using 90% of the card's memory. The rest goes to the driver and the program running the model. The file has to exist: either the published weights or a GGUF version somebody has actually uploaded. Models that would only fit as a hypothetical 4-bit conversion nobody has made are left out, so everything below is something you can download today.

The tables are sorted by downloads on Hugging Face in the last 30 days. That is a measure of what people use, not of quality. Each model name opens its page, where you can see the file to get, the command to run it, and how much context the remaining memory holds. The hardware calculator has the same tables for every card, with the settings adjustable.

Local LLMs on 8 GB of VRAM

The most common card people own, and the tier with the least room. Small models are the honest choice here: Qwen3-4B and the 7B and 8B models at 4 to 5 bits per weight run comfortably with a real amount of context left over.

Local LLMs on 12 GB of VRAM

Four more gigabytes is enough to move 8B models up to 5 or 6 bits, where the difference from the original is hard to notice, and to bring 14B models in at 4 bits. The 30B mixture-of-experts models start to appear at the bottom of the range, at around 2 bits per weight. They run, but at that compression a good 8B model will usually give better answers.

Models under 3B are left out of this table and the ones below: they fit on any card, and they are in the 8 GB table above.

Local LLMs on 16 GB of VRAM

This is where the story changes. Qwen3-Coder-30B-A3B and the other 30B to 35B mixture-of-experts models fit at about 3 bits per weight, and because only 3B of their parameters run for each token, they are fast on a card this size. Three bits is a real compromise, with more repetition and weaker reasoning than the same model at five, but it is the first tier where a 30B-class model is a reasonable daily choice. Everything 8B and under runs at 8 bits, which is effectively the original. Models under 7B are left out here.

Local LLMs on 24 GB of VRAM

The sweet spot for running local LLMs, and the tier most of this site's guides assume. The 30B to 35B mixture-of-experts models fit at 4 to 5 bits per weight, where the loss against the original is small. 14B dense models run at 8 bits. The most downloaded models that fit are the same ones that fit a 32 GB card; the difference is which file each card can afford. Models under 7B are left out here.

Local LLMs on 32 GB of VRAM

Eight more gigabytes than a 4090 does not change which models fit. It changes how much of them survives: the same 35B models at 5 to 6 bits per weight instead of 4 to 5, or, for models the lab published in a 4-bit format like NVFP4, the published weights themselves with no further quantization. If you already own a 24 GB card, the upgrade buys quality, not reach. Models under 14B are left out here: this card is for the big ones.

Which file to download

Every model in the tables names a file, and the file name says how much was rounded away. The number after Q is roughly the bits per weight: Q4 stores each weight in about half a byte, Q8 in a full byte. The letters after it are the scheme. K spends more bits on the layers that matter most, S, M and L are small, medium and large versions of that trade, IQ squeezes lower with a smarter codebook, and UD- is Unsloth's dynamic mix. Two rules of thumb:

• 4 bits and up is where most people stop noticing the difference. Q4_K_M and Q5_K_M are the usual choices.
• Under 3 bits is where the damage is obvious. The tables mark those files. Prefer a smaller model at 5 bits over a larger one at 2.

For a longer explanation, see the largest AI models that fit on your GPU.

Where local LLMs stop

The biggest open-weight models, Kimi-K3 at 2.78T parameters and Qwen3.8-2.4T at 2.45T, need a rack of data-centre GPUs even at 4 bits. Their pages say "Needs data-center GPUs", which means exactly that: no consumer card or combination of them holds the weights. The gap between what fits at home and the frontier is about 80 times, and it is not closing.

Related

• Which AI models can I run on my GPU?: every card, with the settings adjustable
• Best AI models for an 8 GB GPU (RTX 4060, RTX 5060)
• Best AI models for a 24 GB GPU (RTX 4090, RTX 3090)
• The most downloaded AI models that fit on your GPU
• The largest open-weight AI models, by parameter count