Can I run Nanbeige4.3-33B on an RTX 6000 Ada?

Yes, with a GGUF version

The published weights are too big, but the Q8_0 file from bartowski/Nanbeige_Nanbeige4.3-33B-GGUF (35.2 GB) fits, with room for ≈ 37K tokens of context.

Full analysis of Nanbeige4.3-33B Everything that runs on RTX 6000 Ada Get the Q8_0 file on Hugging Face

ollama run hf.co/bartowski/Nanbeige_Nanbeige4.3-33B-GGUF:Q8_0

Files and context on RTX 6000 Ada

Each file of Nanbeige4.3-33B (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.

File	Size	On RTX 6000 Ada	Context (16-bit)	Context (8-bit)
As published	66.2 GB	Doesn't fit	—	—
BF16bartowski/Nanbeige_Nanbeige4.3-33B-GGUF	66.2 GB	Doesn't fit	—	—
Q8_0bartowski/Nanbeige_Nanbeige4.3-33B-GGUF	35.2 GB	Tight	37K	73K
Q6_K_Lbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	30.4 GB	Fits	59K	117K
Q6_Kbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	29.0 GB	Fits	65K	130K
Q6_K_Sbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	27.8 GB	Fits	71K	142K
Q5_K_Mbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	25.1 GB	Fits	83K	166K
Q5_K_Sbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	23.7 GB	Fits	90K	179K
Q4_K_Lbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	22.3 GB	Fits	96K	191K
Q4_1bartowski/Nanbeige_Nanbeige4.3-33B-GGUF	21.2 GB	Fits	101K	202K
IQ4_NLbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	20.9 GB	Fits	102K	205K
Q4_K_Mbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	20.8 GB	Fits	103K	206K
Q4_K_Sbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	19.7 GB	Fits	108K	215K
Q4_0bartowski/Nanbeige_Nanbeige4.3-33B-GGUF	19.3 GB	Fits	109K	219K
IQ4_XSbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	18.6 GB	Fits	113K	225K
IQ3_Mbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	17.8 GB	Fits	116K	233K
Q3_K_Lbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	17.3 GB	Fits	119K	238K
Q3_K_Mbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	15.9 GB	Fits	125K	250K
IQ3_XSbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	15.2 GB	Fits	128K	257K
Q3_K_Sbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	15.2 GB	Fits	128K	257K
IQ3_XXSbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	14.5 GB	Fits	132K	263K
Q2_Kbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	13.5 GB	Fits	136K	272K
IQ2_Mbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	12.2 GB	Fits	142K	284K
IQ2_Sbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	11.2 GB	Fits	147K	294K
IQ2_XSbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	10.6 GB	Fits	149K	299K
IQ2_XXSbartowski/Nanbeige_Nanbeige4.3-33B-GGUF	10.1 GB	Fits	152K	304K
Q8_0AtomicChat/Nanbeige4.3-33B-GGUF	35.2 GB	Tight	37K	73K
Q6_KAtomicChat/Nanbeige4.3-33B-GGUF	30.1 GB	Fits	60K	120K
Q5_KAtomicChat/Nanbeige4.3-33B-GGUF	27.9 GB	Fits	70K	140K
Q5_K_MAtomicChat/Nanbeige4.3-33B-GGUF	24.7 GB	Fits	85K	170K
Nanbeige4.3-33B-Q5_K_MAtomicChat/Nanbeige4.3-33B-GGUF	23.5 GB	Fits	90K	181K
Nanbeige4.3-33B-AD-Q5_K_M-Q4_K_MAtomicChat/Nanbeige4.3-33B-GGUF	22.6 GB	Fits	94K	189K
Q4_K_MAtomicChat/Nanbeige4.3-33B-GGUF	21.2 GB	Fits	101K	202K
IQ4_XSAtomicChat/Nanbeige4.3-33B-GGUF	20.3 GB	Fits	105K	210K
IQ3_SAtomicChat/Nanbeige4.3-33B-GGUF	17.1 GB	Fits	120K	240K
IQ3_XXSAtomicChat/Nanbeige4.3-33B-GGUF	14.9 GB	Fits	130K	259K
IQ2_SAtomicChat/Nanbeige4.3-33B-GGUF	13.7 GB	Fits	135K	270K

Nanbeige4.3-33B on other hardware

Popular models that run on RTX 6000 Ada

Assumes 90% of the card's memory is usable and at least 4K tokens of context for one conversation, with the context cache in 16-bit unless noted. Fits leaves at least 20% of that memory for context; Tight leaves less.