Can I run MiniCPM-V-4.7-35B-A3B on a Radeon RX 7900 XTX?

Yes, with a GGUF version

The published weights are too big, but the Q4_K_M file from openbmb/MiniCPM-V-4.7-35B-A3B-gguf (21.2 GB) fits, with room for ≈ 18K tokens of context.

Full analysis of MiniCPM-V-4.7-35B-A3B Everything that runs on Radeon RX 7900 XTX Get the Q4_K_M file on Hugging Face

ollama run hf.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_M

Files and context on Radeon RX 7900 XTX

Each file of MiniCPM-V-4.7-35B-A3B (and of GGUF versions the site has analyzed), and how much context fits next to it, with the context cache in 16-bit or 8-bit.

File	Size	On Radeon RX 7900 XTX	Context (16-bit)	Context (8-bit)
As published	70.4 GB	Doesn't fit	—	—
F16openbmb/MiniCPM-V-4.7-35B-A3B-gguf	69.4 GB	Doesn't fit	—	—
MiniCPM-V-4_7-35B-A3B-F16-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	69.4 GB	Doesn't fit	—	—
Q8_0openbmb/MiniCPM-V-4.7-35B-A3B-gguf	36.9 GB	Doesn't fit	—	—
MiniCPM-V-4_7-35B-A3B-Q8_0-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	36.9 GB	Doesn't fit	—	—
Q6_Kopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	28.5 GB	Doesn't fit	—	—
MiniCPM-V-4_7-35B-A3B-Q6_K-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	28.5 GB	Doesn't fit	—	—
Q5_1openbmb/MiniCPM-V-4.7-35B-A3B-gguf	26.1 GB	Doesn't fit	—	—
MiniCPM-V-4_7-35B-A3B-Q5_1-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	26.1 GB	Doesn't fit	—	—
Q5_K_Mopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	24.7 GB	Doesn't fit	—	—
MiniCPM-V-4_7-35B-A3B-Q5_K_M-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	24.7 GB	Doesn't fit	—	—
Q5_0openbmb/MiniCPM-V-4.7-35B-A3B-gguf	24.0 GB	Doesn't fit	—	—
Q5_K_Sopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	24.0 GB	Doesn't fit	—	—
MiniCPM-V-4_7-35B-A3B-Q5_0-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	24.0 GB	Doesn't fit	—	—
MiniCPM-V-4_7-35B-A3B-Q5_K_S-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	24.0 GB	Doesn't fit	—	—
Q4_1openbmb/MiniCPM-V-4.7-35B-A3B-gguf	21.8 GB	Doesn't fit	—	—
MiniCPM-V-4_7-35B-A3B-Q4_1-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	21.8 GB	Doesn't fit	—	—
Q4_K_Mopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	21.2 GB	Tight	18K	35K
MiniCPM-V-4_7-35B-A3B-Q4_K_M-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	21.2 GB	Tight	18K	35K
Q4_K_Sopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	19.9 GB	Tight	78K	157K
MiniCPM-V-4_7-35B-A3B-Q4_K_S-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	19.9 GB	Tight	78K	157K
IQ4_NLopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	19.8 GB	Tight	84K	168K
MiniCPM-V-4_7-35B-A3B-IQ4_NL-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	19.8 GB	Tight	84K	168K
Q4_0openbmb/MiniCPM-V-4.7-35B-A3B-gguf	19.7 GB	Tight	87K	174K
MiniCPM-V-4_7-35B-A3B-Q4_0-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	19.7 GB	Tight	87K	174K
IQ4_XSopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	18.7 GB	Tight	134K	256K
MiniCPM-V-4_7-35B-A3B-IQ4_XS-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	18.7 GB	Tight	134K	256K
MiniCPM-V-4_7-35B-A3B-mix_q3gud_imopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	15.5 GB	Fits	256K	256K
MiniCPM-V-4_7-35B-A3B-mix_q3gud_im-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	15.5 GB	Fits	256K	256K
IQ3_Mopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	15.4 GB	Fits	256K	256K
MiniCPM-V-4_7-35B-A3B-IQ3_M-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	15.4 GB	Fits	256K	256K
IQ3_XSopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	14.5 GB	Fits	256K	256K
MiniCPM-V-4_7-35B-A3B-IQ3_XS-thinkingopenbmb/MiniCPM-V-4.7-35B-A3B-gguf	14.5 GB	Fits	256K	256K

MiniCPM-V-4.7-35B-A3B on other hardware

Popular models that run on Radeon RX 7900 XTX

Assumes 90% of the card's memory is usable and at least 4K tokens of context for one conversation, with the context cache in 16-bit unless noted. Fits leaves at least 20% of that memory for context; Tight leaves less.