KORMo-Web-4B

Try Try

KORMo-Web-4B has 5.17B parameters (4.21B active per token) and takes 10.3 GB on disk. See where its parameters live, layer by layer, and what hardware it needs.

At a glance

KORMo-VL/KORMo-Web-4B
Repo createdNov 14, 2026updated Nov 18, 2026
Model typeDenseHybrid attention: 24 linear + 8 full (GQA)
InputsText + images334M vision encoder
Total parameters5.17B
Active per token4.21B81% of the model
ExpertsNone (dense)
Max context (from config)256K tokens
Layers3224 linear + 8 full attention
On disk10.3 GB1 file
PrecisionBF16 (100%)
QuantizationNoneoriginal precision (BF16)
Fewest GPUs1× RTX 3060 12 GBa single 12 GB card · weights only
FamilyFine-tune of Qwen/Qwen3.5-4B
Licenseapache-2.0
GitHubNot linked