Architecture
Retrieved 2026-09-01 at pinned commit f0e1c1e04d40; parameter count from the safetensors index at the same revision.
- Architecture
- minimax_m3_vl
- Layers
- 60
- Hidden size
- 6,144
- Attention heads
- 64
- KV heads
- 4
- Head dimension
- 128
- Feed-forward width
- 3,072
- Vocabulary
- 200,064
- Context ceiling
- 1,048,576
- RoPE theta
- 5,000,000
- Experts
- 128
- Experts / token
- 4
- Shared experts
- 1
- Shared expert width
- 3,072
Where the memory goes
Grouped-query attention shares each key/value head across 16 query heads, so the KV cache is 6% of what multi-head attention would need at the same context.
This is a multimodal checkpoint (vision); the diagram and memory sizing above cover the text decoder. Encoder/audio/image activations are not included in the KV or weight figures.