Architecture
Retrieved 2026-09-01 at pinned commit 492c81c225fb; parameter count from the safetensors index at the same revision.
- Architecture
- yasa_model
- Layers
- 32
- Hidden size
- 4,096
- Attention heads
- 32
- KV heads
- 8
- Head dimension
- 128
- Feed-forward width
- 10,880
- Vocabulary
- 100,352
- Context ceiling
- 16,384
- RoPE theta
- 10,000
Where the memory goes
Grouped-query attention shares each key/value head across 4 query heads, so the KV cache is 25% of what multi-head attention would need at the same context.
This is a multimodal checkpoint (vision); the diagram and memory sizing above cover the text decoder. Encoder/audio/image activations are not included in the KV or weight figures.