Architecture
Retrieved 2026-09-01 at pinned commit 834bf5ffe304; parameter count from the safetensors index at the same revision.
- Layers
- 38
- Hidden size
- 8,192
- Attention heads
- 64
- Vocabulary
- 163,840
- Context ceiling
- 262,144
- RoPE theta
- 1,000,000
- Experts
- 768
- Latent KV rank
- 512