teknium/OpenHermes-2.5-Mistral-7B: what it needs to run
teknium/OpenHermes-2.5-Mistral-7B is a fine-tune of mistralai/Mistral-7B-v0.1. Its configuration matches Mistral-7B-v0.1's on every field that decides memory, so Mistral-7B-v0.1's requirements are this repository's.
Answered by
Geometrypublished data. Both configurations are published files, read at the revisions named below and compared field by field. Not a test that the model loads.Mistral-7B-v0.1 (mistralai/Mistral-7B-v0.1) — every field that decides memory is equal; what differs is noted below and does not change a memory answer.
Mistral-7B-v0.1: weight-file size by format
| Format | Size | Range | Basis |
|---|---|---|---|
| FP16 | 14.5 GB | 14.2 GB – 14.6 GB | reconstructed size |
| Q8_0 | 7.70 GB | 7.56 GB – 7.74 GB | reconstructed size |
| Q6_K | 5.94 GB | 5.83 GB – 5.97 GB | reconstructed size |
| Q5_K_M | 5.13 GB | 5.02 GB – 5.16 GB | reconstructed size |
| Q5_0 | 5.00 GB | 4.89 GB – 5.02 GB | reconstructed size |
| Q4_K_M | 4.37 GB | 4.26 GB – 4.40 GB | reconstructed size |
| Q4_0 | 4.11 GB | 4.00 GB – 4.14 GB | reconstructed size |
| Q3_K_M | 3.56 GB | 3.31 GB – 3.81 GB | reconstructed size |
| Q2_K | 2.65 GB | 2.54 GB – 2.75 GB | reconstructed size |
These are Mistral-7B-v0.1’s sizes. A model with the same geometry quantizes to the same size, to within any difference in parameter count noted above. Which devices hold each, and how fast they run it, is on Mistral-7B-v0.1’s page.
What was read, and where
- teknium/OpenHermes-2.5-Mistral-7B at revision 24c0bea14d53 is tagged by its publisher as a fine-tune of mistralai/Mistral-7B-v0.1.
- teknium/OpenHermes-2.5-Mistral-7B/config.json at 24c0bea14d53 was compared with mistralai/Mistral-7B-v0.1/config.json at 27d67f1b5f57, the revision the catalogue pins: layers and their pattern, widths, head counts, experts, latent and recurrent dimensions, vocabulary, tied embeddings and any vision or audio tower.
Differences that do not change a memory answer
- vocab_size: 32002 here, 32000 in the base (vocabulary within 1%).
Matched through a checked library default rather than a stated value: head_dim.
- Licence, as tagged
- apache-2.0
- Parameters in its safetensors index
- 7.2B
- Task, as tagged
- text-generation
- Last changed on Hugging Face
- 2024-02-19
Publishing this model? A badge for its card
[](https://llmbottleneck.com/hf/teknium/OpenHermes-2.5-Mistral-7B)
It states the memory to run the model at Q4_K_M with 8,192 tokens of context — built on this repository’s own Q4_K_M file when it publishes one — and links back to this page, where the evidence is.
Resolve another repository
The same answer as JSON: GET /v1/resolve?repo=teknium/OpenHermes-2.5-Mistral-7B, with a free key from /keys.