Models / Mixtral 8x7B Instruct

Mixtral 8x7B Instruct

MoE enthusiast
HuggingFace → HuggingBay → Mistral AI → Find for my hardware

46.7B total, 12.9B active per token (8x7B MoE). Requires ~24GB for Q4_K_M.

coding chat general
Parameters
46.7B
Context
32k
License
apache 2.0
Developer
Mistral AI
Origin
🇫🇷 France
Released
Dec 2023

Scores

Coding
62
Reasoning
65
General
68

Related models

Mistral Nemo 12B
12.2B consumer 7GB min RAM $0.02/M in Jul 18, 2024

Save your hardware and every model page answers the real question: will it run on your machine, and how fast?

Join free - save your rig →

Run it locally

Per-quant memory needs and a static "can you run it?" reference - no rig entry required

Q3_K_M
19.0GB 19.0GB min 21.0GB rec
Compact, moderate quality loss
Q4_K_M
26.5GB 26.5GB min 28.0GB rec
Balanced - the usual local sweet spot

The reference hardware

Schematic of the NVIDIA Jetson Orin NX 16GB reference rig - 16GB unified memory, 102 GB/s aggregate bandwidth
Schematic of the Single GTX 1080 Ti (11GB) reference rig - 11GB VRAM, 484 GB/s aggregate bandwidth
Schematic of the 4x H100 80GB (320GB) reference rig - 320GB VRAM, 13400 GB/s aggregate bandwidth
Schematic of the NVIDIA DGX Station 748GB reference rig - 748GB unified memory, 8000 GB/s aggregate bandwidth
Schematic of the 8x RTX 3090 rack (192GB) reference rig - 192GB VRAM, 7489 GB/s aggregate bandwidth
Schematic of the 4x RTX 5090 (128GB) reference rig - 128GB VRAM, 7168 GB/s aggregate bandwidth
Schematic of the AMD Instinct MI300X (192GB) reference rig - 192GB VRAM, 5324 GB/s aggregate bandwidth
Schematic of the 4x RTX 4090 (96GB) reference rig - 96GB VRAM, 4032 GB/s aggregate bandwidth
Schematic of the 2x RTX 5090 (64GB) reference rig - 64GB VRAM, 3584 GB/s aggregate bandwidth
Schematic of the 2x RTX 3090 (48GB) reference rig - 48GB VRAM, 1872 GB/s aggregate bandwidth
Schematic of the Single RTX 5090 (32GB) reference rig - 32GB VRAM, 1792 GB/s aggregate bandwidth
Schematic of the RTX PRO 6000 Blackwell (96GB) reference rig - 96GB VRAM, 1792 GB/s aggregate bandwidth
Schematic of the Mac Studio M4 Ultra 192GB reference rig - 192GB unified memory, 1092 GB/s aggregate bandwidth
Schematic of the Mac Studio M4 Ultra 512GB reference rig - 512GB unified memory, 1092 GB/s aggregate bandwidth
Schematic of the Single RTX 4090 (24GB) reference rig - 24GB VRAM, 1008 GB/s aggregate bandwidth
Schematic of the MacBook Pro M5 Max 128GB reference rig - 128GB unified memory, 614 GB/s aggregate bandwidth
Schematic of the Dual EPYC 9004 + 768GB DDR5-4800 reference rig - 768GB unified memory, 460 GB/s aggregate bandwidth
Schematic of the DGX Spark 128GB unified reference rig - 128GB unified memory, 273 GB/s aggregate bandwidth
Schematic of the Ryzen AI Max+ 395 128GB reference rig - 128GB unified memory, 256 GB/s aggregate bandwidth
Schematic of the Jetson AGX Orin 64GB reference rig - 64GB unified memory, 204 GB/s aggregate bandwidth
Schematic of the Epyc + 512GB DDR4-3200 + 2x RTX 3090 reference rig - 560GB unified memory, 204 GB/s aggregate bandwidth
Schematic of the Epyc + 512GB DDR4-2400 + 2x RTX 3090 reference rig - 560GB unified memory, 153 GB/s aggregate bandwidth

22 reference configs, drawn in-house. Scroll for more.

Can you run it? - reference rigs

Rig Q3_K_M Q4_K_M
NVIDIA Jetson Orin NX 16GB no -> cloud no -> cloud
Single GTX 1080 Ti (11GB) no -> cloud no -> cloud
4x H100 80GB (320GB) fast 1403.7t/s fast 1006.5t/s
NVIDIA DGX Station 748GB fast 838.0t/s fast 600.9t/s
8x RTX 3090 rack (192GB) fast 784.6t/s fast 562.6t/s
4x RTX 5090 (128GB) fast 750.9t/s fast 538.4t/s
AMD Instinct MI300X (192GB) fast 557.8t/s fast 400.0t/s
4x RTX 4090 (96GB) fast 422.4t/s fast 302.9t/s
2x RTX 5090 (64GB) fast 375.4t/s fast 269.2t/s
2x RTX 3090 (48GB) fast 196.1t/s fast 140.6t/s
Single RTX 5090 (32GB) fast 187.7t/s fast 134.6t/s
RTX PRO 6000 Blackwell (96GB) fast 187.7t/s fast 134.6t/s
Mac Studio M4 Ultra 192GB fast 124.8t/s fast 89.5t/s
Mac Studio M4 Ultra 512GB fast 124.8t/s fast 89.5t/s
Single RTX 4090 (24GB) fast 105.6t/s offload
MacBook Pro M5 Max 128GB fast 70.2t/s fast 50.3t/s
Dual EPYC 9004 + 768GB DDR5-4800 fast 48.3t/s fast 34.6t/s
DGX Spark 128GB unified fast 28.6t/s fast 20.5t/s
Ryzen AI Max+ 395 128GB fast 26.8t/s ok 19.2t/s
Jetson AGX Orin 64GB fast 21.5t/s ok 15.4t/s
Epyc + 512GB DDR4-3200 + 2x RTX 3090 fast 21.5t/s ok 15.4t/s
Epyc + 512GB DDR4-2400 + 2x RTX 3090 ok 16.1t/s ok 11.5t/s

Fit tiers use the same will-it-run logic as the rig finder. For comfortable fits, the badge reflects decode speed: fast >=20 t/s, ok 8-20 t/s, slow <8 t/s. t/s is a bandwidth estimate, not a measured benchmark.

Download options

Q3_K_M official -10% vs fp16
19.0GB dl 19.0GB min 21.0GB rec
REC RAM vs largest quant
Q4_K_M official -5% vs fp16
26.5GB dl 26.5GB min 28.0GB rec
REC RAM vs largest quant

Or run it in the cloud

No per-token API provider pricing tracked for Mixtral 8x7B Instruct yet. For flagship list prices, see the calculator.

See who runs Mistral AI in production →

PRICE HISTORY

Inference cost over time

Data accumulates from the first daily sync - longer ranges populate over time. Prices come from OpenRouter snapshots, not a historical API.

Loading price history...