Models / Llama 3.1 8B Instruct

Llama 3.1 8B Instruct

consumer
HuggingFace → Meta → Find for my hardware

AI-generated content marks

The provider reports that this model does not add embedded watermarks or provenance metadata to generated output.

Provider transparency docs →

coding chat general
Parameters
8.0B
Context
128k
License
llama 3
Developer
Meta
Origin
🇺🇸 USA
Released
Jul 2024

Scores

Coding
52
Reasoning
55
General
62

Score per dollar

3100 pts per $/M input

general_score (62) divided by cheapest input price ($0.02/M). Higher is better value. See live pricing.

Related models

Llama 3.3 70B Instruct
70.6B enthusiast 22GB min RAM $0.10/M in Dec 6, 2024
Llama 3.2 1B Instruct
1.2B edge 1GB min RAM $0.03/M in Sep 25, 2024
Llama 3.2 3B Instruct
3.2B edge 2GB min RAM $0.05/M in Sep 25, 2024
Llama 3.1 405B
405.0B premier no local build Jul 23, 2024

Save your hardware and every model page answers the real question: will it run on your machine, and how fast?

Join free - save your rig →

Run it locally

Per-quant memory needs and a static "can you run it?" reference - no rig entry required

Q4_K_M
4.5GB 4.5GB min 6.0GB rec
Balanced - the usual local sweet spot
Q8_0
8.0GB 8.0GB min 9.0GB rec
Near-lossless
FP16
16.0GB 16.0GB min 18.0GB rec
Full quality, largest

The reference hardware

Schematic of the 4x H100 80GB (320GB) reference rig - 320GB VRAM, 13400 GB/s aggregate bandwidth
Schematic of the NVIDIA DGX Station 748GB reference rig - 748GB unified memory, 8000 GB/s aggregate bandwidth
Schematic of the 8x RTX 3090 rack (192GB) reference rig - 192GB VRAM, 7489 GB/s aggregate bandwidth
Schematic of the 4x RTX 5090 (128GB) reference rig - 128GB VRAM, 7168 GB/s aggregate bandwidth
Schematic of the AMD Instinct MI300X (192GB) reference rig - 192GB VRAM, 5324 GB/s aggregate bandwidth
Schematic of the 4x RTX 4090 (96GB) reference rig - 96GB VRAM, 4032 GB/s aggregate bandwidth
Schematic of the 2x RTX 5090 (64GB) reference rig - 64GB VRAM, 3584 GB/s aggregate bandwidth
Schematic of the 2x RTX 3090 (48GB) reference rig - 48GB VRAM, 1872 GB/s aggregate bandwidth
Schematic of the Single RTX 5090 (32GB) reference rig - 32GB VRAM, 1792 GB/s aggregate bandwidth
Schematic of the RTX PRO 6000 Blackwell (96GB) reference rig - 96GB VRAM, 1792 GB/s aggregate bandwidth
Schematic of the Mac Studio M4 Ultra 192GB reference rig - 192GB unified memory, 1092 GB/s aggregate bandwidth
Schematic of the Mac Studio M4 Ultra 512GB reference rig - 512GB unified memory, 1092 GB/s aggregate bandwidth
Schematic of the Single RTX 4090 (24GB) reference rig - 24GB VRAM, 1008 GB/s aggregate bandwidth
Schematic of the MacBook Pro M5 Max 128GB reference rig - 128GB unified memory, 614 GB/s aggregate bandwidth
Schematic of the Single GTX 1080 Ti (11GB) reference rig - 11GB VRAM, 484 GB/s aggregate bandwidth
Schematic of the Dual EPYC 9004 + 768GB DDR5-4800 reference rig - 768GB unified memory, 460 GB/s aggregate bandwidth
Schematic of the DGX Spark 128GB unified reference rig - 128GB unified memory, 273 GB/s aggregate bandwidth
Schematic of the Ryzen AI Max+ 395 128GB reference rig - 128GB unified memory, 256 GB/s aggregate bandwidth
Schematic of the Jetson AGX Orin 64GB reference rig - 64GB unified memory, 204 GB/s aggregate bandwidth
Schematic of the Epyc + 512GB DDR4-3200 + 2x RTX 3090 reference rig - 560GB unified memory, 204 GB/s aggregate bandwidth
Schematic of the Epyc + 512GB DDR4-2400 + 2x RTX 3090 reference rig - 560GB unified memory, 153 GB/s aggregate bandwidth
Schematic of the NVIDIA Jetson Orin NX 16GB reference rig - 16GB unified memory, 102 GB/s aggregate bandwidth

22 reference configs, drawn in-house. Scroll for more.

Can you run it? - reference rigs

Rig Q4_K_M Q8_0 FP16
4x H100 80GB (320GB) fast 1637.1t/s fast 921.0t/s fast 460.6t/s
NVIDIA DGX Station 748GB fast 977.3t/s fast 549.9t/s fast 275.0t/s
8x RTX 3090 rack (192GB) fast 915.0t/s fast 514.8t/s fast 257.4t/s
4x RTX 5090 (128GB) fast 875.7t/s fast 492.7t/s fast 246.4t/s
AMD Instinct MI300X (192GB) fast 650.5t/s fast 366.0t/s fast 183.0t/s
4x RTX 4090 (96GB) fast 492.6t/s fast 277.1t/s fast 138.6t/s
2x RTX 5090 (64GB) fast 437.9t/s fast 246.3t/s fast 123.2t/s
2x RTX 3090 (48GB) fast 228.8t/s fast 128.7t/s fast 64.4t/s
Single RTX 5090 (32GB) fast 218.9t/s fast 123.2t/s fast 61.6t/s
RTX PRO 6000 Blackwell (96GB) fast 218.9t/s fast 123.2t/s fast 61.6t/s
Mac Studio M4 Ultra 192GB fast 145.5t/s fast 81.9t/s fast 40.9t/s
Mac Studio M4 Ultra 512GB fast 145.5t/s fast 81.9t/s fast 40.9t/s
Single RTX 4090 (24GB) fast 123.2t/s fast 69.3t/s fast 34.7t/s
MacBook Pro M5 Max 128GB fast 81.8t/s fast 46.0t/s fast 23.0t/s
Single GTX 1080 Ti (11GB) fast 59.1t/s fast 33.3t/s offload
Dual EPYC 9004 + 768GB DDR5-4800 fast 56.3t/s fast 31.7t/s ok 15.8t/s
DGX Spark 128GB unified fast 33.4t/s ok 18.8t/s ok 9.4t/s
Ryzen AI Max+ 395 128GB fast 31.3t/s ok 17.6t/s ok 8.8t/s
Jetson AGX Orin 64GB fast 25.0t/s ok 14.1t/s slow 7.0t/s
Epyc + 512GB DDR4-3200 + 2x RTX 3090 fast 25.0t/s ok 14.1t/s slow 7.0t/s
Epyc + 512GB DDR4-2400 + 2x RTX 3090 ok 18.8t/s ok 10.6t/s slow 5.3t/s
NVIDIA Jetson Orin NX 16GB ok 12.5t/s slow 7.0t/s tight

Fit tiers use the same will-it-run logic as the rig finder. For comfortable fits, the badge reflects decode speed: fast >=20 t/s, ok 8-20 t/s, slow <8 t/s. t/s is a bandwidth estimate, not a measured benchmark.

Download options

Q4_K_M official -4% vs fp16
4.5GB dl 4.5GB min 6.0GB rec
REC RAM vs largest quant
Q8_0 official -1% vs fp16
8.0GB dl 8.0GB min 9.0GB rec
REC RAM vs largest quant
FP16 official
16.0GB dl 16.0GB min 18.0GB rec
REC RAM vs largest quant

Or run it in the cloud

Live per-provider pricing, throughput and uptime - refreshed about 19 hours ago via OpenRouter. Click a column to sort.

some pricing may be stale - last verified 2026-10-24

Provider Type Input $/M Output $/M Cache $/M Tok/s Latency Uptime Value
DeepInfra
API 0.02 0.04 0.025 - - 100.00% best uptime
Novita
API 0.02 0.05 0.025 - - 100.00%
Groq
API 0.05 0.08 0.025 - - 100.00%
Cloudflare
API 0.15 0.29 0.025 - - 100.00%
Together AI stale
API 0.18 0.18 - - - -
Fireworks AI stale
API 0.20 0.20 - - - -
CoreWeave
API 0.22 0.22 0.220 - - 100.00%

Default order: throughput among 95%+ uptime providers, then latency; subscriptions last. Sort by any column. Subscription rows show $/mo in the Value column - per-token columns are "-". Affiliate links are marked sponsored / nofollow. Confirm current pricing on the provider's site before committing.

Detailed API pricing page + JSON endpoint →

See who runs Meta in production →

PRICE HISTORY

Inference cost over time

Data accumulates from the first daily sync - longer ranges populate over time. Prices come from OpenRouter snapshots, not a historical API.

Loading price history...