Run it locally
Per-quant memory needs and a static "can you run it?" reference - no rig entry required
Q8_0
0.3GB
0.5GB min
0.8GB rec
Near-lossless
QAT
0.2GB
0.5GB min
0.8GB rec
Quantization-aware training - near-lossless at Q4 size
The reference hardware
22 reference configs, drawn in-house. Scroll for more.
Can you run it? - reference rigs
| Rig |
Q8_0 |
QAT |
| 4x H100 80GB (320GB) |
fast 25406.8t/s
|
fast 30698.1t/s
|
| NVIDIA DGX Station 748GB |
fast 15168.2t/s
|
fast 18327.2t/s
|
| 8x RTX 3090 rack (192GB) |
fast 14200.5t/s
|
fast 17158.0t/s
|
| 4x RTX 5090 (128GB) |
fast 13590.7t/s
|
fast 16421.2t/s
|
| AMD Instinct MI300X (192GB) |
fast 10096.0t/s
|
fast 12198.6t/s
|
| 4x RTX 4090 (96GB) |
fast 7644.8t/s
|
fast 9236.9t/s
|
| 2x RTX 5090 (64GB) |
fast 6795.4t/s
|
fast 8210.6t/s
|
| 2x RTX 3090 (48GB) |
fast 3550.1t/s
|
fast 4289.5t/s
|
| Single RTX 5090 (32GB) |
fast 3397.7t/s
|
fast 4105.3t/s
|
| RTX PRO 6000 Blackwell (96GB) |
fast 3397.7t/s
|
fast 4105.3t/s
|
| Mac Studio M4 Ultra 192GB |
fast 2258.7t/s
|
fast 2729.1t/s
|
| Mac Studio M4 Ultra 512GB |
fast 2258.7t/s
|
fast 2729.1t/s
|
| Single RTX 4090 (24GB) |
fast 1911.2t/s
|
fast 2309.2t/s
|
| MacBook Pro M5 Max 128GB |
fast 1270.0t/s
|
fast 1534.5t/s
|
| Single GTX 1080 Ti (11GB) |
fast 917.7t/s
|
fast 1108.8t/s
|
| Dual EPYC 9004 + 768GB DDR5-4800 |
fast 873.7t/s
|
fast 1055.7t/s
|
| DGX Spark 128GB unified |
fast 517.6t/s
|
fast 625.4t/s
|
| Ryzen AI Max+ 395 128GB |
fast 485.4t/s
|
fast 586.5t/s
|
| Jetson AGX Orin 64GB |
fast 388.3t/s
|
fast 469.2t/s
|
| Epyc + 512GB DDR4-3200 + 2x RTX 3090 |
fast 388.3t/s
|
fast 469.2t/s
|
| Epyc + 512GB DDR4-2400 + 2x RTX 3090 |
fast 291.2t/s
|
fast 351.9t/s
|
| NVIDIA Jetson Orin NX 16GB |
fast 194.2t/s
|
fast 234.6t/s
|
Fit tiers use the same will-it-run logic as the rig finder. For comfortable fits, the badge reflects decode speed: fast >=20 t/s, ok 8-20 t/s, slow <8 t/s. t/s is a bandwidth estimate, not a measured benchmark.
PRICE HISTORY
Inference cost over time
Data accumulates from the first daily sync - longer ranges populate over time. Prices come from OpenRouter snapshots, not a historical API.
Loading price history...