NVIDIA’s desk-side datacenter machine: GB300 on a Grace Blackwell NVL
architecture with 748GB of coherent unified memory at 7,100 GB/s - over
6x the M5 Ultra’s bandwidth. Shipped 2025 at $85,000. 1,600W TDP, so it
needs a 20A circuit, not a wall outlet budget.
What it does well:
-
Bandwidth: 7,100 GB/s decodes the largest open checkpoints faster
than any consumer machine on this site - single-stream rates measured
in the hundreds of tok/s on 100B-class models.
-
Full-precision serving: FP8 and NVFP4 formats run natively, and
748GB holds long-context KV caches on top of full checkpoints.
Where it falls short: price and power put it out of individual reach -
it is a lab and enterprise purchase. It draws more than ten times the
DGX Spark’s power for a bandwidth-first trade.
Run it locally: the same DGX OS stack as the Spark, with vLLM and
SGLang as the intended serving layer. The
modeldex results rank checkpoints by fit
and estimated tok/s.