AMD’s desktop GPUs, two generations on this site: RX 7900 XTX (24GB
GDDR6, 960 GB/s, 355W, $999 MSRP, 2022) and RX 9070 / 9070 XT (16GB
GDDR6, 640 GB/s, $549 / $799, 2025).
What it does well:
-
Bandwidth per dollar: the 7900 XTX’s 960 GB/s at $999 beats every
NVIDIA card in the same bracket for raw decode headroom on models
that fit.
-
24GB tier: the 7900 XTX holds 70B dense at 3-bit or 30B at 8-bit
at consumer prices.
Where it falls short: ROCm support in the mainstream runtimes
(Ollama, llama.cpp) has historically lagged CUDA releases; quantization
formats like EXL3 are CUDA-first, so the practical stack is GGUF via
Vulkan or ROCm.
Run it locally: Ollama or llama.cpp with ROCm/Vulkan builds. The
modeldex results rank checkpoints by fit
and estimated tok/s.