NVIDIA’s embedded module family, three generations on this site: Orin
Nano Super (8GB, 68 GB/s, $249, 2024 - the “Super” refresh of the 2023
board), Orin NX 16GB (102 GB/s, $499, 2023), and AGX Orin 64GB (205 GB/s,
$899, 2022). All run JetPack Linux on ARM with CUDA.
What it does well:
-
Power envelope: 10-60W total, passively or lightly cooled - the
only CUDA platform here that runs on a battery or a solar rig.
-
CUDA on a budget: $249 buys a real NVIDIA stack for 1-8B models,
with the full llama.cpp and MNN toolchain.
Where it falls short: 68 GB/s on the Nano Super caps single-stream
decode near single-digit tok/s on 7B-class models; 8-64GB means only
small dense models or heavily quantized MoE fit.
Run it locally: Jetson AI Lab containers, Ollama, or llama.cpp on
JetPack. The modeldex results rank
checkpoints by fit and estimated tok/s.