Can I run it? / Jetson Orin Nano
Best LLMs to run on a Jetson Orin Nano
Updated November 2026
As of November 2026, the best model to run on a Jetson Orin Nano is Nanbeige4.2 3B by Nanbeige (Kanzhun) (Small LM Index 67.2), at about 20 tokens/s with 4-bit weights. On the Jetson Orin NX 16 GB, Ternary Bonsai 2 27B (71.2) fits.
The Orin Nano Super has 8 GB shared between the system and the GPU, at 102 GB/s: six times the bandwidth of a Raspberry Pi, so 3–4B models run at chat speed. About 6.5 GB is usable for a model, at 102 GB/s. The figures below assume 4-bit weights and a 4K-token context, and only include models that generate at least 5 tokens per second.
Models that run well on a Jetson Orin Nano
The 25 best of the 45 models that run at 5+ tokens/s, ranked by the Small LM Index.
| # | Model | Index | Memory | Speed |
|---|---|---|---|---|
| 1 | Nanbeige4.2 3B Nanbeige (Kanzhun) · 4-bit · Apache-2.0 | 67.2 | 3.5 GB | ≈ 20 tok/s |
| 2 | Spark-X2.6 4B SparkLLM (XHToken) · 4-bit · Apache-2.0 | 66.5 | 3.3 GB | ≈ 20 tok/s |
| 3 | Spark-X2.5 4B SparkLLM (XHToken) · 4-bit · Apache-2.0 | 65.9 | 3.3 GB | ≈ 20 tok/s |
| 4 | Qwen3.5 9B Alibaba (Qwen) · 4-bit · Apache-2.0 | 64.1 | 6.3 GB | ≈ 9 tok/s |
| 5 | Granite 4.2 8B IBM (Granite) · 4-bit · Apache-2.0 | 62.7 | 6.2 GB | ≈ 11 tok/s |
| 6 | LFM3 1.7B Liquid AI · 4-bit · LFM Open License | 61.2 | 1.6 GB | ≈ 49 tok/s |
| 7 | MiniCPM5 2B OpenBMB (MiniCPM) · 4-bit · Apache-2.0 | 60.9 | 2.3 GB | ≈ 33 tok/s |
| 8 | Qwen3 4B Thinking 2507 Alibaba (Qwen) · 4-bit · Apache-2.0 | 59.8 | 3.6 GB | ≈ 21 tok/s |
| 9 | Qwen3.5 4B Alibaba (Qwen) · 4-bit · Apache-2.0 | 59.7 | 3.1 GB | ≈ 21 tok/s |
| 10 | Nemotron Labs Dory 1.0 4B NVIDIA · 4-bit · OpenMDW-1.1 | 58.6 | 3.3 GB | ≈ 21 tok/s |
| 11 | Falcon H1R 7B TII (Falcon) · 4-bit · Falcon LLM License | 58.6 | 5.4 GB | ≈ 11 tok/s |
| 12 | Ministral 3 8B Mistral AI · 4-bit · Apache-2.0 | 57.7 | 6.1 GB | ≈ 11 tok/s |
| 13 | Granite 4.2 3B IBM (Granite) · 4-bit · Apache-2.0 | 57.3 | 2.7 GB | ≈ 28 tok/s |
| 14 | DeepSeek-R1-0528-Qwen3-8B DeepSeek · 4-bit · MIT | 56.3 | 6.2 GB | ≈ 10 tok/s |
| 15 | Hunyuan 7B Instruct Tencent (Hunyuan) · 4-bit · Tencent Hunyuan Community | 55.8 | 5.7 GB | ≈ 11 tok/s |
| 16 | Ling 3.0 Tiny InclusionAI (Ling) · 4-bit · MIT | 54.9 | 5.6 GB | ≈ 30 tok/s |
| 17 | Qwen3 8B Alibaba (Qwen) · 4-bit · Apache-2.0 | 53.9 | 6.2 GB | ≈ 10 tok/s |
| 18 | Ministral 3 3B Mistral AI · 4-bit · Apache-2.0 | 53.8 | 2.8 GB | ≈ 28 tok/s |
| 19 | Nemotron Nano 9B v2 NVIDIA · 4-bit · NVIDIA Open Model | 52.9 | 6.1 GB | ≈ 9 tok/s |
| 20 | Spark-X2.6 1.7B SparkLLM (XHToken) · 4-bit · Apache-2.0 | 52.3 | 1.7 GB | ≈ 49 tok/s |
| 21 | Edge0 8B-A1B (preview) Edge0 · 4-bit · Apache-2.0 | 50.1 | 5.6 GB | ≈ 30 tok/s |
| 22 | MiMo-7B-RL Xiaomi (MiMo) · 4-bit · MIT | 45.8 | 6.0 GB | ≈ 11 tok/s |
| 23 | DeepSeek-R1-Distill-Qwen-7B DeepSeek · 4-bit · MIT | 45.6 | 5.5 GB | ≈ 11 tok/s |
| 24 | LFM2.5 2.6B Liquid AI · 4-bit · LFM Open License | 44.7 | 2.3 GB | ≈ 31 tok/s |
| 25 | EXAONE 4.0 1.2B LG AI Research (EXAONE) · 4-bit · EXAONE License (non-commercial) | 42.5 | 1.6 GB | ≈ 66 tok/s |
Change the context length or quantization, or see models that only just fit, in Can I run it? for this device.
How to run Nanbeige4.2 3B on a Jetson Orin Nano
- Install Ollama. On JetPack 6, Ollama runs on the Jetson GPU with CUDA. NVIDIA's Jetson AI Lab also has containers for llama.cpp, MLC and vLLM.
curl -fsSL https://ollama.com/install.sh | sh
- Run Nanbeige4.2 3B. Downloads the 4-bit GGUF and opens a chat.
ollama run hf.co/bartowski/Nanbeige_Nanbeige4.2-3B-GGUF:Q4_K_M
- Use the fastest power mode. Pick the MAXN SUPER power mode (from the power menu or with nvpmodel), then lock the clocks for the best speed.
sudo jetson_clocks
Every model page lists the same commands for its own files. Nanbeige4.2 3B's page also has a memory table for other context lengths.
Questions
What is the best LLM for a Jetson Orin Nano?
As of November 2026, the best model to run on a Jetson Orin Nano is Nanbeige4.2 3B by Nanbeige (Kanzhun) (Small LM Index 67.2), at about 20 tokens/s with 4-bit weights. On the Jetson Orin NX 16 GB, Ternary Bonsai 2 27B (71.2) fits.
What is the biggest model that runs on a Jetson Orin Nano?
Qwen3.5 9B (9B parameters) fits at 4-bit, needing 6.3 GB and generating about 9 tokens/s. Bigger models may fit at lower quality or run more slowly; try them in the Can I run it? tool.
How much memory can a model use on a Jetson Orin Nano?
About 6.5 GB of the 8 GB. The Orin Nano Super has 8 GB shared between the system and the GPU, at 102 GB/s: six times the bandwidth of a Raspberry Pi, so 3–4B models run at chat speed.
How accurate are these speeds?
They are estimates from memory bandwidth (102 GB/s here), which limits how fast a model generates text, with a typical efficiency for this kind of hardware. Expect real speeds within about a third either way. Long prompts take longer to read before the first word appears.