Best LLMs for edge devices
Updated November 2026
The strongest open models that run on phones, Raspberry Pi, NVIDIA Jetson and laptops without a graphics card. On small hardware, memory speed limits you more than size, so each pick must also generate at least 5 tokens per second (about reading speed). Figures assume 4-bit weights and a 4K-token context. For other settings, use Can I run it?
Phone, 8 GB
About 4.0 GB usable · 60 GB/s · 29 models run at 5+ tokens/s
- Best quality · Best with Apache 2.0 or MITNanbeige4.2 3B 67.2Q4_K_M · 3.5 GB · ≈ 9 tok/s · Apache-2.0
- Fast (20+ tok/s)LFM3 1.7B 61.2Q4_K_M · 1.6 GB · ≈ 23 tok/s · LFM Open License
Phone, 12 GB
About 6.0 GB usable · 77 GB/s · 38 models run at 5+ tokens/s
- Best quality · Best with Apache 2.0 or MITNanbeige4.2 3B 67.2Q4_K_M · 3.5 GB · ≈ 12 tok/s · Apache-2.0
- Fast (20+ tok/s)LFM3 1.7B 61.2Q4_K_M · 1.6 GB · ≈ 29 tok/s · LFM Open License
Raspberry Pi 5
About 6.5 GB usable · 17 GB/s · 20 models run at 5+ tokens/s
- Best qualityLFM3 1.7B 61.2Q4_K_M · 1.6 GB · ≈ 8 tok/s · LFM Open License
- Best with Apache 2.0 or MITMiniCPM5 2B 60.9Q4_K_M · 2.3 GB · ≈ 6 tok/s · Apache-2.0
Jetson Orin Nano Super, 8 GB
About 6.5 GB usable · 102 GB/s · 45 models run at 5+ tokens/s
- Best quality · Fast (20+ tok/s) · Best with Apache 2.0 or MITNanbeige4.2 3B 67.2Q4_K_M · 3.5 GB · ≈ 20 tok/s · Apache-2.0
Laptop or mini PC without a GPU, 16 GB
About 11 GB usable · 51 GB/s · 48 models run at 5+ tokens/s
- Best quality · Best with Apache 2.0 or MITNanbeige4.2 3B 67.2Q4_K_M · 3.5 GB · ≈ 10 tok/s · Apache-2.0
- Fast (20+ tok/s)LFM3 1.7B 61.2Q4_K_M · 1.6 GB · ≈ 24 tok/s · LFM Open License
Jetson AGX Orin, 64 GB
About 63 GB usable · 205 GB/s · 99 models run at 5+ tokens/s
- Best quality · Best with Apache 2.0 or MITQwen3.8 27B 74.6Q4_K_M · 18 GB · ≈ 6 tok/s · Apache-2.0
- Fast (20+ tok/s)Kolibri-1 71.1Q4_K_M · 50 GB · ≈ 24 tok/s · Apache-2.0
Quality is the Small LM Index. The Raspberry Pi 5 has the same speed with 8 or 16 GB; the extra memory only fits bigger, slower models. Phones leave about half of their memory to apps; boards keep about 1.5 GB for the system. Memory includes the context cache and runtime buffers.
Smallest model for each level of quality
Sorted by memory. Each model here scores higher than every model that needs less memory, so if you have a memory budget, the best choice is the last row that fits.
| Model | Index | Memory |
|---|---|---|
| Gemma 3 270MGoogle · 270M · Gemma | 12.8 | ~0.7 GB |
| LFM2.5 350MLiquid AI · 350M · LFM Open License | 16.6 | ~0.8 GB |
| MiniCPM5 1BOpenBMB (MiniCPM) · 1.1B · Apache-2.0 | 29.4 | ~1.3 GB |
| LFM2.5 1.2B ThinkingLiquid AI · 1.2B · LFM Open License | 34.7 | ~1.3 GB |
| EXAONE 4.0 1.2BLG AI Research (EXAONE) · 1.3B · EXAONE License (non-commercial) | 42.5 | ~1.6 GB |
| LFM3 1.7BLiquid AI · 1.7B · LFM Open License | 61.2 | ~1.6 GB |
| Spark-X2.6 4BSparkLLM (XHToken) · 4.1B · Apache-2.0 | 66.5 | ~3.3 GB |
| Nanbeige4.2 3BNanbeige (Kanzhun) · 4.2B · Apache-2.0 | 67.2 | ~3.5 GB |
| Ternary Bonsai 2 27BPrismML · 27.4B · Apache-2.0 | 71.2 | ~9.0 GB (Q2_0) |
Guides by device
How to run them
- Phones. Apps such as PocketPal AI (iOS and Android) load GGUF files from Hugging Face; Google AI Edge Gallery runs Gemma and other LiteRT models. For your own app: llama.cpp, MLX Swift on iPhone, or LiteRT-LM and ExecuTorch on Android.
- Raspberry Pi and other ARM boards. llama.cpp or Ollama on the CPU. Use the 4-bit GGUF; 8-bit roughly halves the speed.
- NVIDIA Jetson. llama.cpp and Ollama with CUDA, or MLC LLM and vLLM from NVIDIA's Jetson AI Lab containers.
- Laptops without a GPU. Ollama, LM Studio or llama.cpp. Mixture-of-Experts models with few active parameters run surprisingly fast on a CPU if you have the memory.
Every model page lists the exact Ollama, llama.cpp, LM Studio and MLX commands, using the GGUF and MLX files listed above.
Questions
What is the best LLM to run on a phone?
On a phone with 8 GB of memory, Nanbeige4.2 3B (Small LM Index 67.2) is the strongest model that fits at 4-bit and still generates about 9 tokens/s. A phone lets one app use only about half of its memory, so an 8 GB phone runs models of up to about 4 GB.
Can a Raspberry Pi 5 run an LLM?
Yes, on its CPU with llama.cpp or Ollama. Its memory is slow (about 17 GB/s), which limits speed more than size: LFM3 1.7B is the best scoring model that still reaches about 8 tokens/s. Bigger models fit, especially on the 16 GB board, but generate only a few words per second.
What is the best LLM for a Jetson Orin Nano?
Nanbeige4.2 3B (index 67.2), at about 20 tokens/s with 4-bit weights. The Orin Nano Super's 8 GB is shared with the system, which leaves about 6.5 GB for a model.
What is the best model under 2 GB or 4 GB of memory?
Under 2 GB: LFM3 1.7B (index 61.2, about 1.6 GB). Under 4 GB: Nanbeige4.2 3B (index 67.2, about 3.5 GB). Both at 4-bit with a 4K-token context.
How accurate are the speed estimates?
They estimate generation speed from memory bandwidth, which is what limits small devices, with a typical efficiency for each kind of hardware. Expect real speeds within about a third either way. Reading a long prompt is much slower on phones and CPUs, which have little compute, and phones slow down further when they get hot.