Can I run it? / RTX 4060 (8 GB)
Best LLMs to run on an RTX 4060 (8 GB)
Updated November 2026
As of November 2026, the best model to run on an RTX 4060 (8 GB) is Nanbeige4.2 3B by Nanbeige (Kanzhun) (Small LM Index 67.2), at about 43 tokens/s with 8-bit weights.
An 8 GB card fits models up to about 8B at 4-bit. Everything that fits runs fast; memory, not speed, is the limit. About 7.4 GB is usable for a model, at 272 GB/s. The figures below assume the best quality that fits (8-bit down to 4-bit) and a 8K-token context, and only include models that generate at least 5 tokens per second.
Models that run well on an RTX 4060 (8 GB)
The 25 best of the 47 models that run at 5+ tokens/s, ranked by the Small LM Index.
| # | Model | Index | Memory | Speed |
|---|---|---|---|---|
| 1 | Nanbeige4.2 3B Nanbeige (Kanzhun) · 8-bit · Apache-2.0 | 67.2 | 5.8 GB | ≈ 43 tok/s |
| 2 | Spark-X2.6 4B SparkLLM (XHToken) · 8-bit · Apache-2.0 | 66.5 | 5.3 GB | ≈ 44 tok/s |
| 3 | Spark-X2.5 4B SparkLLM (XHToken) · 8-bit · Apache-2.0 | 65.9 | 5.3 GB | ≈ 44 tok/s |
| 4 | Qwen3.5 9B Alibaba (Qwen) · 5-bit · Apache-2.0 | 64.1 | 7.3 GB | ≈ 30 tok/s |
| 5 | Granite 4.2 8B IBM (Granite) · 4-bit · Apache-2.0 | 62.7 | 6.9 GB | ≈ 39 tok/s |
| 6 | LFM3 1.7B Liquid AI · 8-bit · LFM Open License | 61.2 | 2.5 GB | ≈ 104 tok/s |
| 7 | MiniCPM5 2B OpenBMB (MiniCPM) · 8-bit · Apache-2.0 | 60.9 | 3.6 GB | ≈ 71 tok/s |
| 8 | Qwen3 4B Thinking 2507 Alibaba (Qwen) · 8-bit · Apache-2.0 | 59.8 | 6.0 GB | ≈ 45 tok/s |
| 9 | Qwen3.5 4B Alibaba (Qwen) · 8-bit · Apache-2.0 | 59.7 | 5.1 GB | ≈ 45 tok/s |
| 10 | Nemotron Labs Dory 1.0 4B NVIDIA · 8-bit · OpenMDW-1.1 | 58.6 | 5.2 GB | ≈ 45 tok/s |
| 11 | Falcon H1R 7B TII (Falcon) · 6-bit · Falcon LLM License | 58.6 | 7.2 GB | ≈ 31 tok/s |
| 12 | Ministral 3 8B Mistral AI · 4-bit · Apache-2.0 | 57.7 | 6.7 GB | ≈ 39 tok/s |
| 13 | Granite 4.2 3B IBM (Granite) · 8-bit · Apache-2.0 | 57.3 | 4.4 GB | ≈ 60 tok/s |
| 14 | MiniCPM-SALA OpenBMB (MiniCPM) · 4-bit · Apache-2.0 | 56.9 | 6.7 GB | ≈ 33 tok/s |
| 15 | DeepSeek-R1-0528-Qwen3-8B DeepSeek · 4-bit · MIT | 56.3 | 6.8 GB | ≈ 38 tok/s |
| 16 | Hunyuan 7B Instruct Tencent (Hunyuan) · 5-bit · Tencent Hunyuan Community | 55.8 | 7.1 GB | ≈ 36 tok/s |
| 17 | Ling 3.0 Tiny InclusionAI (Ling) · 6-bit · MIT | 54.9 | 7.4 GB | ≈ 83 tok/s |
| 18 | Qwen3 8B Alibaba (Qwen) · 4-bit · Apache-2.0 | 53.9 | 6.8 GB | ≈ 38 tok/s |
| 19 | Ministral 3 3B Mistral AI · 8-bit · Apache-2.0 | 53.8 | 4.6 GB | ≈ 60 tok/s |
| 20 | Nemotron Nano 9B v2 NVIDIA · 5-bit · NVIDIA Open Model | 52.9 | 7.1 GB | ≈ 30 tok/s |
| 21 | Spark-X2.6 1.7B SparkLLM (XHToken) · 8-bit · Apache-2.0 | 52.3 | 2.5 GB | ≈ 105 tok/s |
| 22 | Edge0 8B-A1B (preview) Edge0 · 6-bit · Apache-2.0 | 50.1 | 7.4 GB | ≈ 83 tok/s |
| 23 | MiMo-7B-RL Xiaomi (MiMo) · 4-bit · MIT | 45.8 | 6.6 GB | ≈ 40 tok/s |
| 24 | DeepSeek-R1-Distill-Qwen-7B DeepSeek · 6-bit · MIT | 45.6 | 7.4 GB | ≈ 31 tok/s |
| 25 | LFM2.5 2.6B Liquid AI · 8-bit · LFM Open License | 44.7 | 3.6 GB | ≈ 66 tok/s |
Change the context length or quantization, or see models that only just fit, in Can I run it? for this device.
How to run Nanbeige4.2 3B on an RTX 4060 (8 GB)
- Easiest: Ollama or LM Studio. Both use the GPU automatically on Windows and Linux.
ollama run hf.co/bartowski/Nanbeige_Nanbeige4.2-3B-GGUF:Q4_K_M
- With llama.cpp. Starts a local server with a chat window at http://localhost:8080.
llama-server -hf bartowski/Nanbeige_Nanbeige4.2-3B-GGUF:Q4_K_M
Every model page lists the same commands for its own files. Nanbeige4.2 3B's page also has a memory table for other context lengths.
Questions
What is the best LLM for an RTX 4060 (8 GB)?
As of November 2026, the best model to run on an RTX 4060 (8 GB) is Nanbeige4.2 3B by Nanbeige (Kanzhun) (Small LM Index 67.2), at about 43 tokens/s with 8-bit weights.
What is the biggest model that runs on an RTX 4060 (8 GB)?
MiniCPM-SALA (9.5B parameters) fits at 4-bit, needing 6.7 GB and generating about 33 tokens/s. Bigger models may fit at lower quality or run more slowly; try them in the Can I run it? tool.
How much memory can a model use on an RTX 4060 (8 GB)?
About 7.4 GB of the 8 GB. An 8 GB card fits models up to about 8B at 4-bit. Everything that fits runs fast; memory, not speed, is the limit.
How accurate are these speeds?
They are estimates from memory bandwidth (272 GB/s here), which limits how fast a model generates text, with a typical efficiency for this kind of hardware. Expect real speeds within about a third either way. Long prompts take longer to read before the first word appears.