Can I run it? / RTX 3060 (12 GB)
Best LLMs to run on an RTX 3060 (12 GB)
Updated November 2026
As of November 2026, the best model to run on an RTX 3060 (12 GB) is Ternary Bonsai 2 27B by PrismML (Small LM Index 71.2), at about 33 tokens/s with Q2_0 (as released) weights.
The 12 GB RTX 3060 is the cheapest card with room for 12–14B models at 4-bit, and a popular first GPU for local models. About 11 GB is usable for a model, at 360 GB/s. The figures below assume the best quality that fits (8-bit down to 4-bit) and a 8K-token context, and only include models that generate at least 5 tokens per second.
Models that run well on an RTX 3060 (12 GB)
The 25 best of the 59 models that run at 5+ tokens/s, ranked by the Small LM Index.
| # | Model | Index | Memory | Speed |
|---|---|---|---|---|
| 1 | Ternary Bonsai 2 27B PrismML · Q2_0 (as released) · Apache-2.0 | 71.2 | 9.3 GB | ≈ 33 tok/s |
| 2 | Nanbeige4.2 3B Nanbeige (Kanzhun) · 8-bit · Apache-2.0 | 67.2 | 5.8 GB | ≈ 57 tok/s |
| 3 | Spark-X2.6 4B SparkLLM (XHToken) · 8-bit · Apache-2.0 | 66.5 | 5.3 GB | ≈ 58 tok/s |
| 4 | Spark-X2.5 4B SparkLLM (XHToken) · 8-bit · Apache-2.0 | 65.9 | 5.3 GB | ≈ 58 tok/s |
| 5 | Qwen3.5 9B Alibaba (Qwen) · 8-bit · Apache-2.0 | 64.1 | 11 GB | ≈ 26 tok/s |
| 6 | Granite 4.2 8B IBM (Granite) · 8-bit · Apache-2.0 | 62.7 | 11 GB | ≈ 30 tok/s |
| 7 | Ministral 3 14B Mistral AI · 4-bit · Apache-2.0 | 61.6 | 11 GB | ≈ 30 tok/s |
| 8 | LFM3 1.7B Liquid AI · 8-bit · LFM Open License | 61.2 | 2.5 GB | ≈ 137 tok/s |
| 9 | MiniCPM5 2B OpenBMB (MiniCPM) · 8-bit · Apache-2.0 | 60.9 | 3.6 GB | ≈ 94 tok/s |
| 10 | Mellum2.1 12B-A2.5B Thinking JetBrains (Mellum) · 6-bit · Apache-2.0 | 60.8 | 11 GB | ≈ 61 tok/s |
| 11 | Qwen3 4B Thinking 2507 Alibaba (Qwen) · 8-bit · Apache-2.0 | 59.8 | 6.0 GB | ≈ 59 tok/s |
| 12 | Qwen3.5 4B Alibaba (Qwen) · 8-bit · Apache-2.0 | 59.7 | 5.1 GB | ≈ 59 tok/s |
| 13 | Nemotron Labs Dory 1.0 4B NVIDIA · 8-bit · OpenMDW-1.1 | 58.6 | 5.2 GB | ≈ 60 tok/s |
| 14 | Falcon H1R 7B TII (Falcon) · 8-bit · Falcon LLM License | 58.6 | 9.1 GB | ≈ 31 tok/s |
| 15 | Gemma 4 12B Google · 5-bit · Apache-2.0 | 58.3 | 10 GB | ≈ 30 tok/s |
| 16 | Ministral 3 8B Mistral AI · 8-bit · Apache-2.0 | 57.7 | 10 GB | ≈ 30 tok/s |
| 17 | Granite 4.2 3B IBM (Granite) · 8-bit · Apache-2.0 | 57.3 | 4.4 GB | ≈ 79 tok/s |
| 18 | Phi-4-reasoning-plus Microsoft (Phi) · 4-bit · MIT | 57.2 | 11 GB | ≈ 28 tok/s |
| 19 | MiniCPM-SALA OpenBMB (MiniCPM) · 8-bit · Apache-2.0 | 56.9 | 11 GB | ≈ 25 tok/s |
| 20 | DeepSeek-R1-0528-Qwen3-8B DeepSeek · 8-bit · MIT | 56.3 | 11 GB | ≈ 29 tok/s |
| 21 | Hunyuan 7B Instruct Tencent (Hunyuan) · 8-bit · Tencent Hunyuan Community | 55.8 | 9.7 GB | ≈ 32 tok/s |
| 22 | Ling 3.0 Tiny InclusionAI (Ling) · 8-bit · MIT | 54.9 | 9.3 GB | ≈ 85 tok/s |
| 23 | Qwen3 8B Alibaba (Qwen) · 8-bit · Apache-2.0 | 53.9 | 11 GB | ≈ 29 tok/s |
| 24 | Ministral 3 3B Mistral AI · 8-bit · Apache-2.0 | 53.8 | 4.6 GB | ≈ 79 tok/s |
| 25 | Step3-VL-10B StepFun · 6-bit · Apache-2.0 | 53.2 | 10 GB | ≈ 31 tok/s |
Change the context length or quantization, or see models that only just fit, in Can I run it? for this device.
How to run Ternary Bonsai 2 27B on an RTX 3060 (12 GB)
- Easiest: Ollama or LM Studio. Both use the GPU automatically on Windows and Linux.
- With llama.cpp. Starts a local server with a chat window at http://localhost:8080.
llama-server -hf prism-ml/Ternary-Bonsai-2-27B-gguf:Q2_0
Every model page lists the same commands for its own files. Ternary Bonsai 2 27B's page also has a memory table for other context lengths.
Questions
What is the best LLM for an RTX 3060 (12 GB)?
As of November 2026, the best model to run on an RTX 3060 (12 GB) is Ternary Bonsai 2 27B by PrismML (Small LM Index 71.2), at about 33 tokens/s with Q2_0 (as released) weights.
What is the biggest model that runs on an RTX 3060 (12 GB)?
Ternary Bonsai 2 27B (27.4B parameters) fits at Q2_0 (as released), needing 9.3 GB and generating about 33 tokens/s. Bigger models may fit at lower quality or run more slowly; try them in the Can I run it? tool.
How much memory can a model use on an RTX 3060 (12 GB)?
About 11 GB of the 12 GB. The 12 GB RTX 3060 is the cheapest card with room for 12–14B models at 4-bit, and a popular first GPU for local models.
How accurate are these speeds?
They are estimates from memory bandwidth (360 GB/s here), which limits how fast a model generates text, with a typical efficiency for this kind of hardware. Expect real speeds within about a third either way. Long prompts take longer to read before the first word appears.