Can I run it? / RTX 5090 (32 GB)
Best LLMs to run on an RTX 5090 (32 GB)
Updated November 2026
As of November 2026, the best model to run on an RTX 5090 (32 GB) is Qwen3.8 27B by Alibaba (Qwen) (Small LM Index 74.6), at about 44 tokens/s with 8-bit weights.
The 5090 has 32 GB and nearly twice the bandwidth of a 4090, so 30B-class models fit with long contexts and generate quickly. About 31 GB is usable for a model, at 1792 GB/s. The figures below assume the best quality that fits (8-bit down to 4-bit) and a 8K-token context, and only include models that generate at least 5 tokens per second.
Models that run well on an RTX 5090 (32 GB)
The 25 best of the 97 models that run at 5+ tokens/s, ranked by the Small LM Index.
| # | Model | Index | Memory | Speed |
|---|---|---|---|---|
| 1 | Qwen3.8 27B Alibaba (Qwen) · 8-bit · Apache-2.0 | 74.6 | 30 GB | ≈ 44 tok/s |
| 2 | Ternary Bonsai 2 27B PrismML · Q2_0 (as released) · Apache-2.0 | 71.2 | 9.3 GB | ≈ 163 tok/s |
| 3 | Nanbeige4.3 33B Nanbeige (Kanzhun) · 6-bit · Apache-2.0 | 70.9 | 30 GB | ≈ 46 tok/s |
| 4 | Muse Glimmer 30B Meta · 6-bit · Apache-2.0 | 70.1 | 26 GB | ≈ 52 tok/s |
| 5 | Qwen3.6 27B Alibaba (Qwen) · 8-bit · Apache-2.0 | 69.7 | 30 GB | ≈ 44 tok/s |
| 6 | Qwen3.6 35B-A3B Alibaba (Qwen) · 6-bit · Apache-2.0 | 68.3 | 30 GB | ≈ 255 tok/s |
| 7 | Qwen3.5 27B Alibaba (Qwen) · 8-bit · Apache-2.0 | 68.3 | 30 GB | ≈ 44 tok/s |
| 8 | Nanbeige4.2 3B Nanbeige (Kanzhun) · 8-bit · Apache-2.0 | 67.2 | 5.8 GB | ≈ 283 tok/s |
| 9 | Qwen3.5 35B-A3B Alibaba (Qwen) · 6-bit · Apache-2.0 | 66.9 | 30 GB | ≈ 255 tok/s |
| 10 | Spark-X2.6 4B SparkLLM (XHToken) · 8-bit · Apache-2.0 | 66.5 | 5.3 GB | ≈ 287 tok/s |
| 11 | Gemma 4 31B Google · 6-bit · Apache-2.0 | 66.2 | 28 GB | ≈ 50 tok/s |
| 12 | GLM-4.7-Flash Z.ai (GLM) · 6-bit · MIT | 66.0 | 27 GB | ≈ 255 tok/s |
| 13 | Spark-X2.5 4B SparkLLM (XHToken) · 8-bit · Apache-2.0 | 65.9 | 5.3 GB | ≈ 287 tok/s |
| 14 | Granite 4.2 30B IBM (Granite) · 6-bit · Apache-2.0 | 65.8 | 28 GB | ≈ 51 tok/s |
| 15 | Qwen3 30B-A3B Thinking 2507 Alibaba (Qwen) · 6-bit · Apache-2.0 | 65.4 | 27 GB | ≈ 232 tok/s |
| 16 | Gemma 4 26B-A4B Google · 8-bit · Apache-2.0 | 64.8 | 28 GB | ≈ 155 tok/s |
| 17 | North Mini Code Cohere · 6-bit · Unverified | 64.7 | 28 GB | ≈ 255 tok/s |
| 18 | gpt-oss-20b OpenAI · 8-bit · Apache-2.0 | 64.7 | 23 GB | ≈ 164 tok/s |
| 19 | Nemotron 3 Nano 30B-A3B NVIDIA · 6-bit · NVIDIA Open Model | 64.4 | 27 GB | ≈ 219 tok/s |
| 20 | Qwen3.5 9B Alibaba (Qwen) · 8-bit · Apache-2.0 | 64.1 | 11 GB | ≈ 131 tok/s |
| 21 | Nemotron 3.5 Lightning NVIDIA · 6-bit · OpenMDW-1.1 | 63.3 | 29 GB | ≈ 212 tok/s |
| 22 | Apriel 1.6 15B Thinker ServiceNow (Apriel) · 8-bit · MIT | 63.3 | 18 GB | ≈ 79 tok/s |
| 23 | LFM2.5 24B-A2B Liquid AI · 8-bit · LFM Open License | 63.1 | 27 GB | ≈ 295 tok/s |
| 24 | Edge0 35B-A3B (preview) Edge0 · 6-bit · Apache-2.0 | 63.0 | 30 GB | ≈ 255 tok/s |
| 25 | Sarvam 30B Sarvam AI · 6-bit · Apache-2.0 | 62.8 | 28 GB | ≈ 319 tok/s |
Change the context length or quantization, or see models that only just fit, in Can I run it? for this device.
How to run Qwen3.8 27B on an RTX 5090 (32 GB)
- Easiest: Ollama or LM Studio. Both use the GPU automatically on Windows and Linux.
ollama run hf.co/ggml-org/Qwen3.8-27B-GGUF:Q4_K_M
- With llama.cpp. Starts a local server with a chat window at http://localhost:8080.
llama-server -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M
Every model page lists the same commands for its own files. Qwen3.8 27B's page also has a memory table for other context lengths.
Questions
What is the best LLM for an RTX 5090 (32 GB)?
As of November 2026, the best model to run on an RTX 5090 (32 GB) is Qwen3.8 27B by Alibaba (Qwen) (Small LM Index 74.6), at about 44 tokens/s with 8-bit weights.
What is the biggest model that runs on an RTX 5090 (32 GB)?
Kimi Linear 48B-A3B (48B parameters, 3B active) fits at 4-bit, needing 31 GB and generating about 345 tokens/s. Bigger models may fit at lower quality or run more slowly; try them in the Can I run it? tool.
How much memory can a model use on an RTX 5090 (32 GB)?
About 31 GB of the 32 GB. The 5090 has 32 GB and nearly twice the bandwidth of a 4090, so 30B-class models fit with long contexts and generate quickly.
How accurate are these speeds?
They are estimates from memory bandwidth (1792 GB/s here), which limits how fast a model generates text, with a typical efficiency for this kind of hardware. Expect real speeds within about a third either way. Long prompts take longer to read before the first word appears.