tinyanalysis

Compare / LFM2.5 24B-A2B vs Devstral Small 2

LFM2.5 24B-A2B vs Devstral Small 2

Updated November 2026

Verdict

LFM2.5 24B-A2B scores higher on the Small LM Index: 63.1 vs 37.3. On the 5 benchmarks both labs report, LFM2.5 24B-A2B wins 4 and Devstral Small 2 wins 1. Both need about 16.9 GB of memory at 4-bit quantization. Only LFM2.5 24B-A2B has a hosted API on OpenRouter ($0.21 per million tokens); the other has to be self-hosted.

Pick LFM2.5 24B-A2B if…

  • you want the strongest model overall
  • you need precise instruction following
  • you want a hosted API rather than running it yourself

Pick Devstral Small 2 if…

  • you work with long documents
  • you need a permissive license (Apache-2.0)
  • you need image input
  • you want the more widely used model (more quantized builds and community support)

Specifications

LFM2.5 24B-A2BDevstral Small 2
Small LM Index63.1 (likely 56–69)37.3 (likely 34–40)
LabLiquid AI · United StatesMistral AI · Europe
Released2026-10-132025-12-09
Parameters24B (2B active, MoE)24B (dense)
Memory at 4-bit~15.7 GB~16.9 GB
Context window128K tokens256K tokens
Inputtexttext, image
ReasoningNo thinkingNo thinking
LicenseLFM Open LicenseApache-2.0
API price / 1M tokens$0.21 ($0.050 in, $0.70 out)Self-host only
Hugging Face downloads (30 days)34.3K123.7K

Benchmarks

Higher is better. Bold marks the winner where both labs report a score. ≈ marks the index model's estimate for a score that wasn't reported, ✓ an independent measurement and * a score measured by another lab.

BenchmarkLFM2.5 24B-A2BDevstral Small 2
MMLU-Pro Knowledge≈ 8167.8✓
GPQA Diamond Knowledge74.453.2✓
Humanity's Last Exam Reasoning7.43.5✓
AIME Math≈ 8734.3✓
LiveCodeBench Coding≈ 7334.8✓
SWE-bench Verified Coding51.068.0
Terminal-Bench Agentic—22.5*
τ²-Bench Agentic—23.4✓
IFBench Instruction following74.731.2✓
IFEval Instruction following96.8—
BFCL Agentic65.0—
MATH-500 Math96.0—
AA-LCR Long context—28.0✓
SciCode Coding32.732.4✓

Scores are mostly reported by each lab with its own settings, so small gaps are noise. See the methodology. Open in the interactive comparison to add more models.

Questions

Which is better, LFM2.5 24B-A2B or Devstral Small 2?

LFM2.5 24B-A2B scores higher on the Small LM Index: 63.1 vs 37.3. On the 5 benchmarks both labs report, LFM2.5 24B-A2B wins 4 and Devstral Small 2 wins 1.

Which is cheaper to run?

Only LFM2.5 24B-A2B has a hosted API on OpenRouter ($0.21 per million tokens); the other has to be self-hosted. Both need about 16.9 GB of memory at 4-bit quantization.

Which has the longer context window?

Devstral Small 2 has the longer context window: 256K tokens against 128K.

Can I use LFM2.5 24B-A2B and Devstral Small 2 commercially?

LFM2.5 24B-A2B is released under LFM Open License and Devstral Small 2 under Apache-2.0. Apache 2.0 and MIT allow commercial use; other licenses can add conditions, so read the license before shipping a product.

Related comparisons

Guides featuring these models