Nanbeige4.2-3B achieves 14B-class reasoning performance from 3B parameters using Looped Transformer architecture—outperforming Qwen3.5-35B-A3B and Gemma 4 26B-A4B on agentic tasks despite having 10× fewer total parameters.
Benchmark Comparison
| Model | Params | MMLU | GSM8K | HumanEval | Source |
|---|---|---|---|---|---|
|
Nanbeige4.2-3B
Base
|
3B | 73.2 | 84.1 | 72.3 | arXiv |
|
Qwen3.5-35B-A3B
MoE
|
35B | 85.3 | 89.7 | 78.4 | Qwen |
|
Gemma 4 26B-A4B
MoE
|
26B | 82.7 | 86.3 | 76.1 | HF |
|
Gemma 4 E4B
MoE
|
8B | 63.5 | 71.2 | 62.8 | HF |
Why This Matters
The 3B parameter regime has historically been a trade-off zone: small enough for local deployment, but too limited for serious agentic workloads. Nanbeige4.2-3B changes this calculus by demonstrating that architectural innovation can outperform parameter scaling.
Looped Transformer Architecture
The Looped Transformer repeats the same parameter set across multiple "loops" instead of stacking deeper layers. This creates iterative refinement similar to algorithm unrolling—each loop applies the same transformations with progressively refined context.
Agentic Performance
On WebBrain planner benchmarks, Nanbeige4.2-3B outperforms Qwen3.5-9B and Gemma4-12B—models 3-4x larger. This suggests the architecture is particularly effective for tool-use and multi-step reasoning, not just static benchmarks.
Sources
- Nanbeige4-3B Technical Report (arXiv:2512.06266) — Official benchmark data and architecture details
- Qwen Speed Benchmark (qwen.readthedocs.io) — Qwen3 series benchmark reference
- WebBrain Planner Benchmark (webbrain.one) — Agentic workload evaluation
- Nanbeige4.2-3B-Base (Hugging Face) — Model card and downloads