# Qwen3.8-Flash-Next: The 180B Model With 6B Active That Streams Its Table From SSD

Alibaba's Qwen3.8-Flash-Next (Aug 26, 2026) is the first open-weight Qwen4 preview: 125B MoE + a 51B N-gram embedding table, 6B active per token. The table lives in RAM or on an NVMe SSD — and decode doesn't slow down.

## 01 What It Is

Released August 26, 2026 as open weights, with a 28-page technical report. It plays the role Qwen3-Next played for Qwen3.5: the architecture ships early, before the full Qwen4 family is built on it. In llama.cpp the architecture is literally named `qwen4exp`.

Multimodal (text + image + video in, text out), built from three parameter populations:

- **125B main model** — 48-layer MoE backbone, 6B active per token

- **51B N-gram embedding table** — a second memory layer, held off the accelerator

- **~4B MTP head** — multi-token prediction for built-in speculative decoding

The efficiency framing
Versus its 397B-A17B predecessor (Qwen3.7-Plus): leads 8 of 14 pre-training benchmarks, trails the rest by ≤2.6 points — while activating ~3× fewer parameters per token, training on ~⅓ the tokens, and using ~1/9 the training FLOPs.

### Specs

| **Spec** | **Value**|
--- | ---
| Layers | 48 — 12 × (3× Gated DeltaNet + 1× Qwen Sparse Attention)|
| MoE experts | 512 total; 10 routed + 1 shared active|
| N-gram table | 20M slots (bigrams + trigrams) at Layer 2 = 51B params|
| Hidden dim / vocab | 2,560 / 248,320 (padded)|
| Context | 262,144 native; 1,000,000 via YaRN|
| Thinking | Hybrid, on by default (xhigh/medium/low/none)|
| License | Qwen Community License 1.0 (not Apache 2.0)|

source: qwen official model card & technical report, august 2026

## 02 The Core Idea: N-gram Embedding

Every transformer looks each token up in an embedding table — but the key is the token alone. "Bank" gets the same vector in *river bank* as in *bank account*; disambiguation is left to the layers above. Flash-Next adds a second lookup: at each position it also keys on the 2- and 3-token sequences ending there (bigrams, trigrams) and adds the retrieved vector to the token's representation.

- **A lookup, not a matmul** — negligible per-token FLOPs

- **Deterministic** — the keys for the next tokens are known from the token stream, so rows can be prefetched

- **Sparse** — ~16 lookups per token; a few dozen rows out of 20M at any instant

Why it matters
Because access is deterministic and sparse, the table is offloadable by construction. Keep it in host RAM or on NVMe and prefetch the rows ahead of demand — a 160-byte read from a fast SSD, overlapped with GPU compute, costs effectively nothing. That's how a third of a frontier model's weights can sit on disk without becoming the bottleneck.

### What the ablations show

- **Placement:** single-layer wins, Layer 2 chosen — it lets prefetching overlap with Layer 1's compute

- **Capacity:** growing the table (20V→200V) lowers loss monotonically while downstream accuracy saturates — the report's headline finding that loss and accuracy don't always move together

- **vs MoE budget:** swapping experts for N-gram slots gains nothing — the two "play distinct roles in scaling capacity"

## 03 Streaming It From SSD

**Official design:** the table lives in host RAM, never VRAM. Only the rows the current context needs are transferred, asynchronously — the fetch for the next rows starts while the GPU computes the current token. No community patch required.

**Community implementations** have pushed it one level further, from RAM to NVMe:

| **Implementation** | **Streams** | **How**|
--- | --- | ---
| sglang-ssd-stream (Apache-2.0) | 51.2B FP8 table (47.68 GiB) | Rows to a read-only SSD sidecar file → io_uring concurrent reads → FP8 conversion on a side CUDA stream. 32 MiB page pool, no multi-GB cache|
| llama.cpp PR #27742 (merged Aug 27) | Engram table in GGUFs | Row lookups served via mmap from the model file on disk|
| slotstream (Swift/MLX, MIT) | MoE experts | 105 GB MLX 4-bit on a 48 GB M5 Pro via expert lookahead — 15.86 tok/s measured|

### The measurement that proves the point

| **RTX PRO 6000 (96 GB), NVFP4 build** | **Result**|
--- | ---
| Table in RAM | 47.68 GiB occupied · 148.5–156.2 tok/s|
| Table on SSD (sglang-ssd-stream) | ~64 MiB working RAM · 164.7 tok/s · 47.6 GiB RAM returned|
| Content-heavy (mostly unseen rows) | 126–137 tok/s|

source: sglang-ssd-stream public benchmarks, matched 1,024-token completions · independent community measurement

Removing the table from RAM did **not** make it the decode bottleneck — SSD was invisible to the GPU. And it's not just the table: only the 12 QSA layers keep a KV cache; the 36 DeltaNet layers hold constant-size recurrent state. Measured KV footprint: **~6 GiB even at ~250K-token context**. Long context stops being a memory wall and becomes a latency problem.

Net effect for local hardware
With 6B active per token, the bandwidth math that makes RAM inference slow elsewhere barely applies — Unsloth notes CPU+RAM vs GPU+VRAM "may make relatively little difference" for this model. A 128 GB DDR5 machine with fast NVMe, no large GPU, is a legitimate deployment. The honest trade: prefill is a clock problem — a Strix Halo box handled a 201,647-token prompt (KV fit in ~6 GiB) in 17.3 minutes to first token.

## 04 Benchmarks

Qwen's official comparison set. All vendor-reported — no independent replication published as of September 2026. Best value in each row marked.

| **Task** | **Benchmark** | **Flash-Next** | **Qwen3.7-Plus** | **DeepSeek-V4-Flash** | **Opus 4.6**|
--- | --- | --- | --- | --- | ---
| Coding | DeepSWE 1.1 | 58.7 | 16.5 | 54.4 | —|
| SWE-bench Pro | 62.5 | 55.8 | 56.0 | 53.4|
| SWE-bench Multilingual | 81.0 | 75.8 | — | 77.5|
| Agentic | CoWorkBench | 73.9 | 65.1 | 45.1 | 68.2|
| JobBench | 55.7 | 27.6 | 41.3 | 36.6|
| General | GPQA Diamond | 91.7 | 90.3 | 90.8 | 91.3|
| LiveCodeBench v6 | 91.9 | 89.6 | 90.6 | 88.8|
| Vision | AndroidWorld | 84.5 | 81.0 | — | 62.0|
| MathVision | 95.7 | 88.7 | — | 65.5|

qwen official model card · harness: claude code / mini-swe-agent, 256k context · full 21-row set incl. qwen3.8-27b, hle, lvbench, osworld 2.0 in the model card

Flash-Next beats its dense 27B sibling on all nine shared benchmarks (avg +4.2), with the biggest gap in agentic coding — DeepSWE 42.2 → 58.7. The honest read: 397B-A17B-class coding at 6B active, on hardware that costs a few thousand dollars.

## 05 Running It Locally

### Memory by quant

| **Quant** | **Total RAM+VRAM** | **Notes**|
--- | --- | ---
| 1-bit | 75 GB | Smallest. N-gram layer stays 4-bit — random-access tables degrade under heavy quant|
| 2-bit | 79 GB | |
| 4-bit | 96–114 GB | 92.9 GB download; ~54.5 GB must stay resident, table streams|
| 8-bit | 200 GB | |
| BF16 | 355 GB | |

unsloth quant docs · kld: ~80% top-1% accuracy recovery at 79% less disk vs bf16

### Measured speeds — Strix Halo, 128 GB (abliter8-ai)

| **llama.cpp build** | **Prose** | **Code** | **Structured**|
--- | --- | --- | ---
| Reference HIP, no speculation | 22.1 | 21.9 | 22.1|
| Vulkan + MTP | 22.3 | 30.0 | 33.9|
| Tuned HIP + MTP (q8_0 KV) | 21.9 | 66.0 | 55.3|
| Tuned HIP + MTP (f16 KV) | 23.1 | 82.0 | 41.7|

median-of-5 decode, single amd ryzen ai max+ 395, tok/s

The spread is the story. MTP (built-in speculative decoding) only helps when output is predictable: acceptance 0.93 for structured, 0.88 for code, 0.59 for prose. Tuned, the box nearly quadrupled on code and bought almost nothing on prose. Agentic coding on a 128 GB mini PC gets RTX-class throughput; essays, plan on ~22 tok/s.

### Deployment traps

1. The Ollama library tag is Mac-only (MLX engine). On CUDA, use llama.cpp directly.

1. llama.cpp must post-date Aug 27, 2026 — older builds die with `unknown model architecture: 'qwen4exp'`.

1. CUDA grid-dimension overflow at full native 262,144 — practical ceiling is 261,888, set `--ctx-size` accordingly.

1. QSA is approximate by design — output can diverge slightly on ~3% of positions past the 8K sparse-attention budget.

## 06 The API (Qwen3.8-Flash)

Don't confuse the names: **Qwen3.8-Flash** on QwenCloud is the production model built on Flash-Next — 1M context by default, official built-in tools. Flash-Next is the open-weight preview. Pricing (QwenCloud, mirrored on OpenRouter):

**$0.15***Input / 1M tokens*

**$0.47***Output / 1M tokens*

**$0.016***Cached input / 1M*

**991K***Max input (131K output)*

## 07 The Bottom Line

- **96–128 GB unified memory:** the best local open model that tier can run today — designed for it

- **24 GB GPU:** it doesn't fit, even at 1-bit (75 GB floor). Keep Qwen3.8-27B on those cards

- **CPU tower:** 128 GB DDR5 + fast NVMe is now a legitimate frontier-inference build

- **API users:** $0.15/$0.47 per million with 1M context — for prose, renting still wins

- **Researchers:** the report is unusually candid about failures — watch what Qwen4 ships, this is the preview

Caveats
Benchmark scores are from Qwen's own model card — no independent replication as of September 20, 2026. SSD-streaming (sglang-ssd-stream), Strix Halo tok/s (abliter8-ai), quant tables (Unsloth) and slotstream numbers are independent community measurements. NVIDIA single-GPU (RTX 3090/4090/5090) offload benchmarks were unverified at research time — treat any Flash-Next-on-RTX figure as such.

## ↺ Related Articles

[Qwen3.8: Alibaba's 2.4T Open-Weight Bet — What We Actually Know](/qwen3-8-alibaba-2-4t-open-weight-frontier-model.html)
[Qwen3.8 vs Kimi K3: The 2026 Chinese AI Model Showdown](/qwen3-8-vs-kimi-k3.html)
[GLM-5.2: The New Open-Source LLM That's Top of the AI Leaderboard](/glm-5-2-new-open-source-llm-beats-claude-opus-frontier.html)
[Tencent HY3: Open-Source MoE That Beats Flagship Models](/tencent-hy3-open-source-moe-llm-beats-flagship.html)

**Sources:** Qwen official model card (Hugging Face) · Qwen3.8-Flash-Next technical report (Aug 2026) · QwenLM GitHub · Unsloth documentation · sglang-ssd-stream public benchmarks · slotstream · llama.cpp PR #27742 · QwenCloud model page · abliter8-ai Strix Halo deployment · RunAIHome hardware guide.

This article was drafted with AI assistance and reviewed by a human editor.

ZVHH — AI Tools & Software Reviews © 2026
[Home](/)[Agent feed](/index.md)