Qwen3.8-Flash-Next: The 180B Model With 6B Active That Streams Its Table From SSD

01 What It Is

Released August 26, 2026 as open weights, with a 28-page technical report. It plays the role Qwen3-Next played for Qwen3.5: the architecture ships early, before the full Qwen4 family is built on it. In llama.cpp the architecture is literally named qwen4exp.

Multimodal (text + image + video in, text out), built from three parameter populations:

• 125B main model — 48-layer MoE backbone, 6B active per token
• 51B N-gram embedding table — a second memory layer, held off the accelerator
• ~4B MTP head — multi-token prediction for built-in speculative decoding

Specs

source: qwen official model card & technical report, august 2026

02 The Core Idea: N-gram Embedding

Every transformer looks each token up in an embedding table — but the key is the token alone. "Bank" gets the same vector in river bank as in bank account; disambiguation is left to the layers above. Flash-Next adds a second lookup: at each position it also keys on the 2- and 3-token sequences ending there (bigrams, trigrams) and adds the retrieved vector to the token's representation.

• A lookup, not a matmul — negligible per-token FLOPs
• Deterministic — the keys for the next tokens are known from the token stream, so rows can be prefetched
• Sparse — ~16 lookups per token; a few dozen rows out of 20M at any instant

What the ablations show

• Placement: single-layer wins, Layer 2 chosen — it lets prefetching overlap with Layer 1's compute
• Capacity: growing the table (20V→200V) lowers loss monotonically while downstream accuracy saturates — the report's headline finding that loss and accuracy don't always move together
• vs MoE budget: swapping experts for N-gram slots gains nothing — the two "play distinct roles in scaling capacity"

03 Streaming It From SSD

Official design: the table lives in host RAM, never VRAM. Only the rows the current context needs are transferred, asynchronously — the fetch for the next rows starts while the GPU computes the current token. No community patch required.

Community implementations have pushed it one level further, from RAM to NVMe:

The measurement that proves the point

source: sglang-ssd-stream public benchmarks, matched 1,024-token completions · independent community measurement

Removing the table from RAM did not make it the decode bottleneck — SSD was invisible to the GPU. And it's not just the table: only the 12 QSA layers keep a KV cache; the 36 DeltaNet layers hold constant-size recurrent state. Measured KV footprint: ~6 GiB even at ~250K-token context. Long context stops being a memory wall and becomes a latency problem.

04 Benchmarks

Qwen's official comparison set. All vendor-reported — no independent replication published as of September 2026. Best value in each row marked.

qwen official model card · harness: claude code / mini-swe-agent, 256k context · full 21-row set incl. qwen3.8-27b, hle, lvbench, osworld 2.0 in the model card

Flash-Next beats its dense 27B sibling on all nine shared benchmarks (avg +4.2), with the biggest gap in agentic coding — DeepSWE 42.2 → 58.7. The honest read: 397B-A17B-class coding at 6B active, on hardware that costs a few thousand dollars.

05 Running It Locally

Memory by quant

unsloth quant docs · kld: ~80% top-1% accuracy recovery at 79% less disk vs bf16

Measured speeds — Strix Halo, 128 GB (abliter8-ai)

median-of-5 decode, single amd ryzen ai max+ 395, tok/s

The spread is the story. MTP (built-in speculative decoding) only helps when output is predictable: acceptance 0.93 for structured, 0.88 for code, 0.59 for prose. Tuned, the box nearly quadrupled on code and bought almost nothing on prose. Agentic coding on a 128 GB mini PC gets RTX-class throughput; essays, plan on ~22 tok/s.

Deployment traps

1. The Ollama library tag is Mac-only (MLX engine). On CUDA, use llama.cpp directly.
2. llama.cpp must post-date Aug 27, 2026 — older builds die with unknown model architecture: 'qwen4exp'.
3. CUDA grid-dimension overflow at full native 262,144 — practical ceiling is 261,888, set --ctx-size accordingly.
4. QSA is approximate by design — output can diverge slightly on ~3% of positions past the 8K sparse-attention budget.

06 The API (Qwen3.8-Flash)

Don't confuse the names: Qwen3.8-Flash on QwenCloud is the production model built on Flash-Next — 1M context by default, official built-in tools. Flash-Next is the open-weight preview. Pricing (QwenCloud, mirrored on OpenRouter):

07 The Bottom Line

• 96–128 GB unified memory: the best local open model that tier can run today — designed for it
• 24 GB GPU: it doesn't fit, even at 1-bit (75 GB floor). Keep Qwen3.8-27B on those cards
• CPU tower: 128 GB DDR5 + fast NVMe is now a legitimate frontier-inference build
• API users: $0.15/$0.47 per million with 1M context — for prose, renting still wins
• Researchers: the report is unusually candid about failures — watch what Qwen4 ships, this is the preview