# Qwen4: The Model Is Missing — But the Blueprint Just Got Published Qwen3.8-Flash-Next (Aug 26, 2026) is an open-weight preview of the Qwen4 architecture. Everything the 28-page technical report confirms about what's coming — and a hard line between confirmed facts and unverified rumor. ## 01 What Actually Happened On **August 26, 2026**, Alibaba's Qwen team released **Qwen3.8-Flash-Next** as open weights, with a GitHub repository and a 28-page technical report. The report frames it explicitly as an architectural preview — the same role Qwen3-Next played as the lead-in to the Qwen3.5 family. In other words, the *architecture* is being shipped early and in the open, before the full model family that will be built on top of it. That's unusual and deliberate: it lets the community read the design, verify it, and start building tooling while the actual Qwen4 models are still in training. The tell that this is really the Qwen4 base is hard to miss — in `llama.cpp`, the architecture is literally registered under the name `qwen4exp`. The precedent Qwen has run this exact playbook before: publish the "Next" architecture preview, then land the full family on top. The preview model is the proof the design works; the family is the product. Flash-Next is the proof. Qwen4 is the product — not yet out. ## 02 The Four Axes the Report Commits To The technical report states the model is "upgraded systematically along four aspects — attention, residual, embedding and optimization." Each is a concrete, named design, not a hand-wave. These are the four pillars the Qwen4 family will stand on. A Attention · GDN + QSA #### Linear attention with sparse escape hatches 36 of 48 layers use **Gated DeltaNet** — linear attention that compresses history into a fixed-size state at linear cost. Every 4th layer uses **Qwen Sparse Attention (QSA)**, which scores context at micro-block granularity via a compressed MQA indexer. At 1M context, QSA's attention module is reported **7.6× faster in prefill and 4.9× faster in decode** than dense attention at the kernel level. Consequence: only the 12 QSA layers keep a KV cache — measured KV footprint is ~6 GiB even at ~250K tokens. R Residual · Gated Residual #### The residual stream, split and gated The residual stream is widened into **four branches**, read through an element-wise, data-dependent gate and written back through per-branch scalar gates — finer-grained information flow, with low reported inference overhead. N Embedding · N-gram table #### The headline: a 51B lookup, not a matmul A **51B-parameter N-gram embedding table** — 20M slots for bigrams and trigrams, placed at Layer 2. It's a lookup, not computation: negligible per-token FLOPs, deterministic and sparsely accessed, so it can live in host RAM or stream from an NVMe SSD without becoming the decode bottleneck. This is the genuinely new idea and the reason the preview can be *run at all* on 96–128 GB machines. O Optimization · Muon + AdamW #### A training recipe that changes the recipe **Muon** for the linear-map weights (attention projections, GDN projections, expert fc1/fc2, N-gram k/v projections) and **AdamW** for embeddings, LM head, and router. The report notes this combination shifts the optimal learning rate and batch size upward enough that **batch-size warmup becomes unnecessary**, cutting total optimizer steps. ## 03 The Proof It's Real: Three Independent Signals "Qwen4" is easy to overclaim because the word keeps surfacing. Three independent, verifiable signals confirm the architecture is real and imminent — separate from each other, so none can be dismissed as a single source's error. AUG 26, 2026 #### The open-weight preview Qwen3.8-Flash-Next weights + 28-page technical report published. The report's own framing: an architecture preview for the coming Qwen4 family, with the four design axes fully documented. AUG 27, 2026 #### llama.cpp ships `qwen4exp` (PR #27742) The architecture is registered and runnable in the mainstream inference stack the day after the release — under the internal name `qwen4exp`. N-gram-table offload to SSD via mmap is included. You can run the preview today. AUG 26 → SEPT 2026 #### The API stepping-stone: Qwen3.8-Flash On QwenCloud, the production model **Qwen3.8-Flash** is built directly on Flash-Next — 1M context by default with official built-in tools, priced at $0.15 / $0.47 per million tokens. A production model on the Qwen4 architecture is already serving traffic, even as the open Qwen4 family isn't out. ## 04 Confirmed vs. Still Missing This is the whole article in one screen. Left column: what a source document actually says. Right column: what does not exist yet — and what to treat as noise. #### ✓ Confirmed & verifiable - The Qwen4 *architecture* is published (28-page tech report) - Four design axes: GDN+QSA attention, gated residual, 51B N-gram embedding, Muon+AdamW - Runnable today in llama.cpp as `qwen4exp` (PR #27742, merged Aug 27) - A production model on this architecture (Qwen3.8-Flash) is live on QwenCloud - Preview model spec: ~180B logical / 6B active / 51B N-gram table, 262K native context - A "Qwen 4" entry exists in Qwen's own site navigation (a teaser) #### ✕ Not yet published - No Qwen4 model card, weights, or checkpoint (zero `Qwen/Qwen4*` on Hugging Face) - No Qwen4 technical report or benchmarks - No confirmed Qwen4 model sizes, context, or modality set - No release date or "coming X" claim from Qwen - NVIDIA single-GPU offload benchmarks for the preview — unverified - Any "Qwen4 leaked benchmark" post online — unverified, treat as noise The trap to avoid The preview model's numbers (180B / 6B active / 51B) are specs for *Flash-Next*, not confirmed Qwen4 specs. The Qwen4 family can be larger, smaller, or structured differently. Until Qwen publishes the family, "Qwen4 is a 180B model" is an inference, not a fact. ## 05 Why "Preview First" Changes the Race Shipping the architecture before the model does three things at once. First, it moves the competitive debate from "who benchmarks highest" to "who's design is sound" — and you can audit a design on paper. Second, it front-loads ecosystem work: `llama.cpp`, Unsloth, and SGLang builds land on the day of the preview, so the Qwen4 family won't have a tooling gap when it drops. Third, it de-risks the announcement — the community has already proven the hard parts (like streaming a 51B table from SSD) work, so the launch is about the model, not the plumbing. The honest read: by the time Qwen4 is announced, the interesting engineering will have been settled months earlier, in the open. That's a rare and aggressive move. ## 06 What to Watch Concrete signals that the Qwen4 family is close. Any of these is stronger evidence than a teaser nav item: 1. A `Qwen/Qwen4*` model card or weights appearing on Hugging Face — the definitive trigger 1. A dated Qwen4 post behind the existing site teaser 1. A Qwen4 technical report (distinct from the Flash-Next preview report) 1. Independent benchmark runs by third parties (Artificial Analysis, etc.) 1. Official size/context disclosure for the family this article updates when any of these fire — the status pill above reflects it ## 07 The Bottom Line Qwen4 is the rare announcement that's already half-happened. The blueprint is public, peer-auditable, and runnable; the model is not. If you're betting on what Qwen4 will be, the best public evidence is the Flash-Next technical report — four named axes, a genuinely new 51B embedding design, and a training recipe that removes warmup. Everything else is speculation until a model card lands. Caveats This article is a status report as of September 22, 2026. All architecture facts are from the Qwen3.8-Flash-Next technical report and official model card (vendor-published). The Qwen3.8-Flash API is a separate, production artifact — its presence does not confirm Qwen4 specs. No Qwen4 benchmarks exist; any that appear in posts are unverified. ## ↺ Related Articles [Qwen3.8-Flash-Next: The 180B Model With 6B Active That Streams Its Table From SSD](/qwen3-8-flash-next-ngram-ssd-streaming.html) [Qwen3.8: Alibaba's 2.4T Open-Weight Bet — What We Actually Know](/qwen3-8-alibaba-2-4t-open-weight-frontier-model.html) [Qwen3.8 vs Kimi K3: The 2026 Chinese AI Model Showdown](/qwen3-8-vs-kimi-k3.html) [Kimi K3: Moonshot's 2.8T Open Frontier Intelligence Model](/kimi-k3-open-frontier-intelligence.html) **Sources:** Qwen3.8-Flash-Next technical report (Aug 2026) · Qwen official model card (Hugging Face) · QwenLM GitHub · llama.cpp PR #27742 (`qwen4exp`, merged Aug 27, 2026) · QwenCloud model page (Qwen3.8-Flash) · qwen.ai site navigation · Hugging Face Qwen org model listing (verified Sep 22, 2026). This article was drafted with AI assistance and reviewed by a human editor. ZVHH — AI Tools & Software Reviews © 2026 [Home](/)[Agent feed](/index.md)