01 What Actually Happened
On August 26, 2026, Alibaba's Qwen team released Qwen3.8-Flash-Next as open weights, with a GitHub repository and a 28-page technical report. The report frames it explicitly as an architectural preview — the same role Qwen3-Next played as the lead-in to the Qwen3.5 family.
In other words, the architecture is being shipped early and in the open, before the full model family that will be built on top of it. That's unusual and deliberate: it lets the community read the design, verify it, and start building tooling while the actual Qwen4 models are still in training. The tell that this is really the Qwen4 base is hard to miss — in llama.cpp, the architecture is literally registered under the name qwen4exp.
02 The Four Axes the Report Commits To
The technical report states the model is "upgraded systematically along four aspects — attention, residual, embedding and optimization." Each is a concrete, named design, not a hand-wave. These are the four pillars the Qwen4 family will stand on.
Linear attention with sparse escape hatches
36 of 48 layers use Gated DeltaNet — linear attention that compresses history into a fixed-size state at linear cost. Every 4th layer uses Qwen Sparse Attention (QSA), which scores context at micro-block granularity via a compressed MQA indexer. At 1M context, QSA's attention module is reported 7.6× faster in prefill and 4.9× faster in decode than dense attention at the kernel level. Consequence: only the 12 QSA layers keep a KV cache — measured KV footprint is ~6 GiB even at ~250K tokens.
The residual stream, split and gated
The residual stream is widened into four branches, read through an element-wise, data-dependent gate and written back through per-branch scalar gates — finer-grained information flow, with low reported inference overhead.
The headline: a 51B lookup, not a matmul
A 51B-parameter N-gram embedding table — 20M slots for bigrams and trigrams, placed at Layer 2. It's a lookup, not computation: negligible per-token FLOPs, deterministic and sparsely accessed, so it can live in host RAM or stream from an NVMe SSD without becoming the decode bottleneck. This is the genuinely new idea and the reason the preview can be run at all on 96–128 GB machines.
A training recipe that changes the recipe
Muon for the linear-map weights (attention projections, GDN projections, expert fc1/fc2, N-gram k/v projections) and AdamW for embeddings, LM head, and router. The report notes this combination shifts the optimal learning rate and batch size upward enough that batch-size warmup becomes unnecessary, cutting total optimizer steps.
03 The Proof It's Real: Three Independent Signals
"Qwen4" is easy to overclaim because the word keeps surfacing. Three independent, verifiable signals confirm the architecture is real and imminent — separate from each other, so none can be dismissed as a single source's error.
The open-weight preview
Qwen3.8-Flash-Next weights + 28-page technical report published. The report's own framing: an architecture preview for the coming Qwen4 family, with the four design axes fully documented.
llama.cpp ships qwen4exp (PR #27742)
The architecture is registered and runnable in the mainstream inference stack the day after the release — under the internal name qwen4exp. N-gram-table offload to SSD via mmap is included. You can run the preview today.
The API stepping-stone: Qwen3.8-Flash
On QwenCloud, the production model Qwen3.8-Flash is built directly on Flash-Next — 1M context by default with official built-in tools, priced at $0.15 / $0.47 per million tokens. A production model on the Qwen4 architecture is already serving traffic, even as the open Qwen4 family isn't out.
04 Confirmed vs. Still Missing
This is the whole article in one screen. Left column: what a source document actually says. Right column: what does not exist yet — and what to treat as noise.
✓ Confirmed & verifiable
- The Qwen4 architecture is published (28-page tech report)
- Four design axes: GDN+QSA attention, gated residual, 51B N-gram embedding, Muon+AdamW
- Runnable today in llama.cpp as
qwen4exp(PR #27742, merged Aug 27) - A production model on this architecture (Qwen3.8-Flash) is live on QwenCloud
- Preview model spec: ~180B logical / 6B active / 51B N-gram table, 262K native context
- A "Qwen 4" entry exists in Qwen's own site navigation (a teaser)
✕ Not yet published
- No Qwen4 model card, weights, or checkpoint (zero
Qwen/Qwen4*on Hugging Face) - No Qwen4 technical report or benchmarks
- No confirmed Qwen4 model sizes, context, or modality set
- No release date or "coming X" claim from Qwen
- NVIDIA single-GPU offload benchmarks for the preview — unverified
- Any "Qwen4 leaked benchmark" post online — unverified, treat as noise
05 Why "Preview First" Changes the Race
Shipping the architecture before the model does three things at once. First, it moves the competitive debate from "who benchmarks highest" to "who's design is sound" — and you can audit a design on paper. Second, it front-loads ecosystem work: llama.cpp, Unsloth, and SGLang builds land on the day of the preview, so the Qwen4 family won't have a tooling gap when it drops. Third, it de-risks the announcement — the community has already proven the hard parts (like streaming a 51B table from SSD) work, so the launch is about the model, not the plumbing.
The honest read: by the time Qwen4 is announced, the interesting engineering will have been settled months earlier, in the open. That's a rare and aggressive move.
06 What to Watch
Concrete signals that the Qwen4 family is close. Any of these is stronger evidence than a teaser nav item:
- A
Qwen/Qwen4*model card or weights appearing on Hugging Face — the definitive trigger - A dated Qwen4 post behind the existing site teaser
- A Qwen4 technical report (distinct from the Flash-Next preview report)
- Independent benchmark runs by third parties (Artificial Analysis, etc.)
- Official size/context disclosure for the family
this article updates when any of these fire — the status pill above reflects it
07 The Bottom Line
Qwen4 is the rare announcement that's already half-happened. The blueprint is public, peer-auditable, and runnable; the model is not. If you're betting on what Qwen4 will be, the best public evidence is the Flash-Next technical report — four named axes, a genuinely new 51B embedding design, and a training recipe that removes warmup. Everything else is speculation until a model card lands.
qwen4exp, merged Aug 27, 2026) · QwenCloud model page (Qwen3.8-Flash) · qwen.ai site navigation · Hugging Face Qwen org model listing (verified Sep 22, 2026).
This article was drafted with AI assistance and reviewed by a human editor.