For the first time, a 27-billion-parameter language model with full reasoning, tool-calling, and multimodal capabilities runs interactively on a standard laptop. That sounds impossible by any conventional measure — until you understand what PrismML has done to the model's weights.

Released July 14, 2026, Bonsai 27B is the latest (and largest) member of PrismML's Bonsai family, built on Qwen3.6-27B and compressed end-to-end using ternary weights. The result: a 27B-class model that occupies less storage than a full-precision 2B model, while retaining 95% of the original's intelligence.

This is not a pruning experiment or a knowledge-distillation hack. Every single weight in the language network — embeddings, attention projections, MLPs, and the LM head — is quantized to one of three values: {−1, 0, +1}. No high-precision escape hatches. No mixed-precision tricks hiding behind a low-bit label.

The Bonsai Family: A Progressive Compression Revolution

PrismML has been quietly building toward this release for over a year. The Bonsai family represents a deliberate, step-by-step expansion of what's possible at extreme compression ratios:

Phase 1: 1-bit Bonsai (March 31, 2026)

PrismML emerged from stealth on March 31, 2026, with 1-bit Bonsai — a family of binary-weight models in 8B, 4B, and 1.7B sizes. The flagship 1-bit Bonsai 8B fit in just 1.15 GB while matching its full-precision baseline's benchmark performance. Backed by $16.25M from Khosla Ventures, Cerberus, and Google, the Caltech-born team had demonstrated that binary weights ({−1, +1}) could produce commercially useful language models.

Phase 2: Ternary Bonsai (April 2026)

Just weeks later, PrismML introduced the Ternary Bonsai family (8B, 4B, 1.7B), adding a zero state to create three weight values {−1, 0, +1}. The extra state — seemingly trivial — delivered a ~5-point average improvement across benchmarks for only ~600MB of additional memory. The zero state proved critical: it gives the model a "neutral" weight option, preserving more of the original weight distribution than binary's forced bipolar quantization.

Phase 3: Bonsai 27B (July 14, 2026)

Now, the family reaches its largest member. Bonsai 27B scales the ternary technique to a 27B-parameter model — a capability tier that previously required 54GB in FP16 or at minimum 17.6GB in the best conventional 4-bit build. Bonsai 27B delivers the same tier in 7.2GB (ternary) or 3.9GB (1-bit).

How Ternary Quantization Works

The core innovation of Bonsai 27B isn't the model architecture — it's what PrismML does to the weights. To understand why this matters, we need to look at the mechanics.

The Ternary Weight Alphabet

Traditional quantization methods approximate continuous FP16 weights with values from a larger set (e.g., 4-bit quantization uses 16 discrete levels per weight). Bonsai's ternary approach reduces every weight to one of three values:

{−1, 0, +1}

But simply snapping weights to three values would destroy the model. PrismML's solution: group-wise FP16 scaling. Each group of 128 ternary weights shares one FP16 scale factor. The actual mathematical operation becomes:

actual_weight = ternary_value × group_scale_factor

This means the effective weight range is not limited to [−1, +1] — the scale factor adapts per group, allowing the model to preserve important magnitude information across different weight channels. The ternary values act as a directional sign (+1 pushes, −1 pulls back, 0 stays neutral), while the FP16 scale captures the intensity.

Why 1.71 Bits Per Weight?

A binary value (0 or 1) carries exactly 1 bit. A ternary value (−1, 0, or +1) carries log₂(3) ≈ 1.585 bits. Since each group of 128 ternary weights also stores one 16-bit FP16 scale factor, the total storage per weight is:

(1.585 bits × 128 weights + 16 bits scale) / 128 weights = ~1.71 bits/weight

This is a true average — unlike conventional "2-bit" builds that advertise lower bits per weight while actually storing 2.8 bits/weight when you count the full overhead.

End-to-End: No Escape Hatches

What makes Bonsai different from other quantization approaches is the end-to-end coverage. Every component of the language network is ternary:

• Embedding layer
• Attention projections (all 64 layers)
• MLP projections (SwiGLU gates)
• LM head (vocabulary projection)

No layer is left in FP16 or higher precision. The vision tower ships separately in compact 4-bit HQQ format (0.63 GB), loaded only when an image is provided — text-only serving never pays for it.

Architecture & Design

Bonsai 27B is derived from Qwen3.6-27B, a hybrid-attention causal language model. The architecture itself is unchanged — what's different is every weight inside it.

Hybrid Attention Backbone

Qwen3.6-27B uses a hybrid attention pattern: approximately 75% linear attention and 25% full attention. This design choice has a profound impact on Bonsai's practical usability:

• Linear attention layers grow context linearly in memory, making the 262K-token context window practical on consumer hardware
• Only 16 of 64 layers use full attention, keeping the KV cache manageable (~4.3GB at full context in FP16, ~1.1GB at 4-bit compression)
• The hybrid design enables long-document analysis, full-repository code work, and other context-heavy tasks on-device

Model Composition

Component	Parameters	Format
🌳 Language backbone	~24.8B	Ternary {−1,0,+1} @ 1.71 bpw
Embeddings + LM head	~2.5B	Ternary {−1,0,+1} @ 1.71 bpw
Vision tower	~0.46B	HQQ 4-bit (optional)
Total	~27.8B	—

Additional Design Elements

• SwiGLU MLP — gated activation function for improved expressiveness
• RoPE — Rotary Positional Embeddings for positional encoding
• RMSNorm — Root Mean Square Layer Normalization
• Thinking mode — enabled by default, toggleable per request via thinking_budget_tokens

Benchmark Performance

Bonsai 27B was evaluated across 15 benchmarks in six skill categories, all in thinking mode where the model's full reasoning capability is exercised. Here's how it compares to the FP16 baseline and conventional quantization methods:

By Capability

Category	Benchmarks	Qwen3.6-27B FP16	Ternary Bonsai 27B	1-bit Bonsai 27B
🧮 Math	GSM8K, MATH-500, AIME25, AIME26	95.3	93.4	91.7
💻 Coding	HumanEval+, MBPP+, LiveCodeBench	88.7	86.0	81.9
🤖 Agentic / Tool-calling	BFCL v3, TauBench	80.0	74.0	66.0
📋 Instruction Following	IFEval, IFBench	78.4	71.8	65.8
📚 Knowledge / STEM	MMLU-Redux, MuSR	83.1	77.0	73.4
👁️ Vision	MMMU Pro, OCRBench	72.6	65.2	59.6
Overall (15)	—	85.0	80.5	76.1

The Story in the Numbers

The averages tell a clean story, but the breakdown reveals something more important: math and coding are nearly untouched. Math scores are within two points of full precision (93.4 vs 95.3), and coding at 86.0 is a small drop that's barely noticeable in practice. These are exactly the capabilities that matter most for agentic workloads — multi-step reasoning, structured tool use, and code generation.

The gap concentrates in the most demanding categories: agentic tool calling (74.0 vs 80.0) and vision (65.2 vs 72.6). This is not a failure mode — it's the expected shape of compression loss, and it's dramatically better than what conventional methods do.

Benchmark Comparison: Same Footprint, Different Methods

For context, here's how Bonsai 27B stacks against conventional quantization methods at similar sizes:

Variant	True bpw	Footprint	Thinking avg	vs FP16
Qwen3.6-27B FP16	16.0	54 GB	85.07	100%
Qwen3.6-27B Q4_K_XL ("4-bit")	5.2	17.6 GB	84.99	99.9%
Qwen3.6-27B IQ2_XXS ("2-bit")	2.8	9.4 GB	72.73	85.5%
🌳 Ternary Bonsai 27B	1.71	5.9 GB	80.49	94.6%
1-bit Bonsai 27B	1.125	3.9 GB	76.11	89.5%

At 5.9GB, Ternary Bonsai 27B outscores both sub-4-bit conventional builds by more than seven points at one-half to two-thirds of their size. The conventional builds may score higher in aggregate, but their failure mode is qualitatively worse: they maintain surface-level competence while losing reasoning depth.

Intelligence Density: The New Metric

PrismML introduced a new metric with Bonsai 8B — intelligence density — which captures the ratio of capability to deployed size:

D = −log₂(1 − score/100) / size_GB

Variant	Size (GB)	Benchmark avg	Density (1/GB)
1-bit Bonsai 27B	3.9	76.11	0.530
🌳 Ternary Bonsai 27B	5.9	80.49	0.400
Qwen3.6-27B IQ2_XXS	9.4	72.73	0.199
Qwen3.6-27B Q4_K_XL	17.6	84.99	0.155
Qwen3.6-27B FP16	54	85.07	0.051

Ternary Bonsai 27B delivers 2× the density of the densest conventional build (IQ2_XXS at 0.199) and nearly 8× the density of FP16. No conventional build of Qwen3.6-27B or Gemma-4-31B exceeds 0.2. Every stored gigabyte translates into far more usable intelligence.

Real-World Performance

Throughput

Token generation throughput (tg128) was measured over 128 generated tokens, capturing the memory-bandwidth-bound interactive phase:

Platform	Footprint	Generate (tok/s)	Process 512 (tok/s)
M5 Max laptop (Metal)	7.2 GB	44.0	830
M5 Pro laptop (Metal) ⚡	7.2 GB	26.2	393
M4 Pro laptop (Metal)	7.2 GB	18.0	125
H100 GPU (CUDA)	7.2 GB	98.0	2,596
100K context (M5 Pro, 4-bit KV)	10.1 GB	~22	~350

Peak Memory at Context

What a device must actually accommodate is peak memory — weights plus KV cache plus activations (~1.3GB overhead):

Build	Weights	4K ctx	10K ctx	100K ctx
🌳 Ternary Bonsai	7.15 GB	8.4 GB	8.7 GB	14.7 GB
Qwen3.6-27B Q4_K_XL	17.6 GB	19.2 GB	19.6 GB	25.6 GB
Qwen3.6-27B FP16	51.25 GB	52.6 GB	53.3 GB	59.3 GB

The ternary build holds a 100K-token context at 14.7GB without KV-cache compression — fitting mainstream laptops outright. With 4-bit KV cache, the 100K peak drops to ~10.1GB, and the full 262K window fits in ~12.8GB.

DSpark Speculative Decoding

Bonsai 27B ships with a DSpark drafter layer — a semi-autoregressive 6-layer transformer trained against the low-bit target. It uses hidden states tapped from five evenly spaced layers of the target model, with a diffusion-flavored block-denoising objective.

On CUDA (H100), the drafter achieves a 1.34× decode speedup: 98 → 131.8 tok/s at an accepted draft length of τ ≈ 3.7. On Apple Silicon, the batch-1 verification pass does not yet amortize, so the drafter is not enabled by default on-device.

The Bigger Picture: Why This Matters

Bonsai 27B isn't just a smaller model — it enables entirely new deployment patterns that were impossible before.

On-Device Agentic AI

When a model capable of sustained agentic work fits on the device, the marginal cost of a hundred-step loop is zero, and user data never leaves the machine. This unlocks persistent on-device agents, offline assistants, and assistants that reason over private local data by construction.

Hybrid Deployments

The cost-quality spectrum now has a practical local operating point: non-frontier and privacy-sensitive tasks route to Bonsai locally, while frontier cloud models handle the hardest steps. This collapses the cost-per-task of agentic systems dramatically.

Phone-Class AI

The 1-bit variant (3.9GB) fits within the iPhone 17 Pro's memory budget — about 6GB available for on-device apps. It's the first 27B-class model to run on a phone, bringing multi-step reasoning and tool use to mobile devices.

Apache 2.0: Fully Open

Everything is available under the Apache 2.0 license — weights, kernels, demo code. Bonsai 27B runs natively on Apple devices via MLX and NVIDIA GPUs via CUDA, through custom low-bit kernels built for its hybrid-attention architecture. It's also landing in mainline llama.cpp (Q1_0 fully merged, Q2_0 on CPU and Metal, Vulkan in review).

Use Cases

Limitations & Trade-offs

An honest article needs to be clear about what Bonsai 27B doesn't do:

• Doesn't fit a phone (ternary): At ~7.2GB, the ternary build exceeds the ~6GB per-app iOS memory budget. Use the 1-bit companion via MLX Swift for phone deployment.
• Served in 2-bit slots today: The deployed footprint (~7.2GB) sits above the representation's ~5.9GB native target. Native ternary kernels are an active engineering target.
• Agentic coding not yet tuned: Long-horizon, multi-file, run-test-and-repair workflows are not a strong target of this release. A Bonsai 27B variant tuned for agentic coding is next on the roadmap.
• Vision at 4-bit: The vision tower is quantized separately at 4-bit HQQ — it works well but doesn't benefit from the ternary compression that defines the language network.

Getting Started

Bonsai 27B runs through llama.cpp (CUDA, Metal, CPU) and MLX (Apple Silicon). Here's a quick start for CUDA:

# Clone the PrismML fork (includes Q2_0_g128 hybrid-attention kernels)
git clone https://github.com/PrismML-Eng/llama.cpp
cd llama.cpp

# Build with CUDA support
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

# Download the 2-bit GGUF weights
hf download prism-ml/Ternary-Bonsai-27B-gguf \
  Ternary-Bonsai-27B-Q2_0.gguf --local-dir .

# Run inference
./build/bin/llama-cli \
    -m Ternary-Bonsai-27B-Q2_0.gguf \
    -p "Explain quantum computing in simple terms." \
    -n 256 \
    --temp 0.7 --top-p 0.95 --top-k 20 \
    -ngl 99

For the OpenAI-compatible server:

./build/bin/llama-server \
    -m Ternary-Bonsai-27B-Q2_0.gguf \
    --host 0.0.0.0 --port 8080 -ngl 99

Generation parameters (used for all reported benchmarks): temperature 0.7, top-p 0.95, top-k 20.

Available formats:

Format	HF Repository	Size
🌳 Ternary GGUF (Q2_0)	Ternary-Bonsai-27B-gguf	7.17 GB
Ternary MLX (2-bit)	Ternary-Bonsai-27B-mlx-2bit	7.05 GB
1-bit GGUF (Q1_0)	Bonsai-27B-gguf	3.53 GB
1-bit MLX (1-bit)	Bonsai-27B-mlx-1bit	3.92 GB

Each repo ships an mmproj file for the vision tower (+0.63 GB), needed for image input.

ZVHH Scorecard

Criteria	Rating
🧮 Reasoning / Math	★★★★★ 93.4
💻 Coding	★★★★★ 86.0
🤖 Agentic / Tool Calling	★★★★★ 74.0
📚 Knowledge	★★★★★ 77.0
👁️ Vision	★★★★★ 65.2
💎 Intelligence Density	★★★★★ 0.400/GB
📜 Openness	★★★★★ Apache 2.0