Skip to main content
Ternary Bonsai 2 27B is the largest multimodal model in the family: it accepts vision input alongside text. It is based on Qwen3.8 27B, a hybrid-attention causal language model, and its matrix weights are ternary — values in {−1, 0, +1} in a fixed rotated basis, with FP16 group scales.
Before you run it: the recommended sampling settings differ between thinking and instruct mode, and low reasoning effort is not supported. See Best practices.

​
Specifications

On the 20-benchmark thinking-mode suite it averages 83.9, retaining 98.2% of the Qwen3.8-27B FP16 baseline’s 85.4 at roughly a ninth of the size.

​
Artifacts

All Ternary Bonsai 2 repositories are in the Bonsai 2 collection on Hugging Face. The demo downloads PQ2_0 by default; see choosing a packing below. The GGUF repo also ships the vision tower separately, needed for image input: mmproj HQQ 4-bit (0.63 GB, what the demo uses) or the BF16 reference (0.93 GB).
Both GGUF packings need the PrismML llama.cpp fork (prism-b10658 or newer) — the rotated weight basis requires a runtime Walsh–Hadamard transform that is not upstream yet. On a stock build PQ2_0 and PTQ1_0 are refused, and a Q2_0 file loads silently and outputs gibberish. The MLX pack runs on stock MLX.

​
How to run it

Through the demo repo (Ternary Bonsai 2 27B is the default, so no flags needed):
Or a single generation directly with llama.cpp / MLX:
The demo repo is the source of truth for running these models — tested setups and current implementation details live there.

​
Best practices

These apply to both the GGUF and MLX releases.

​
Generation parameters

Use the set that matches the mode you’re running in — thinking and instruct mode want different sampling settings, and mixing them degrades output quality.
min_p = 0.05 drops tokens far less likely than the top choice. In our tests it scored at least as well as 0.0 and followed instructions more reliably. It is also llama.cpp’s default: the GGUF carries top_k, top_p and temperature but not min_p, so llama.cpp applies 0.05 without being told. Other runtimes may default to 0.0, so set it explicitly there. Presence and repetition penalties, and all instruct-mode values, are not in the GGUF; pass them yourself. The published benchmarks used the thinking-mode values with min_p = 0.0. These match the base model’s generation config and are embedded in the GGUF metadata, so clients that read model defaults apply them automatically. The published benchmarks use the thinking-mode values.

​
Reasoning effort

The model uses xhigh by default. medium is about as accurate at roughly a third fewer tokens, and is a good default when responses feel slow. low is not supported — it does not reduce thinking. To bound reasoning, use a token budget instead (see Thinking mode).

​
System prompt

A plain system message works well:

​
Choosing a packing

Pick by hardware target and whether footprint or decode speed matters more.

​
Vision and tool calling

27B is trained for both. Send an image as a normal OpenAI-style image_url content part, and a tools array for function calling — both come back through the standard API, no prompt hacks required:
The response’s choices[0].message.tool_calls carries the call. See Tool calling & MCP in the demo repo for MCP server setup and the full agentic walkthrough, and Open WebUI for a point-and-click demo of the same capabilities.

​
Thinking mode

27B reasons by default. Toggle it per request with thinking_budget_tokens (0 to disable, -1 for unlimited), cap it server-wide with --reasoning-budget N, or turn it off entirely with BONSAI_THINKING=0 ./scripts/start_llama_server.sh. The built-in chat UI has a Reasoning effort picker (Off / Low / Medium / High / Max) that overrides the server default per conversation.
Large images are downscaled to ~1,024 vision tokens by default on Metal, Vulkan, and CPU to keep latency reasonable; CUDA and ROCm run uncapped. Override with BONSAI_IMAGE_MAX_TOKENS (0 disables the cap — useful for reading small text in screenshots).