Skip to main content
Ternary Bonsai 2 27B is the largest multimodal model in the family: it accepts vision input alongside text. It is based on Qwen3.8 27B, a hybrid-attention causal language model, and its matrix weights are ternary — values in {−1, 0, +1} in a fixed rotated basis, with FP16 group scales.

​
Specifications

On the 20-benchmark thinking-mode suite it averages 83.9, retaining 98.2% of the Qwen3.8-27B FP16 baseline’s 85.4 at roughly a ninth of the size. See benchmarks.

​
Artifacts

All Ternary Bonsai 2 repositories are in the Bonsai 2 collection on Hugging Face. PTQ1_0 has the smaller footprint; PQ2_0 uses a simpler 2-bit representation that is cheaper to unpack, so prompt processing is faster. Neither is uniformly faster. The demo downloads PQ2_0 by default. The GGUF repo also ships the vision tower separately, needed for image input: mmproj HQQ 4-bit (0.63 GB, what the demo uses) or the BF16 reference (0.93 GB).
Both GGUF packings need the PrismML llama.cpp fork (prism-b10658 or newer) — the rotated weight basis requires a runtime Walsh–Hadamard transform that is not upstream yet. On a stock build PQ2_0 and PTQ1_0 are refused, and a Q2_0 file loads silently and outputs gibberish. The MLX pack runs on stock MLX.

​
How to run it

Through the demo repo (Ternary Bonsai 2 27B is the default, so no flags needed):
Or a single generation directly with llama.cpp / MLX:

​
Vision and tool calling

27B is trained for both. Send an image as a normal OpenAI-style image_url content part, and a tools array for function calling — both come back through the standard API, no prompt hacks required:
The response’s choices[0].message.tool_calls carries the call. See Tool calling & MCP in the demo repo for MCP server setup and the full agentic walkthrough, and Open WebUI for a point-and-click demo of the same capabilities.

​
Thinking mode

27B reasons by default. Toggle it per request with thinking_budget_tokens (0 to disable, -1 for unlimited), cap it server-wide with --reasoning-budget N, or turn it off entirely with BONSAI_THINKING=0 ./scripts/start_llama_server.sh. The built-in chat UI has a Reasoning effort picker (Off / Low / Medium / High / Max) that overrides the server default per conversation. The model itself inherits two reasoning-effort settings from Qwen3.8-27B — xhigh (the default) and medium. A low setting does not reduce the amount of thinking; use the token budget to bound it.
Large images are downscaled to ~1,024 vision tokens by default on Metal, Vulkan, and CPU to keep latency reasonable; CUDA and ROCm run uncapped. Override with BONSAI_IMAGE_MAX_TOKENS (0 disables the cap — useful for reading small text in screenshots).