Skip to main content
Ternary Bonsai 2 27B is the largest model in the family and the demo repo’s default. It is derived from Qwen3.8-27B, a hybrid-attention causal language model, with the architecture unchanged. Every language-model weight is ternary: {−1, 0, +1} in a fixed rotated basis, with an FP16 scale per group of 128 weights. It takes images as well as text, carries a 262K-token context, calls tools natively, and reasons before it answers.
Before you run it: the recommended sampling settings differ between thinking and instruct mode, and low reasoning effort is not supported. See Best practices.

​
Specifications

On the model card’s 14-benchmark thinking-mode suite it averages 84.78, retaining 98.2% of the Qwen3.8-27B FP16 baseline’s 86.32 at roughly a ninth of the size; the whitepaper’s 20-benchmark suite gives 83.9 against 85.4, also 98.2%. Math stays within half a point of full precision and coding is level with the baseline. The full tables are on the model card and in the whitepaper.

​
Artifacts

All Ternary Bonsai 2 repositories are in the Bonsai 2 collection on Hugging Face. The demo downloads PQ2_0 by default; see choosing a packing below. The GGUF repo also ships the vision tower separately, needed for image input: mmproj Q8_0 (0.63 GB, what the demo uses) or the BF16 reference (0.93 GB), plus the 53.8 GB F16 reference weights for comparison work.
Which runtime runs which file. Upstream llama.cpp (build b11555 or newer) runs the Q2_0 packing; its file still carries a historical prism-fork-required suffix, but compatibility depends on the build, not the name. PQ2_0 and PTQ1_0 need the PrismML llama.cpp binaries, which the demo’s setup.sh installs. The MLX pack loads natively in mlx-vlm 0.7.8 and newer. See Formats & runtime support.

​
How to run it

Through the demo repo, where Ternary Bonsai 2 27B is the default, so no flags are needed:
Or a single generation directly with llama.cpp / MLX:
-n 16384 (or max_tokens) leaves room for the thinking trace: the template defaults to xhigh reasoning effort, so a small cap ends generation mid-thought, before any answer. The demo repo is the source of truth for running these models; tested setups and current implementation details live there.

​
Best practices

These apply to both the GGUF and MLX releases.

​
Generation parameters

Use the set that matches the mode you’re running in. Thinking and instruct mode want different sampling settings, and mixing them degrades output quality.
min_p = 0.05 drops tokens far less likely than the top choice. In our tests it scored at least as well as 0.0 and followed instructions more reliably. It is also llama.cpp’s default: the GGUF carries top_k, top_p and temperature but not min_p, so llama.cpp applies 0.05 without being told. Other runtimes may default to 0.0, so set it explicitly there. Presence and repetition penalties, and all instruct-mode values, are not in the GGUF; pass them yourself. The published benchmarks used the thinking-mode values with min_p = 0.0.

​
Reasoning effort

The model uses xhigh by default. medium thinks noticeably less and, at moderate output limits, is about as accurate, so it is a good default when responses feel slow; the whitepaper’s Appendix C quantifies the cost on the hardest benchmarks. low is not supported: it does not reduce thinking. To bound reasoning, use a token budget instead (see Thinking mode).
Send reasoning_effort per request, or make medium the server-wide default while letting a request still ask for xhigh:

​
System prompt

A plain system message works well:

​
Choosing a packing

Pick by hardware target and whether footprint or decode speed matters more. The Q2_0 file is the one upstream llama.cpp runs; pick it for tools that bundle a stock llama.cpp.

​
Vision and tool calling

27B is trained for both. Send an image as a normal OpenAI-style image_url content part, and a tools array for function calling. Both come back through the standard API, no prompt hacks required:
The response’s choices[0].message.tool_calls carries the call. Use the Q8_0 projector for images, which is what the demo downloads; the BF16 file is the reference-precision alternative. See Tool calling & MCP in the demo repo for MCP server setup and the full agentic walkthrough, and Open WebUI for a point-and-click demo of the same capabilities.

​
Thinking mode

27B reasons by default. Toggle it per request with thinking_budget_tokens (0 to disable, -1 for unlimited), cap it server-wide with --reasoning-budget N, or turn it off entirely by passing --reasoning-budget 0 through the launcher (./scripts/start_llama_server.sh --reasoning-budget 0); --no-think does the same for run_mlx.sh. The built-in chat UI has a Reasoning effort picker (Off / Low / Medium / High / Max) that overrides the server default per conversation. The UI’s levels are fixed token budgets (Medium is 2,048 tokens), which cut thinking off at that length; the model’s own reasoning_effort: "medium" asks it to think less instead, and rarely runs out mid-reasoning.
Large images are downscaled to ~1,024 vision tokens by default on Metal, Vulkan, and CPU to keep latency reasonable; CUDA and ROCm run uncapped. Override with BONSAI_IMAGE_MAX_TOKENS (0 disables the cap, useful for reading small text in screenshots).