> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prismml.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Ternary Bonsai 2 27B

> The flagship Bonsai: multimodal, a 262K-token context, in ternary weights at 1.76 bits.

Ternary Bonsai 2 27B is the largest multimodal model in the family: it accepts vision input alongside text. It is based on Qwen3.8 27B, a hybrid-attention causal language model, and its matrix weights are ternary — values in `{−1, 0, +1}` in a fixed rotated basis, with FP16 group scales.

## Specifications

|                     |                                                                               |
| ------------------- | ----------------------------------------------------------------------------- |
| Parameters          | 27.36B total (24.35B language + 0.47B vision tower + 2.54B embedding/LM head) |
| Architecture        | Hybrid attention (\~75% linear / \~25% full), SwiGLU MLP, RoPE, RMSNorm       |
| Weight format       | Ternary g128 with FP16 group-wise scaling, blockwise Hadamard rotation        |
| Effective bit width | 1.72 bpw true ternary; 1.76 bpw as shipped in `PTQ1_0`                        |
| Max context         | 262,144 tokens                                                                |
| Modalities          | Text + image in, text out                                                     |
| License             | Apache 2.0                                                                    |

On the 20-benchmark thinking-mode suite it averages 83.9, retaining 98.2% of the Qwen3.8-27B FP16 baseline's 85.4 at roughly a ninth of the size. See [benchmarks](/models/benchmarks).

## Artifacts

All Ternary Bonsai 2 repositories are in the [Bonsai 2 collection](https://huggingface.co/collections/prism-ml/bonsai-2) on Hugging Face.

| Format                    | Repository                                                                                                | Weights on disk                             |
| ------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------- |
| GGUF (`PTQ1_0`, 1.76 bpw) | [`prism-ml/Ternary-Bonsai-2-27B-gguf`](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf)         | 5.93 GB                                     |
| GGUF (`PQ2_0`, 2.16 bpw)  | [`prism-ml/Ternary-Bonsai-2-27B-gguf`](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf)         | 7.25 GB                                     |
| MLX (2-bit, 2.25 bpw)     | [`prism-ml/Ternary-Bonsai-2-27B-mlx-2bit`](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit) | 8.49 GB (incl. full-precision vision tower) |

`PTQ1_0` has the smaller footprint; `PQ2_0` uses a simpler 2-bit representation that is cheaper to unpack, so prompt processing is faster. Neither is uniformly faster. The demo downloads `PQ2_0` by default.

The GGUF repo also ships the vision tower separately, needed for image input: `mmproj` HQQ 4-bit (0.63 GB, what the demo uses) or the BF16 reference (0.93 GB).

<Warning>
  Both GGUF packings need the [PrismML llama.cpp fork](https://github.com/PrismML-Eng/llama.cpp) (`prism-b10658` or newer) — the rotated weight basis requires a runtime Walsh–Hadamard transform that is [not upstream yet](https://github.com/ggml-org/llama.cpp/pull/27779). On a stock build `PQ2_0` and `PTQ1_0` are refused, and a `Q2_0` file loads silently and outputs gibberish. The MLX pack runs on stock MLX.
</Warning>

## How to run it

Through the [demo repo](/get-started/quickstart) (Ternary Bonsai 2 27B is the default, so no flags needed):

```bash theme={null}
./scripts/start_llama_server.sh   # OpenAI-compatible API + chat/vision UI on :8080
```

Or a single generation directly with [llama.cpp](/run/llamacpp) / [MLX](/run/mlx):

<CodeGroup>
  ```bash llama.cpp theme={null}
  ./llama-cli -m ./Ternary-Bonsai-2-27B-gguf/Ternary-Bonsai-2-27B-PQ2_0.gguf --mmproj ./Ternary-Bonsai-2-27B-gguf/mmproj.gguf -c 32768 \
    -p "Explain KV cache growth in one paragraph."
  ```

  ```bash MLX (Apple Silicon) theme={null}
  mlx_lm.generate --model prism-ml/Ternary-Bonsai-2-27B-mlx-2bit \
    --prompt "Explain KV cache growth in one paragraph."
  ```
</CodeGroup>

### Vision and tool calling

27B is trained for both. Send an image as a normal OpenAI-style `image_url` content part, and a `tools` array for function calling — both come back through the standard API, no prompt hacks required:

```bash theme={null}
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is the weather in Lisbon?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
      }
    }]
  }'
```

The response's `choices[0].message.tool_calls` carries the call. See [Tool calling & MCP](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/TOOLS.md) in the demo repo for MCP server setup and the full agentic walkthrough, and [Open WebUI](/run/open-webui) for a point-and-click demo of the same capabilities.

### Thinking mode

27B reasons by default. Toggle it per request with `thinking_budget_tokens` (`0` to disable, `-1` for unlimited), cap it server-wide with `--reasoning-budget N`, or turn it off entirely with `BONSAI_THINKING=0 ./scripts/start_llama_server.sh`. The built-in chat UI has a Reasoning effort picker (Off / Low / Medium / High / Max) that overrides the server default per conversation.

The model itself inherits two reasoning-effort settings from Qwen3.8-27B — `xhigh` (the default) and `medium`. A low setting does not reduce the amount of thinking; use the token budget to bound it.

<Note>
  Large images are downscaled to \~1,024 vision tokens by default on Metal, Vulkan, and CPU to keep latency reasonable; CUDA and ROCm run uncapped. Override with `BONSAI_IMAGE_MAX_TOKENS` (`0` disables the cap — useful for reading small text in screenshots).
</Note>