> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prismml.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Bonsai 27B
> The flagship Bonsai: vision-language, native tool calling, and a 262K-token context, in binary or ternary weights.
Bonsai 27B is the largest model in the family and the first with vision input: it accepts photos, screenshots, and PDFs alongside text. It's derived from Qwen3.6-27B, a hybrid-attention model (roughly 75% linear attention, 25% full attention), which is what keeps its long context practical on-device. Like every Bonsai, the language model is quantized end-to-end 1-bit or ternary weights across embeddings, attention, MLPs, and the LM head, with no higher-precision fallback. The vision tower is handled separately, at 4-bit (NF4).
**Ternary is the default** for this size: it's the quality-oriented operating point, and what `./setup.sh` downloads if you don't set `BONSAI_FAMILY`.
## Specifications
| | |
| ----------- | ---------------------------------------------------- |
| Parameters | 27B |
| Max context | 262,144 tokens |
| Modalities | Text + image in, text out |
| Attention | Hybrid: \~75% linear attention, \~25% full attention |
| Base model | Qwen3.6-27B |
| License | Apache-2.0 |
## Artifacts
All 27B repositories are in the [Bonsai 27B collection](https://huggingface.co/collections/prism-ml/bonsai-27b) on Hugging Face.
| Family | Format | Repository | Weights on disk |
| ------------------ | ------------- | ----------------------------------------------------------------------------------------------------- | --------------- |
| Ternary (1.58-bit) | GGUF (`Q2_0`) | [`prism-ml/Ternary-Bonsai-27B-gguf`](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf) | 6.66 GiB |
| Ternary (1.58-bit) | MLX (2-bit) | [`prism-ml/Ternary-Bonsai-27B-mlx-2bit`](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-mlx-2bit) | 7.05 GiB |
| Bonsai (1-bit) | GGUF (`Q1_0`) | [`prism-ml/Bonsai-27B-gguf`](https://huggingface.co/prism-ml/Bonsai-27B-gguf) | 3.53 GiB |
| Bonsai (1-bit) | MLX (1-bit) | [`prism-ml/Bonsai-27B-mlx-1bit`](https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit) | 3.92 GiB |
Every repo also ships an `mmproj` file for the vision tower (+0.9 GiB), needed for image input regardless of which weight format you pick.
## Memory requirements
Total memory is weights + `mmproj` + activations + the FP16 KV cache, which grows with context. The 27B's hybrid attention keeps that cache small relative to its size: 64 KiB per token, so even 100K-token conversations stay well under what a 16-bit model needs just to load.
Peak memory, text-only (add \~0.9 GiB if you're sending images):
| Build | Weights | 4K context | 10K context | 100K context |
| ------------------------------------------------ | --------- | ---------- | ----------- | ------------ |
| Bonsai-27B (1-bit), llama.cpp `Q1_0` | 3.53 GiB | 4.8 GiB | 5.2 GiB | 10.8 GiB |
| Bonsai-27B (1-bit), MLX | 3.92 GiB | 5.5 GiB | 5.9 GiB | 11.4 GiB |
| Ternary-Bonsai-27B, llama.cpp `Q2_0` | 6.66 GiB | 7.8 GiB | 8.1 GiB | 13.7 GiB |
| Ternary-Bonsai-27B, MLX | 7.05 GiB | 8.6 GiB | 8.9 GiB | 14.4 GiB |
| *Reference: 27B FP16* | 47.73 GiB | 49 GiB | 49.6 GiB | 55.2 GiB |
| *Reference: 27B "4-bit" (llama.cpp `UD Q4_K_M`)* | 15.73 GiB | 17.2 GiB | 17.6 GiB | 23.2 GiB |
Both Ternary-Bonsai-27B builds fit comfortably on a standard laptop; the 1-bit build is the one that fits a high-end phone. `-c 0` (the demo scripts' default) auto-fits the context window to available memory instead of pre-allocating the max.
## Run it
Through the [demo repo](/get-started/quickstart) (ternary 27B is the default, so no flags needed):
```bash theme={null}
./scripts/start_llama_server.sh # OpenAI-compatible API + chat/vision UI on :8080
```
Or a single generation directly with [llama.cpp](/run/llamacpp) / [MLX](/run/mlx):
```bash llama.cpp theme={null}
./llama-cli -m ./Ternary-Bonsai-27B-gguf/Ternary-Bonsai-27B-Q2_0.gguf --mmproj ./Ternary-Bonsai-27B-gguf/mmproj.gguf -c 0 \
-p "Explain KV cache growth in one paragraph."
```
```bash MLX (Apple Silicon) theme={null}
mlx_lm.generate --model prism-ml/Ternary-Bonsai-27B-mlx-2bit \
--prompt "Explain KV cache growth in one paragraph."
```
### Vision and tool calling
27B is trained for both. Send an image as a normal OpenAI-style `image_url` content part, and a `tools` array for function calling — both come back through the standard API, no prompt hacks required:
```bash theme={null}
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [{"role": "user", "content": "What is the weather in Lisbon?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
}
}]
}'
```
The response's `choices[0].message.tool_calls` carries the call. See [Tool calling & MCP](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/TOOLS.md) in the demo repo for MCP server setup and the full agentic walkthrough, and [Open WebUI](/run/open-webui) for a point-and-click demo of the same capabilities.
### Thinking mode
27B reasons by default. Toggle it per request with `thinking_budget_tokens` (`0` to disable, `-1` for unlimited), cap it server-wide with `--reasoning-budget N`, or turn it off entirely with `BONSAI_THINKING=0 ./scripts/start_llama_server.sh`. The built-in chat UI has a Reasoning effort picker (Off / Low / Medium / High / Max) that overrides the server default per conversation.
Large images are downscaled to \~1,024 vision tokens by default on Metal, Vulkan, and CPU to keep latency reasonable; CUDA and ROCm run uncapped. Override with `BONSAI_IMAGE_MAX_TOKENS` (`0` disables the cap — useful for reading small text in screenshots).