Bonsai 27B

Bonsai 27B is the largest multimodal model in the family: it accepts vision input alongside text. It is based on Qwen3.6 27B. Like every Bonsai, the language model is quantized end-to-end 1-bit or ternary weights across embeddings, attention, MLPs, and the LM head. The vision tower is handled separately, at 4-bit.

Specifications


Parameters	27B
Max context	262,144 tokens
Modalities	Text + image in, text out
License	Apache 2.0

Artifacts

All 27B repositories are in the Bonsai 27B collection on Hugging Face.

Family	Format	Repository	Weights on disk
Ternary (1.58-bit)	GGUF (`Q2_0`)	`prism-ml/Ternary-Bonsai-27B-gguf`	6.66 GiB
Ternary (1.58-bit)	MLX (2-bit)	`prism-ml/Ternary-Bonsai-27B-mlx-2bit`	7.05 GiB
Bonsai (1-bit)	GGUF (`Q1_0`)	`prism-ml/Bonsai-27B-gguf`	3.53 GiB
Bonsai (1-bit)	MLX (1-bit)	`prism-ml/Bonsai-27B-mlx-1bit`	3.92 GiB

Every repo also ships an mmproj file for the vision tower (+0.9 GiB), needed for image input regardless of which weight format you pick.

How to run it

Through the demo repo (ternary 27B is the default, so no flags needed):

./scripts/start_llama_server.sh   # OpenAI-compatible API + chat/vision UI on :8080

Or a single generation directly with llama.cpp / MLX:

./llama-cli -m ./Ternary-Bonsai-27B-gguf/Ternary-Bonsai-27B-Q2_0.gguf --mmproj ./Ternary-Bonsai-27B-gguf/mmproj.gguf -c 0 \
  -p "Explain KV cache growth in one paragraph."

mlx_lm.generate --model prism-ml/Ternary-Bonsai-27B-mlx-2bit \
  --prompt "Explain KV cache growth in one paragraph."

Vision and tool calling

27B is trained for both. Send an image as a normal OpenAI-style image_url content part, and a tools array for function calling — both come back through the standard API, no prompt hacks required:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is the weather in Lisbon?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
      }
    }]
  }'

The response’s choices[0].message.tool_calls carries the call. See Tool calling & MCP in the demo repo for MCP server setup and the full agentic walkthrough, and Open WebUI for a point-and-click demo of the same capabilities.

Thinking mode

27B reasons by default. Toggle it per request with thinking_budget_tokens (0 to disable, -1 for unlimited), cap it server-wide with --reasoning-budget N, or turn it off entirely with BONSAI_THINKING=0 ./scripts/start_llama_server.sh. The built-in chat UI has a Reasoning effort picker (Off / Low / Medium / High / Max) that overrides the server default per conversation.

Large images are downscaled to ~1,024 vision tokens by default on Metal, Vulkan, and CPU to keep latency reasonable; CUDA and ROCm run uncapped. Override with BONSAI_IMAGE_MAX_TOKENS (0 disables the cap — useful for reading small text in screenshots).

Get started

Models

Download

Run Locally

Integrations

Resources

Getting Started

Specifications

Artifacts

How to run it

Vision and tool calling

Thinking mode

​Specifications

​Artifacts

​How to run it

​Vision and tool calling

​Thinking mode

Specifications

Artifacts

How to run it

Vision and tool calling

Thinking mode