Skip to main content
Bonsai 27B is the largest model in the family and the first with vision input: it accepts photos, screenshots, and PDFs alongside text. It’s derived from Qwen3.6-27B, a hybrid-attention model (roughly 75% linear attention, 25% full attention), which is what keeps its long context practical on-device. Like every Bonsai, the language model is quantized end-to-end 1-bit or ternary weights across embeddings, attention, MLPs, and the LM head, with no higher-precision fallback. The vision tower is handled separately, at 4-bit (NF4). Ternary is the default for this size: it’s the quality-oriented operating point, and what ./setup.sh downloads if you don’t set BONSAI_FAMILY.

​
Specifications

Parameters27B
Max context262,144 tokens
ModalitiesText + image in, text out
AttentionHybrid: ~75% linear attention, ~25% full attention
Base modelQwen3.6-27B
LicenseApache-2.0

​
Artifacts

All 27B repositories are in the Bonsai 27B collection on Hugging Face.
FamilyFormatRepositoryWeights on disk
Ternary (1.58-bit)GGUF (Q2_0)prism-ml/Ternary-Bonsai-27B-gguf6.66 GiB
Ternary (1.58-bit)MLX (2-bit)prism-ml/Ternary-Bonsai-27B-mlx-2bit7.05 GiB
Bonsai (1-bit)GGUF (Q1_0)prism-ml/Bonsai-27B-gguf3.53 GiB
Bonsai (1-bit)MLX (1-bit)prism-ml/Bonsai-27B-mlx-1bit3.92 GiB
Every repo also ships an mmproj file for the vision tower (+0.9 GiB), needed for image input regardless of which weight format you pick.

​
Memory requirements

Total memory is weights + mmproj + activations + the FP16 KV cache, which grows with context. The 27B’s hybrid attention keeps that cache small relative to its size: 64 KiB per token, so even 100K-token conversations stay well under what a 16-bit model needs just to load. Peak memory, text-only (add ~0.9 GiB if you’re sending images):
BuildWeights4K context10K context100K context
Bonsai-27B (1-bit), llama.cpp Q1_03.53 GiB4.8 GiB5.2 GiB10.8 GiB
Bonsai-27B (1-bit), MLX3.92 GiB5.5 GiB5.9 GiB11.4 GiB
Ternary-Bonsai-27B, llama.cpp Q2_06.66 GiB7.8 GiB8.1 GiB13.7 GiB
Ternary-Bonsai-27B, MLX7.05 GiB8.6 GiB8.9 GiB14.4 GiB
Reference: 27B FP1647.73 GiB49 GiB49.6 GiB55.2 GiB
Reference: 27B “4-bit” (llama.cpp UD Q4_K_M)15.73 GiB17.2 GiB17.6 GiB23.2 GiB
Both Ternary-Bonsai-27B builds fit comfortably on a standard laptop; the 1-bit build is the one that fits a high-end phone. -c 0 (the demo scripts’ default) auto-fits the context window to available memory instead of pre-allocating the max.

​
Run it

Through the demo repo (ternary 27B is the default, so no flags needed): 
./scripts/start_llama_server.sh   # OpenAI-compatible API + chat/vision UI on :8080
Or a single generation directly with llama.cpp / MLX: 
./llama-cli -m ./Ternary-Bonsai-27B-gguf/Ternary-Bonsai-27B-Q2_0.gguf --mmproj ./Ternary-Bonsai-27B-gguf/mmproj.gguf -c 0 \
  -p "Explain KV cache growth in one paragraph."

mlx_lm.generate --model prism-ml/Ternary-Bonsai-27B-mlx-2bit \
  --prompt "Explain KV cache growth in one paragraph."

​
Vision and tool calling

27B is trained for both. Send an image as a normal OpenAI-style image_url content part, and a tools array for function calling — both come back through the standard API, no prompt hacks required: 
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is the weather in Lisbon?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
      }
    }]
  }'
The response’s choices[0].message.tool_calls carries the call. See Tool calling & MCP in the demo repo for MCP server setup and the full agentic walkthrough, and Open WebUI for a point-and-click demo of the same capabilities.

​
Thinking mode

27B reasons by default. Toggle it per request with thinking_budget_tokens (0 to disable, -1 for unlimited), cap it server-wide with --reasoning-budget N, or turn it off entirely with BONSAI_THINKING=0 ./scripts/start_llama_server.sh. The built-in chat UI has a Reasoning effort picker (Off / Low / Medium / High / Max) that overrides the server default per conversation.
Large images are downscaled to ~1,024 vision tokens by default on Metal, Vulkan, and CPU to keep latency reasonable; CUDA and ROCm run uncapped. Override with BONSAI_IMAGE_MAX_TOKENS (0 disables the cap — useful for reading small text in screenshots).