Skip to main content
Bonsai 8B is the efficient middle of the family: small enough to run on a phone, fast enough to feel instant on a laptop GPU. Pick it over 27B when you don’t need vision, tool calling, or 27B-scale reasoning; pick it over 4B/1.7B when you need more capability and have the extra headroom.

​
Specifications

Parameters8.2B
Max context65,536 tokens (native 16,384, extended 4x via YaRN)
ModalitiesText
Layers36
Hidden size4,096 (MLP intermediate 12,288)
Attention32 heads, 8 KV heads (GQA), head dim 128
Vocabulary151,669
RoPE theta1,000,000
LicenseApache-2.0

​
Artifacts

FamilyFormatRepositoryOn disk
Bonsai (1-bit)GGUFprism-ml/Bonsai-8B-gguf1.16 GB
Bonsai (1-bit)MLXprism-ml/Bonsai-8B-mlx-1bit1.28 GB
Ternary (1.58-bit)GGUFprism-ml/Ternary-Bonsai-8B-gguf2.18 GB
Ternary (1.58-bit)MLXprism-ml/Ternary-Bonsai-8B-mlx-2bit2.30 GB
The FP16 reference weights (16.38 GB) are also in the ternary GGUF repo for comparison work.

​
Run it

Through the demo repo: 
BONSAI_MODEL=8B ./scripts/run_llama.sh -p "Explain KV cache growth in one paragraph."
BONSAI_MODEL=8B BONSAI_FAMILY=ternary ./scripts/run_llama.sh -p "..."   # ternary instead
BONSAI_MODEL=8B ./scripts/start_llama_server.sh   # OpenAI-compatible API on :8080
Or directly with llama.cpp / MLX: 
./llama-cli -m ./Bonsai-8B-gguf/Bonsai-8B-Q1_0.gguf -c 0 \
  -p "Explain KV cache growth in one paragraph."

mlx_lm.generate --model prism-ml/Ternary-Bonsai-8B-mlx-2bit \
  --prompt "Explain KV cache growth in one paragraph."
Ternary GGUF needs the PrismML llama.cpp fork; 1-bit runs on upstream llama.cpp. See Formats & runtime support.