Skip to main content
Bonsai 4B targets edge deployments where the 8B doesn’t fit or where you need headroom for other workloads. At 0.57 GB (1-bit), the weights are small enough to power applications on wearables, and mobile devices.

​
Specifications

Parameters~4B
Max context32,768 tokens (native 8,192, extended 4x via YaRN)
ModalitiesText
Layers36
Hidden size2,560
Attention32 heads, 8 KV heads (GQA)
Vocabulary151,669 (embeddings tied with LM head)
LicenseApache-2.0

​
Artifacts

FamilyFormatRepositoryOn disk
Bonsai (1-bit)GGUFprism-ml/Bonsai-4B-gguf0.57 GB
Bonsai (1-bit)MLXprism-ml/Bonsai-4B-mlx-1bit0.63 GB
Ternary (1.58-bit)GGUFprism-ml/Ternary-Bonsai-4B-gguf1.07 GB
Ternary (1.58-bit)MLXprism-ml/Ternary-Bonsai-4B-mlx-2bit1.13 GB
The FP16 reference weights (8.05 GB) are in the ternary GGUF repo.

​
Run it

Through the demo repo: 
BONSAI_MODEL=4B ./scripts/run_llama.sh -p "Summarize this file: ..."
BONSAI_MODEL=4B ./scripts/start_llama_server.sh   # OpenAI-compatible API on :8080
Or directly with llama.cpp / MLX: 
./llama-cli -m ./Bonsai-4B-gguf/Bonsai-4B-Q1_0.gguf -c 0 -p "Hello"

mlx_lm.generate --model prism-ml/Ternary-Bonsai-4B-mlx-2bit --prompt "Hello"