Skip to main content
Bonsai 1.7B is the smallest model in the family. At 0.25 GB (1-bit), it targets smart glasses, wearables, and always-on background tasks where other models are too heavy.

​
Specifications

Parameters~1.7B
Max context32,768 tokens (native 8,192, extended 4x via YaRN)
ModalitiesText
Layers28
Hidden size2,048
Attention16 heads, 8 KV heads (GQA)
Vocabulary151,669 (embeddings tied with LM head)
LicenseApache-2.0

​
Artifacts

FamilyFormatRepositoryOn disk
Bonsai (1-bit)GGUFprism-ml/Bonsai-1.7B-gguf0.25 GB
Bonsai (1-bit)MLXprism-ml/Bonsai-1.7B-mlx-1bit0.27 GB
Ternary (1.58-bit)GGUFprism-ml/Ternary-Bonsai-1.7B-gguf0.46 GB
Ternary (1.58-bit)MLXprism-ml/Ternary-Bonsai-1.7B-mlx-2bit0.48 GB
The FP16 reference weights (3.45 GB) are in the ternary GGUF repo.

​
Run it

Through the demo repo: 
BONSAI_MODEL=1.7B ./scripts/run_llama.sh -p "Classify this sentence as positive or negative: ..."
BONSAI_MODEL=1.7B ./scripts/start_llama_server.sh   # OpenAI-compatible API on :8080
Or directly with llama.cpp / MLX: 
./llama-cli -m ./Bonsai-1.7B-gguf/Bonsai-1.7B-Q1_0.gguf -c 0 -p "Hello"

mlx_lm.generate --model prism-ml/Ternary-Bonsai-1.7B-mlx-2bit --prompt "Hello"