Bonsai 4B - Bonsai

Bonsai 4B targets edge deployments where the bigger Bonsai models don’t fit or where you need headroom for other workloads. At 0.57 GB (1-bit), the weights are small enough to power applications on wearables, and mobile devices.

Specifications


Parameters	~4B
Max context	32,768 tokens (native 8,192, extended 4x via YaRN)
Modalities	Text
License	Apache 2.0

Artifacts

Family	Format	Repository	On disk
Bonsai (1-bit)	GGUF	`prism-ml/Bonsai-4B-gguf`	0.57 GB
Bonsai (1-bit)	MLX	`prism-ml/Bonsai-4B-mlx-1bit`	0.63 GB
Ternary (1.58-bit)	GGUF	`prism-ml/Ternary-Bonsai-4B-gguf`	1.07 GB
Ternary (1.58-bit)	MLX	`prism-ml/Ternary-Bonsai-4B-mlx-2bit`	1.13 GB

The FP16 reference weights (8.05 GB) are in the ternary GGUF repo.

Run it

Through the demo repo:

BONSAI_MODEL=4B ./scripts/run_llama.sh -p "Summarize this file: ..."
BONSAI_MODEL=4B ./scripts/start_llama_server.sh   # OpenAI-compatible API on :8080

Or directly with llama.cpp / MLX:

./llama-cli -m ./Bonsai-4B-gguf/Bonsai-4B-Q1_0.gguf -c 0 -p "Hello"

mlx_lm.generate --model prism-ml/Ternary-Bonsai-4B-mlx-2bit --prompt "Hello"

⌘I