Bonsai 8B

Bonsai 8B is the efficient middle of the family: small enough to run on a phone, fast enough to feel instant on a laptop GPU. Pick it over 27B when you don’t need vision, or 27B-scale reasoning, and operating on stricter memory budget.

Specifications


Parameters	8.2B
Max context	65,536 tokens (native 16,384, extended 4x via YaRN)
Modalities	Text
License	Apache 2.0

Artifacts

Family	Format	Repository	On disk
Bonsai (1-bit)	GGUF	`prism-ml/Bonsai-8B-gguf`	1.16 GB
Bonsai (1-bit)	MLX	`prism-ml/Bonsai-8B-mlx-1bit`	1.28 GB
Ternary (1.58-bit)	GGUF	`prism-ml/Ternary-Bonsai-8B-gguf`	2.18 GB
Ternary (1.58-bit)	MLX	`prism-ml/Ternary-Bonsai-8B-mlx-2bit`	2.30 GB

The FP16 reference weights (16.38 GB) are also in the ternary GGUF repo for comparison work.

Run it

Through the demo repo:

BONSAI_MODEL=8B ./scripts/run_llama.sh -p "Explain KV cache growth in one paragraph."
BONSAI_MODEL=8B BONSAI_FAMILY=ternary ./scripts/run_llama.sh -p "..."   # ternary instead
BONSAI_MODEL=8B ./scripts/start_llama_server.sh   # OpenAI-compatible API on :8080

Or directly with llama.cpp / MLX:

./llama-cli -m ./Bonsai-8B-gguf/Bonsai-8B-Q1_0.gguf -c 0 \
  -p "Explain KV cache growth in one paragraph."

mlx_lm.generate --model prism-ml/Ternary-Bonsai-8B-mlx-2bit \
  --prompt "Explain KV cache growth in one paragraph."

Ternary GGUF needs the PrismML llama.cpp fork; 1-bit runs on upstream llama.cpp. See Formats & runtime support.

⌘I

Get started

Models

Download

Run Locally

Integrations

Resources

Getting Started

Specifications

Artifacts

Run it

​Specifications

​Artifacts

​Run it

Specifications

Artifacts

Run it