> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prismml.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Bonsai 8B

> 8.2B parameters in 1.16 GB — runs on laptops and phones at up to 368 tok/s.

Bonsai 8B is the efficient middle of the family: small enough to run on a phone, fast enough to feel instant on a laptop GPU. Pick it over [27B](/models/bonsai-27b) when you don't need vision, tool calling, or 27B-scale reasoning; pick it over [4B](/models/bonsai-4b)/[1.7B](/models/bonsai-1-7b) when you need more capability and have the extra headroom.

## Specifications

|             |                                                     |
| ----------- | --------------------------------------------------- |
| Parameters  | 8.2B                                                |
| Max context | 65,536 tokens (native 16,384, extended 4x via YaRN) |
| Modalities  | Text                                                |
| Layers      | 36                                                  |
| Hidden size | 4,096 (MLP intermediate 12,288)                     |
| Attention   | 32 heads, 8 KV heads (GQA), head dim 128            |
| Vocabulary  | 151,669                                             |
| RoPE theta  | 1,000,000                                           |
| License     | Apache-2.0                                          |

## Artifacts

| Family             | Format | Repository                                                                                          | On disk |
| ------------------ | ------ | --------------------------------------------------------------------------------------------------- | ------- |
| Bonsai (1-bit)     | GGUF   | [`prism-ml/Bonsai-8B-gguf`](https://huggingface.co/prism-ml/Bonsai-8B-gguf)                         | 1.16 GB |
| Bonsai (1-bit)     | MLX    | [`prism-ml/Bonsai-8B-mlx-1bit`](https://huggingface.co/prism-ml/Bonsai-8B-mlx-1bit)                 | 1.28 GB |
| Ternary (1.58-bit) | GGUF   | [`prism-ml/Ternary-Bonsai-8B-gguf`](https://huggingface.co/prism-ml/Ternary-Bonsai-8B-gguf)         | 2.18 GB |
| Ternary (1.58-bit) | MLX    | [`prism-ml/Ternary-Bonsai-8B-mlx-2bit`](https://huggingface.co/prism-ml/Ternary-Bonsai-8B-mlx-2bit) | 2.30 GB |

The FP16 reference weights (16.38 GB) are also in the ternary GGUF repo for comparison work.

## Run it

Through the [demo repo](/get-started/quickstart):

```bash theme={null}
BONSAI_MODEL=8B ./scripts/run_llama.sh -p "Explain KV cache growth in one paragraph."
BONSAI_MODEL=8B BONSAI_FAMILY=ternary ./scripts/run_llama.sh -p "..."   # ternary instead
BONSAI_MODEL=8B ./scripts/start_llama_server.sh   # OpenAI-compatible API on :8080
```

Or directly with [llama.cpp](/run/llamacpp) / [MLX](/run/mlx):

<CodeGroup>
  ```bash llama.cpp theme={null}
  ./llama-cli -m ./Bonsai-8B-gguf/Bonsai-8B-Q1_0.gguf -c 0 \
    -p "Explain KV cache growth in one paragraph."
  ```

  ```bash MLX (Apple Silicon) theme={null}
  mlx_lm.generate --model prism-ml/Ternary-Bonsai-8B-mlx-2bit \
    --prompt "Explain KV cache growth in one paragraph."
  ```
</CodeGroup>

<Note>
  Ternary GGUF needs the [PrismML llama.cpp fork](https://github.com/PrismML-Eng/llama.cpp); 1-bit runs on upstream llama.cpp. See [Formats & runtime support](/download/formats).
</Note>