> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prismml.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Bonsai 4B

> Half the footprint of 8B for edge deployments

Bonsai 4B targets edge deployments where the [8B](/models/bonsai-8b) doesn't fit or where you need headroom for other workloads. At 0.57 GB (1-bit), the weights are small enough to power applications on wearables, and mobile devices.

## Specifications

|             |                                                    |
| ----------- | -------------------------------------------------- |
| Parameters  | \~4B                                               |
| Max context | 32,768 tokens (native 8,192, extended 4x via YaRN) |
| Modalities  | Text                                               |
| Layers      | 36                                                 |
| Hidden size | 2,560                                              |
| Attention   | 32 heads, 8 KV heads (GQA)                         |
| Vocabulary  | 151,669 (embeddings tied with LM head)             |
| License     | Apache-2.0                                         |

## Artifacts

| Family             | Format | Repository                                                                                          | On disk |
| ------------------ | ------ | --------------------------------------------------------------------------------------------------- | ------- |
| Bonsai (1-bit)     | GGUF   | [`prism-ml/Bonsai-4B-gguf`](https://huggingface.co/prism-ml/Bonsai-4B-gguf)                         | 0.57 GB |
| Bonsai (1-bit)     | MLX    | [`prism-ml/Bonsai-4B-mlx-1bit`](https://huggingface.co/prism-ml/Bonsai-4B-mlx-1bit)                 | 0.63 GB |
| Ternary (1.58-bit) | GGUF   | [`prism-ml/Ternary-Bonsai-4B-gguf`](https://huggingface.co/prism-ml/Ternary-Bonsai-4B-gguf)         | 1.07 GB |
| Ternary (1.58-bit) | MLX    | [`prism-ml/Ternary-Bonsai-4B-mlx-2bit`](https://huggingface.co/prism-ml/Ternary-Bonsai-4B-mlx-2bit) | 1.13 GB |

The FP16 reference weights (8.05 GB) are in the ternary GGUF repo.

## Run it

Through the [demo repo](/get-started/quickstart):

```bash theme={null}
BONSAI_MODEL=4B ./scripts/run_llama.sh -p "Summarize this file: ..."
BONSAI_MODEL=4B ./scripts/start_llama_server.sh   # OpenAI-compatible API on :8080
```

Or directly with [llama.cpp](/run/llamacpp) / [MLX](/run/mlx):

<CodeGroup>
  ```bash llama.cpp theme={null}
  ./llama-cli -m ./Bonsai-4B-gguf/Bonsai-4B-Q1_0.gguf -c 0 -p "Hello"
  ```

  ```bash MLX (Apple Silicon) theme={null}
  mlx_lm.generate --model prism-ml/Ternary-Bonsai-4B-mlx-2bit --prompt "Hello"
  ```
</CodeGroup>