> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prismml.com/llms.txt
> Use this file to discover all available pages before exploring further.

# MLX

> Run Bonsai MLX weights on Apple Silicon for the best Mac performance.

[MLX](https://github.com/ml-explore/mlx) is Apple's array framework, tuned for Apple Silicon. It's the fastest way to run Bonsai on an M-series Mac, using the **MLX** format weights, and it's what the iPhone/iPad apps build on.

<Info>
  **Ternary (2-bit)** works with stock MLX today. **1-bit** support is pending upstream ([mlx#3161](https://github.com/ml-explore/mlx/pull/3161)); until it merges, use the [PrismML MLX fork](https://github.com/PrismML-Eng/mlx). See [Formats & runtime support](/download/formats).
</Info>

## Install

```bash theme={null}
pip install mlx-lm
```

For 1-bit weights before the upstream PR merges, install from the fork instead:

```bash theme={null}
pip install "mlx-lm @ git+https://github.com/PrismML-Eng/mlx"
```

The demo repo's `setup.sh` builds MLX from source automatically on Apple Silicon, and `./scripts/run_mlx.sh` wraps generation with the recommended settings.

## Run a chat

```bash theme={null}
mlx_lm.generate \
  --model prism-ml/Ternary-Bonsai-8B-mlx-2bit \
  --temp 0.5 --top-p 0.85 \
  --prompt "What is the capital of France?"
```

`--model` accepts a `prism-ml` repository name (downloads on first use) or a local directory (see [Download the models](/download/models)). `--temp 0.5 --top-p 0.85` are Bonsai's recommended sampling settings.

## Start a server

```bash theme={null}
mlx_lm.server --model prism-ml/Ternary-Bonsai-8B-mlx-2bit --port 8081
```

This exposes the same OpenAI-compatible API described in [Run the server](/run/server) (the demo repo's `start_mlx_server.sh` does exactly this on port 8081), so every [integration](/integrations/overview) works against it too.

## GGUF or MLX on a Mac?

Both run well on Apple Silicon. MLX generally gives the best raw decode speed; GGUF via [llama.cpp](/run/llamacpp) is more portable across the wider tooling ecosystem (Ollama, anything llama.cpp-based). If you plan to use desktop apps, pick GGUF; if you're scripting or chasing tokens per second, pick MLX.

<Note>
  On [Bonsai 27B](/models/bonsai-27b), the ternary MLX build gets vision and native tool calling through `mlx-vlm` (`setup.sh` installs it automatically; opt out with `BONSAI_MLX_VLM=0`). The 1-bit 27B MLX build isn't verified for vision yet — use ternary, or the llama.cpp build, if you need images on the 1-bit family. MLX also has no cross-request prompt cache, so multi-turn conversations reprocess the full history (including image tokens) on every turn — for interactive multi-turn use, prefer [llama.cpp](/run/llamacpp), which caches the prefix.
</Note>