Skip to main content
MLX is Apple’s array framework, tuned for Apple Silicon. It’s the fastest way to run Bonsai on an M-series Mac, using the MLX format weights, and it’s what the iPhone/iPad apps build on.
Ternary (2-bit) works with stock MLX today. 1-bit support is pending upstream (mlx#3161); until it merges, use the PrismML MLX fork. See Formats & runtime support.

​
Install

pip install mlx-lm
For 1-bit weights before the upstream PR merges, install from the fork instead: 
pip install "mlx-lm @ git+https://github.com/PrismML-Eng/mlx"
The demo repo’s setup.sh builds MLX from source automatically on Apple Silicon, and ./scripts/run_mlx.sh wraps generation with the recommended settings.

​
Run a chat

mlx_lm.generate \
  --model prism-ml/Ternary-Bonsai-8B-mlx-2bit \
  --temp 0.5 --top-p 0.85 \
  --prompt "What is the capital of France?"
--model accepts a prism-ml repository name (downloads on first use) or a local directory (see Download the models). --temp 0.5 --top-p 0.85 are Bonsai’s recommended sampling settings.

​
Start a server

mlx_lm.server --model prism-ml/Ternary-Bonsai-8B-mlx-2bit --port 8081
This exposes the same OpenAI-compatible API described in Run the server (the demo repo’s start_mlx_server.sh does exactly this on port 8081), so every integration works against it too.

​
GGUF or MLX on a Mac?

Both run well on Apple Silicon. MLX generally gives the best raw decode speed; GGUF via llama.cpp is more portable across the wider tooling ecosystem (Ollama, anything llama.cpp-based). If you plan to use desktop apps, pick GGUF; if you’re scripting or chasing tokens per second, pick MLX.
On Bonsai 27B, the ternary MLX build gets vision and native tool calling through mlx-vlm (setup.sh installs it automatically; opt out with BONSAI_MLX_VLM=0). The 1-bit 27B MLX build isn’t verified for vision yet — use ternary, or the llama.cpp build, if you need images on the 1-bit family. MLX also has no cross-request prompt cache, so multi-turn conversations reprocess the full history (including image tokens) on every turn — for interactive multi-turn use, prefer llama.cpp, which caches the prefix.