Skip to main content
MLX is Apple’s array framework, tuned for Apple Silicon. It’s the fastest way to run Bonsai on an M-series Mac, using the MLX format weights, and it’s what the iPhone/iPad apps build on.
Ternary (2-bit), including Ternary Bonsai 2, works with stock MLX today. 1-bit support is pending upstream (mlx#3161); until it merges, use the PrismML MLX fork (branch prism). See Formats & runtime support.

​
Install

For 1-bit weights before the upstream PR merges, install from the fork instead:
The demo repo’s setup.sh builds MLX from source automatically on Apple Silicon, and ./scripts/run_mlx.sh wraps generation with the recommended settings.
Tested versions. The released 27B packs were validated with Python 3.11, mlx-lm==0.31.2, and the prism branch at commit 88c9c20. setup.sh builds the branch tip; to pin the validated runtime, clone and check out that commit first — setup reuses an existing ./mlx checkout.

​
Run a chat

--model accepts a prism-ml repository name (downloads on first use) or a local directory (see Download the models). --temp 0.5 --top-p 0.85 are Bonsai’s recommended sampling settings. The Ternary Bonsai 2 MLX package is 8.49 GB: its block format stores an FP16 bias alongside the scale (2.25 bpw versus 2.13 in GGUF), and it keeps the vision tower in full precision.

​
Start a server

This exposes the same OpenAI-compatible API described in Run the server (the demo repo’s start_mlx_server.sh does exactly this on port 8081), so every integration works against it too.

​
GGUF or MLX on a Mac?

Both run well on Apple Silicon. MLX generally gives the best raw decode speed; GGUF via llama.cpp is more portable across the wider tooling ecosystem (Ollama, anything llama.cpp-based). If you plan to use desktop apps, pick GGUF; if you’re scripting or chasing tokens per second, pick MLX.
On the 27B, the ternary MLX build gets vision and native tool calling through mlx-vlm (setup.sh installs it automatically; opt out with BONSAI_MLX_VLM=0). The 1-bit 27B MLX build isn’t verified for vision yet — use ternary, or the llama.cpp build, if you need images on the 1-bit family. MLX also has no cross-request prompt cache, so multi-turn conversations reprocess the full history (including image tokens) on every turn — for interactive multi-turn use, prefer llama.cpp, which caches the prefix.