Skip to main content
Bonsai ships in two formats. Which you want depends on your hardware and the runtime you plan to use.

GGUF

For llama.cpp and the ecosystem built on it. Cross-platform: CPU, CUDA, Metal, Vulkan, ROCm.

MLX

Apple’s array framework, tuned for Apple Silicon. Best raw performance on M-series Macs and the basis for the iPhone/iPad apps.
On a Mac, both work; MLX is usually faster, GGUF is more portable across tools. Everywhere else, use GGUF.

​
Quantization types

​
Runtime support matrix

This is the part that matters most for a low-bit model: a runtime without native kernels either refuses the file or runs it dequantized, silently losing the speed and memory advantage.

​
1-bit (Q1_0): merged upstream

Because Q1_0 is upstream, any llama.cpp-based tool built from a recent enough version runs 1-bit Bonsai. When in doubt, use the PrismML llama.cpp binaries, which are known-good.

​
Ternary (Q2_0): PrismML fork for now

Ternary GGUF requires the PrismML llama.cpp fork (prism branch) or its pre-built binaries. Tools that bundle their own stock llama.cpp, including current Ollama releases, cannot run ternary Q2_0 yet. For ternary on Apple Silicon, MLX works out of the box.

​
Pre-built binaries

The PrismML llama.cpp release covers 1-bit and ternary on: setup.sh in the demo repo picks the right one automatically. Build-from-source instructions are on the llama.cpp page.