Skip to main content
LM Studio is a desktop app for running local models, with a bundled lms CLI and an OpenAI-compatible HTTP server. It’s the fastest way to get Bonsai running on a Mac — and because it serves /v1/chat/completions, the same instance doubles as the backend for any OpenAI-compatible client or agent.
LM Studio support is MLX only. Use prism-ml/Ternary-Bonsai-27B-mlx-2bit, which requires an Apple Silicon Mac (M-series).

​
Requirements

Weights are ~8.52 GB, but KV cache grows with context length and concurrency — budget headroom above the weight footprint.

​
Step 1: Install LM Studio

Download the macOS build from lmstudio.ai.

​
Step 2: Download the model

​
Step 3: Load and chat

1

Load the model

Pick it from the model selector at the top of the app’s Chat tab. Set context length in the load panel; start at 32K–64K and raise it once you’ve seen memory pressure at your workload.
2

Chat

Go the Chat tab from the panel, and select the ‘Ternary-Bonsai-27B-mlx-2bit’ at top.

​
Step 4: Serve the model

The same install exposes an OpenAI-compatible server, so LM Studio can act as the Bonsai endpoint for external tools.
Available endpoints: GET /v1/models, POST /v1/chat/completions, POST /v1/completions, POST /v1/embeddings, POST /v1/responses. Don’t hardcode the model ID — read it back from the server:
Then call it:
Authentication is off by default and the server binds to localhost. To reach it from another machine, enable network serving in the Developer tab and put it behind a reverse proxy with auth — don’t expose it directly.
Anywhere our docs ask for an OpenAI-compatible Bonsai endpoint, http://localhost:1234/v1 works. For Hermes, substitute it for the demo-repo base URL in Step 1 and use any non-empty string as the API key.