> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prismml.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Quickstart

> Clone the demo repo, run one setup command, and chat with Bonsai locally.

This page gets you from zero to a running **Bonsai 27B** on macOS, Linux, or Windows. One script installs build tools, downloads the model weights and pre-built inference binaries, and leaves you ready to chat.

**Prerequisites:** `git`, `curl`, and enough disk for the model you pick (see [sizes](/models/overview#the-lineup)). Everything else, including Python tooling and the inference engine, is installed by the setup script.

<Note>
  Examples use `BONSAI_MODEL=27B`. Set `8B`, `4B`, or `1.7B` for a smaller model; everything else is identical. If you skip step 2 entirely, setup uses its defaults: `BONSAI_MODEL=27B`, `BONSAI_FAMILY=ternary`.
</Note>

## Install and run

<Steps>
  <Step title="Clone the repo">
    ```bash theme={null}
    git clone https://github.com/PrismML-Eng/Bonsai-demo.git
    cd Bonsai-demo
    ```
  </Step>

  <Step title="Choose your model (optional)">
    <CodeGroup>
      ```bash macOS / Linux theme={null}
      export BONSAI_MODEL=27B        # 27B (flagship, default), 8B, 4B, or 1.7B
      export BONSAI_FAMILY=ternary   # ternary (1.58-bit, default) or bonsai (1-bit)
      ```

      ```powershell Windows (PowerShell) theme={null}
      $env:BONSAI_MODEL = "27B"
      $env:BONSAI_FAMILY = "ternary"
      ```
    </CodeGroup>

    Not sure which to pick? **Ternary is higher quality; 1-bit is smaller and faster.** The [model overview](/models/overview) has the numbers.
  </Step>

  <Step title="Run setup">
    <CodeGroup>
      ```bash macOS / Linux theme={null}
      ./setup.sh
      ```

      ```powershell Windows (PowerShell) theme={null}
      Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
      .\setup.ps1
      ```
    </CodeGroup>

    Re-running setup is safe: it skips steps that are already done. To download a different model later, change the environment variables and run `./scripts/download_models.sh` (or re-run setup).
  </Step>

  <Step title="Chat">
    <CodeGroup>
      ```bash macOS / Linux theme={null}
      ./scripts/run_llama.sh -p "What is the capital of France?"
      ```

      ```powershell Windows (PowerShell) theme={null}
      .\scripts\run_llama.ps1 -p "What is the capital of France?"
      ```
    </CodeGroup>

    On Apple Silicon you can also run through MLX, which is typically faster on M-series chips:

    ```bash theme={null}
    source .venv/bin/activate
    ./scripts/run_mlx.sh -p "What is the capital of France?"
    ```
  </Step>
</Steps>

## Start a server instead

Prefer a browser UI or an API endpoint? The repo ships an OpenAI-compatible server:

```bash theme={null}
./scripts/start_llama_server.sh    # API + web chat at http://localhost:8080
```

Most tools in [Integrations](/integrations/overview) connects to this endpoint. For a full ChatGPT-style interface, `./scripts/start_openwebui.sh` starts the server and [Open WebUI](/run/open-webui) together. See [Run the server](/run/server) for flags, sampling defaults, and API examples.

## What setup actually does

Even on a fresh machine, `setup.sh` / `setup.ps1`:

1. Installs system build tools (Xcode Command Line Tools on macOS, `build-essential` on Linux)
2. Installs [uv](https://docs.astral.sh/uv/) and creates a Python virtualenv in `.venv/`
3. Downloads the selected model from [Hugging Face](https://huggingface.co/prism-ml) into `models/`
4. Downloads pre-built [llama.cpp binaries](https://github.com/PrismML-Eng/llama.cpp/releases/tag/prism-b9570-0ad1dab) for your platform into `bin/`, or builds from source if none match
5. On Apple Silicon, builds MLX from source for GPU acceleration

Nothing is installed globally except the system build tools; deleting the `Bonsai-demo` directory removes everything else.

## Next steps

<CardGroup cols={2}>
  <Card title="Run the server" icon="server" href="/run/server">
    The OpenAI-compatible endpoint, its defaults, and API examples.
  </Card>

  <Card title="Connect your tools" icon="plug" href="/integrations/overview">
    OpenClaw, Hermes, Ollama, and anything OpenAI-compatible.
  </Card>

  <Card title="Pick the right model" icon="layer-group" href="/models/overview">
    Size, family, and format trade-offs with real numbers.
  </Card>

  <Card title="Troubleshooting" icon="wrench" href="/resources/troubleshooting">
    Port conflicts, slow generation, models that won't load.
  </Card>
</CardGroup>