Skip to main content
Open WebUI gives you a full agentic demo on top of local Bonsai 27B: chat with images, tool calling against live tools, and a database with a hidden story to investigate. One command configures everything — tools, model settings, and a demo database — with nothing to click through by hand.

​
Setup (once)

source .venv/bin/activate
uv pip install open-webui

​
Run

./scripts/start_openwebui.sh
This starts llama-server if it isn’t already running, seeds the demo (tools, model settings, demo database), and opens http://localhost:9090. First boot takes about a minute for database migrations; Ctrl+C stops everything it started. Prefer the MLX backend on a Mac? 
BONSAI_BACKEND=mlx ./scripts/start_openwebui.sh
Only one backend runs at a time — two resident 27B models is too heavy for most machines. The MLX backend is noticeably slower per token and has no cross-request prompt cache (each follow-up reprocesses the whole conversation, image tokens included), so multi-turn chats are slower than on llama.cpp. For interactive multi-turn use, stick with the default llama.cpp backend.

​
What to try

  • Vision — click the + in the message box, upload a photo or screenshot, and ask about it. Follow-up questions about the same image are near-instant on the llama.cpp backend (prompt cache).
  • Weather tool — “What’s the weather in Lisbon right now?”
  • Web fetch tool — “Fetch https://prismml.com and summarize it.”
  • MCP servers — Hugging Face Hub and DeepWiki are connected but off by default. Enable one from the tool menu, then try “Find trending vision-language models on Hugging Face.”
  • The agentic analyst — the demo ships a SQLite database of a fictional B2B company (2024–2026) with a hidden story in it. Ask: “Our EMEA numbers looked bad in the second half of 2025. Figure out what happened and whether it was preventable.” and watch it explore the schema, run queries, and piece the answer together.
  • Thinking — answers show a collapsible thought block. Turn thinking off for snappier replies with BONSAI_THINKING=0 ./scripts/start_openwebui.sh, or cap it per request.

​
How it works

  • The three built-in tools live in scripts/openwebui/ as plain Python Open WebUI “Tools” and are re-seeded on every start — edit them and restart to change the demo.
  • The demo database is generated into .openwebui/demo.db on first start.
  • Chats persist in .openwebui/ between runs; delete that directory for a factory-fresh demo.
  • Configuration comes from start_openwebui.sh env vars on every launch (auth disabled, single backend, background title/tag/follow-up generation off so the UI doesn’t keep the model busy after each reply).

​
Adding your own tools or MCP servers

Edit TOOL_SERVER_CONNECTIONS in scripts/start_openwebui.sh and restart — that script, not the admin panel, is the source of truth (the demo runs with persistent config disabled, so panel edits don’t survive a restart). Full guide, including the optional Brave Search MCP server: TOOLS.md in the demo repo.

Run the server

The raw OpenAI-compatible API underneath Open WebUI.

Bonsai 27B

The model this demo is built around: vision, tool calling, thinking.