Run easily a LLM in local as backend for development along with a Chat UI.

Using Ollama and Open WebUI.

All installed via docker compose.

• docker compose (recommended V2).
• nvidia-container-toolkit installed if you have gpu.

1. Configure .env.

• COMPOSE_PROFILES. gpu (you need nvidia-container-toolkit installed) or cpu.

2. Run docker compose.

docker compose up -d

• UI: http://localhost:3003
• OpenAI API: http://localhost:11434

Common docker compose commands useful in daily execution:

1. Download a ollama model from cli:

docker compose exec ollama-gpu ollama pull <model_name>

2. Stop.

docker compose stop

3. Show logs.

docker compose logs -f

4. Remove all.

docker compose down -v

5. Update all the containers.

docker compose up --build -d

• Simple. Using your local LLM as Open AI replacement.
• Multi MCP client. Using a multi MCP client with your local LLM.