Docker

Run CrabLLM in Docker with persistent storage and manage it with crabctl from the host.

Quick start

docker run -d \
  --name crabllm \
  -p 8080:8080 \
  -v ./crabllm.toml:/etc/crabllm/crabllm.toml \
  -e OPENAI_API_KEY \
  -e ANTHROPIC_API_KEY \
  ghcr.io/crabtalk/crabllm \
  serve --config /etc/crabllm/crabllm.toml --bind 0.0.0.0:8080

Configuration

Mount your crabllm.toml into the container. Environment variables referenced with ${VAR} in the config are passed through via -e:

listen = "0.0.0.0:8080"
admin_token = "${CRABLLM_ADMIN_TOKEN}"

[providers.openai]
kind = "openai"
api_key = "${OPENAI_API_KEY}"
models = ["gpt-4o", "gpt-4o-mini"]

[providers.anthropic]
kind = "anthropic"
api_key = "${ANTHROPIC_API_KEY}"
models = ["claude-sonnet-4-20250514"]

[storage]
kind = "sqlite"
path = "/data/crabllm.db"

[extensions.cache]
ttl = 300

[extensions.rate_limit]
rpm = 60
tpm = 100000

Persistent storage

For dynamic keys and providers to survive container restarts, use SQLite with a volume:

docker run -d \
  --name crabllm \
  -p 8080:8080 \
  -v ./crabllm.toml:/etc/crabllm/crabllm.toml \
  -v crabllm-data:/data \
  -e OPENAI_API_KEY \
  -e CRABLLM_ADMIN_TOKEN \
  ghcr.io/crabtalk/crabllm \
  serve --config /etc/crabllm/crabllm.toml --bind 0.0.0.0:8080

Managing with crabctl

Install crabctl on your host machine:

cargo install crabctl

Point it at the running container:

export CRABCTL_URL=http://localhost:8080
export CRABCTL_TOKEN=$CRABLLM_ADMIN_TOKEN

Now manage providers and keys without restarting the container:

# Add a new provider at runtime
crabctl providers create groq \
  --kind openai \
  --api-key "$GROQ_API_KEY" \
  --base-url "https://api.groq.com/openai/v1" \
  --models llama-3.3-70b-versatile

# Create a rate-limited key for a team
crabctl keys create team-backend \
  --models gpt-4o,claude-sonnet-4-20250514 \
  --rpm 120 --tpm 200000

# Check usage
crabctl usage --key team-backend

# View budget status
crabctl budget

# Query audit logs
crabctl logs --limit 20

Dynamic providers and keys are stored in SQLite and persist across container restarts. See Management for the full CLI and API reference.

Docker Compose

services:
  crabllm:
    image: ghcr.io/crabtalk/crabllm
    command: serve --config /etc/crabllm/crabllm.toml --bind 0.0.0.0:8080
    ports:
      - "8080:8080"
    volumes:
      - ./crabllm.toml:/etc/crabllm/crabllm.toml
      - crabllm-data:/data
    environment:
      - OPENAI_API_KEY
      - ANTHROPIC_API_KEY
      - CRABLLM_ADMIN_TOKEN

volumes:
  crabllm-data:

Health check

curl http://localhost:8080/health

OpenAPI docs

With openapi = true (the default), browse the interactive API docs at http://localhost:8080/docs.