A Single vllm serve Isn't Production.

The Production Topology

Docker Compose — The Skeleton

This gets you past "it works on my machine." One file, two services, GPU passthrough. The manual adds health checks, resource limits, and multi-node config.

# docker-compose.yml — production skeleton
version: '3.8'
services:
  vllm:
    image: vllm/vllm-openai:latest
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    command: >
      --model glm-5-2/fp8
      --tensor-parallel-size 8
      --max-model-len 131072
      --gpu-memory-utilization 0.90
      --enable-prefix-caching
    ports:
      - "8000:8000"
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    ipc: host
    shm_size: 8gb

  nginx:
    image: nginx:alpine
    ports:
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./certs:/etc/nginx/certs:ro
    depends_on:
      - vllm

This skeleton runs. But it's missing health checks (Docker will mark vLLM "running" before the model loads), resource limits (OOM kills the wrong container), and the systemd unit that survives a reboot. The manual covers all three.

Nginx — The Gateway

Your model speaks OpenAI-compatible API. Clients shouldn't talk to it directly. nginx adds the missing production layer.

# nginx.conf — minimal production gateway
upstream glm_backend {
    server vllm:8000 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

server {
    listen 443 ssl;
    server_name glm-api.yourdomain.com;

    ssl_certificate     /etc/nginx/certs/fullchain.pem;
    ssl_certificate_key /etc/nginx/certs/privkey.pem;

    # Rate limit: 60 requests per minute per IP
    limit_req_zone $binary_remote_addr zone=glm_limit:10m rate=60r/m;
    limit_req zone=glm_limit burst=20 nodelay;

    # Max 128K context — prevent OOM from oversized prompts
    client_max_body_size 2m;

    location /v1/ {
        proxy_pass http://glm_backend;
        proxy_set_header Host $host;
        proxy_read_timeout 300s;

        # API key validation (JWT example — manual has full auth setup)
        auth_request /auth;
    }

    location /health {
        proxy_pass http://glm_backend/health;
        access_log off;
    }

    location = /auth {
        internal;
        proxy_pass http://auth-service/validate;
    }
}

rate limiting + TLS + auth endpoint — the three things that separate a demo from a production API. The manual adds JWT token generation, Cloudflare Tunnel as an alternative, and multi-user queue management for burst traffic.

What the Manual Adds (Ch.1)