Nginx

sudo nginx -t                    # validate config; ALWAYS before reload
sudo systemctl reload nginx      # graceful reload (preserves connections)
sudo systemctl restart nginx     # hard restart (drops connections)
sudo tail -f /var/log/nginx/error.log /var/log/nginx/access.log
sudo nginx -T 2>/dev/null | less # dump resolved config (useful for debugging)

Never skip nginx -t before reload. An invalid config that reaches a reload will return an error; a restart with bad config leaves nginx off. -t catches both cases cheaply.

Adding a new project vhost

1. Write the vhost in the project repo (e.g. myproj/ops/nginx/myproj.example.com.conf).
2. Install into nginx — always via the repo's ops/nginx/install.sh helper, never a bare install -m 0644. The helper re-shapes the vhost through certbot after copy, which preserves the :443 block. A bare copy silently overwrites certbot's edits and takes HTTPS offline (vs-15s incident — root cause of the first guidebook deploy outage). The wrapper pattern:

# In ops/nginx/install.sh (excerpt):
install -m 0644 ./ops/nginx/${HOST}.conf /etc/nginx/sites-available/
ln -sf /etc/nginx/sites-available/${HOST}.conf \
    /etc/nginx/sites-enabled/${HOST}.conf
certbot --nginx -d ${HOST} --non-interactive --expand --redirect \
    --agree-tos --register-unsafely-without-email
nginx -t && systemctl reload nginx

When an ops/<service>/install.sh edits location blocks in the shared vhost template, the operator must re-run sudo ./ops/nginx/install.sh (or the service's install.sh must call it) after the template change lands.

3. If it's a new public hostname, either:
   • DNS the host at this server's IP first, THEN
   • sudo certbot --nginx -d myproj.example.com — certbot edits the vhost in place to add the listen 443 ssl block + cert paths.

See the "TLS" section below for the cert flow.

Minimal vhost template

HTTP-only for initial DNS propagation testing:

server {
    listen 80;
    listen [::]:80;
    server_name myproj.example.com;

    location / {
        proxy_pass http://127.0.0.1:<BACKEND_PORT>;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
        proxy_buffering off;
    }
}

Path-based routing (one host, multiple backends):

server {
    listen 80;
    server_name myproj.example.com;

    # Public download — one narrow path, rewrite-proxy to backend
    location = /client.zip {
        proxy_pass http://127.0.0.1:5000/instances/myproj/binaries/SS14.Client.zip;
    }

    # Admin API — gated by the backend's own auth; HTTPS gives wire protection
    location /admin/ {
        proxy_pass http://127.0.0.1:5000/;
    }

    # Block anything else
    location / {
        return 404;
    }
}

TLS via certbot

# One-time install
sudo apt-get install -y certbot python3-certbot-nginx

# Issue + install cert for an HTTP-configured vhost
sudo certbot --nginx -d myproj.example.com

# Test renewal (dry-run)
sudo certbot renew --dry-run

Certbot installs a systemd timer (certbot.timer) that renews certs twice daily automatically. The timer status is idempotent — safe to