Managing Remote Servers

When to Use

• User wants to check server status (Docker containers)
• User needs to deploy new code (MUST use GitHub Actions)
• User wants to run database migrations on production (Inside Docker)
• User needs to check remote logs (Docker logs)

Workflow: Automated Deployment (GitHub Actions)

Protocol: This is the ONLY authorized method for deployment to ensure memory safety and consistency.

1. Prerequisite: GitHub Secrets

The current deploy.yml requires these specific secrets:

• DOCKERHUB_USERNAME: Docker Hub profile name.
• DOCKERHUB_TOKEN: Personal Access Token.
• VPS_HOST: Server IP (82.112.253.29)
• VPS_USER: deploy
• VPS_SSH_KEY: SSH Private Key (Password auth is NOT enabled in current workflow)

2. Triggering Deployment

Push to main. The workflow handles:

1. Building & Pushing images to Docker Hub.
2. SSH-ing into VPS to pull images and restart containers.
3. Restarting Nginx Proxy to prevent 502 Bad Gateway errors.

Workflow: Manual Deployment (Emergency Only)

WARNING: Manual builds on the VPS often cause Out-Of-Memory (OOM) crashes. Only proceed if GitHub Actions is broken.

1. Laapak Report System (VPS Build)

[!CAUTION] DO NOT RUN THIS unless absolutely necessary.

# Full manual update
sshpass -p "0000" ssh [email protected] "cd /home/deploy/laapak-projects/reports && git pull origin main && docker compose -f remote-docker-compose.yml up -d --build"

• Note: The remote-docker-compose.yml still contains build blocks to allow this manual fallback.

2. Central Infrastructure (Nginx / Shared DBs)

sshpass -p "0000" ssh [email protected] "cd /home/deploy/fz-projects/system/docker-infra && docker compose up -d"

Common Commands

1. Monitoring & Logs

# Check all container status
docker ps

# Check logs for Laapak Backend
docker logs --tail 100 -f report-system

# Check logs for Laapak React Frontend
docker logs --tail 100 -f laapak-frontend-react

# System resources
docker stats --no-stream
df -h

2. Service Restart (No Rebuild)

# Restart reports stack
cd /home/deploy/laapak-projects/reports && docker compose -f remote-docker-compose.yml restart

3. Service Mapping

• FixZone Main: system.fixzzone.com -> fz-frontend (80), fz-system (4000)
• Evolution WhatsApp API: wa.fixzzone.com -> evolution-api (8080)
• n8n Automation: n8n-auto.fixzzone.com -> n8n (5678)
• Laapak Reports Backend: 82.112.253.29:3001 (report-system)
• Laapak Reports Frontend: 82.112.253.29:3000 (laapak-frontend-react)
• ETA Invoicing API: eta.laapak.com -> invoicing-api (Port 3003 on host, 3000 internal). Nginx config at /home/deploy/fz-projects/system/docker-infra/nginx/conf.d/eta_laapak.conf
• Database: mysql-db (MySQL 8.0), postgres-db (PostgreSQL)

Security & Best Practices

• Password Prompt: I will prompt for the password (0000) if needed.
• Root Password: The MySQL root password is rootpassword (or 0000 inside docker-compose).
• Database Migrations: Run inside fz-system: docker exec -it fz-system npm run migrate.

Accessing Secured Database (MySQL)

Since port 3306 is bound to localhost only:

1. Do NOT connect to 82.112.253.29:3306 directly.
2. Use SSH Tunnel:

   ssh -L 3306:127.0.0.1:3306 [email protected]
   

3. Connect Client: Host: 127.0.0.1, Port: 3306, User: root, Pass: 0000.

Troubleshooting & Common Pitfalls

1. "Code Not Updating" (Stale Containers)

If you deploy but changes don't appear:

• Diagnosis: Check if the running container has the old code.

  docker exec fz-system grep "new_string" /app/path/to/file.js
  

• Cause: Docker build might have failed silently, or you are looking at the wrong directory.
• Fix: Rebuild explicitly: docker compose -f remote-docker-compose.yml up -d --build --no-cache.
• Note: Ensure build contexts are correctly defined in the compose file.

2. Nginx Proxy Issues (Docker-managed)

• Context: Nginx is now a Docker container (nginx-proxy).
• Configs: Found in /home/deploy/fz-projects/system/docker-infra/nginx/conf.d/.
• Restart: docker compose restart nginx-proxy.
• Refresh DNS/Upstreams: docker exec nginx-proxy nginx -s reload.
• Important: Host-level Nginx must remain DISABLED to avoid port 80/443 conflicts.

3. PM2 / Host Services

• Note: PM2 and Host-level Nginx have been removed/uninstalled. All logic must reside in Docker containers.

3. SQL Errors (500 Internal Server Error)

• Constraint: The production database runs in Strict Mode (only_full_group_by).
• Symptom: Queries working locally fail on production with Expression #X of SELECT list is not in GROUP BY clause.
• Fix: Ensure all non-aggregated columns in the SELECT list are present in the GROUP BY clause.

Standard Maintenance Procedures

1. File Organization

• Backups: Store all SQL dumps in /home/deploy/backups_sql/. Do not clutter the home root.
• Periodicity: Run a cleanup script monthly to archive old backups.

2. Security Hardening

• Database Ports: limit 3306 access.
  • Best Practice: Do not map 3306:3306 in docker-compose.yml. Use 127.0.0.1:3306:3306 if local access is needed, or rely on Docker network/SSH Tunnels.
• Firewall: Ensure ufw limits SSH to known IPs if possible.

3. Debugging Health Checks

• Problem: Alpine images often resolve localhost to IPv6 (::1), causing wget connection refused errors if the service listens on IPv4 only.
• Solution: Always use http://127.0.0.1/ in HEALTHCHECK commands for Alpine-based containers.

High-CPU Spike Management

1. Diagnosis

If CPU is pinned at 100%:

1. Run top -b -n 1 to find the process name.
2. If it's a Docker process (containerd, node, mysql), check docker stats.
3. If it's a host process (e.g., systemp), it might be malware or a rogue service.

2. Mandatory Resource Limits

Every service in docker-compose.yml MUST have resource limits defined to protect the host: