Maintain the UVACompute hub (DO droplet) including disk cleanup, service deployment, and KubeVirt management. Use when hub has disk pressure, services need deployment, or KubeVirt is unhealthy.
The hub (uvacompute-ssh) runs on a DigitalOcean droplet with limited disk (8.6G).
ssh root@***REDACTED_IP*** "df -h /; kubectl get nodes; kubectl get pods -n kubevirt"
node.kubernetes.io/disk-pressure:NoSchedulessh root@***REDACTED_IP*** "
df -h /
kubectl describe node uvacompute-ssh | grep -A2 Taints
"
ssh root@***REDACTED_IP*** "
# Prune unused container images
crictl rmi --prune
# Vacuum old journal logs
journalctl --vacuum-size=100M
# Truncate large log files
truncate -s 0 /var/log/auth.log /var/log/kern.log /var/log/syslog
# Remove old rotated logs
rm -f /var/log/*.1 /var/log/*.gz
# Check result
df -h /
"
ssh root@***REDACTED_IP*** "kubectl taint nodes uvacompute-ssh node.kubernetes.io/disk-pressure-"
cd apps/vm-orchestration-service
ssh root@***REDACTED_IP*** "systemctl stop vm-orchestration"
make build-linux && make deploy-hub
The binary was likely copied but systemd wasn't reloaded:
ssh root@***REDACTED_IP*** "systemctl daemon-reload && systemctl restart vm-orchestration && systemctl status vm-orchestration"
ssh root@***REDACTED_IP*** "
systemctl status vm-orchestration
journalctl -u vm-orchestration -n 20 --no-pager
"
Usually disk pressure. Fix disk first, then:
ssh root@***REDACTED_IP*** "kubectl delete pods -n kubevirt --all"
Restart the virt-operator:
ssh root@***REDACTED_IP*** "
kubectl delete pod -n kubevirt -l kubevirt.io=virt-operator
sleep 15
kubectl get pods -n kubevirt
"
ssh root@***REDACTED_IP*** "kubectl delete pods -n kubevirt --field-selector=status.phase=Failed"
ssh root@***REDACTED_IP*** "kubectl get kubevirt -n kubevirt -o yaml | head -50"
# List VMs
ssh root@***REDACTED_IP*** "kubectl get vm -n uvacompute"
# Delete all VMs
ssh root@***REDACTED_IP*** "kubectl delete vm -n uvacompute --all"
# Delete specific VM
ssh root@***REDACTED_IP*** "kubectl delete vm VM_ID -n uvacompute"
| Method | Command |
|---|---|
| SSH (public IP) | ssh root@***REDACTED_IP*** |
| SSH (Tailscale) | Check tailscale status for current IP |
| File | Path |
|---|---|
| Service binary | /usr/local/bin/vm-orchestration |
| Systemd unit | /etc/systemd/system/vm-orchestration.service |
| Service logs | journalctl -u vm-orchestration |