Recurring Tasks
Maintenance tasks that should be performed on a regular schedule.
Weekly
| Task |
Command |
What to check |
| Review Grafana alerts |
Grafana UI → Alerting |
Any firing or silenced alerts that need attention |
| Check backup integrity |
ls -lh /opt/docker/backups/daily/ |
Recent backup exists, reasonable file size |
| Review container health |
docker ps -a |
No containers in Exited or Restarting state |
Monthly
| Task |
Command |
What to check |
| Disk usage audit |
df -h / && docker system df |
Disk usage below 70%, prune if needed |
| Review Docker images |
docker image ls |
Remove unused images |
| Check SSL cert expiry |
Grafana → blackbox SSL panel |
All certs renewing properly (> 30 days remaining) |
| Review fail2ban |
sudo fail2ban-client status |
Active jails, recent bans |
| PostgreSQL vacuum |
Check postgresql Grafana dashboard |
Dead tuple count, last autovacuum time |
| Check Loki storage |
du -sh /opt/docker/monitoring/loki/ |
Volume not growing unexpectedly (expect ~1-2 GB with 7d retention) |
Quarterly
| Task |
Command |
What to check |
| Full disaster recovery test |
cd /opt/docker/aether/repo && ./test-dr.sh |
Validates DR procedure non-destructively (see DR doc) |
| Review monitoring thresholds |
monitoring/prometheus/alerts/ |
Thresholds still appropriate for current load |
| Security updates |
sudo apt update && sudo apt list --upgradable |
Apply OS security patches |
| Review Docker Compose versions |
docker-compose.yml files |
Upgrade images to latest stable |
| Rotate secrets |
See procedure below |
All secrets updated and encrypted |
How to rotate secrets
- Identify which secret to rotate (database password, API key, SMTP credentials, etc.)
- Update the plaintext value on the server:
# Example: edit the production env file
sudo nano /opt/docker/aletheia/envs/.env.prod
- If rotating a database password, also update it in PostgreSQL:
docker exec shared_postgres psql -U admin -c \
"ALTER USER aletheia_prod WITH PASSWORD 'new-password';"
- Restart the affected service so it picks up the new credentials:
cd /opt/docker/aletheia/repo && make restart ENV=prod
- Verify the service is healthy after restart
- Encrypt the updated secrets back into the repo:
cd /opt/docker/aether/repo && make encrypt
- Commit and push the updated
.enc files
Warning
Always restart and verify the service before encrypting. If the new password
breaks something, the old plaintext value is still on disk until you encrypt.
Annual
| Task |
What to check |
| Review server sizing |
Current vs. projected resource needs |
| SSL certificate authority |
Certbot / Let's Encrypt still appropriate |
| Backup retention policy |
Current retention meets compliance needs |
| Access audit |
Remove stale user accounts, review SSH keys |