Credential Rotation¶

Rotate credentials when they may have been exposed, or on a regular 90-day schedule.

PostgreSQL User Passwords¶

Step 1: Change passwords¶

docker exec -it $(docker ps -qf name=data_postgres) psql -U postgres

ALTER USER postgres WITH PASSWORD 'new_secure_password';
ALTER USER recipicity_production_owner WITH PASSWORD 'new_secure_password';
ALTER USER recipicity_staging_owner WITH PASSWORD 'new_secure_password';
-- Generate passwords with: openssl rand -base64 32

Step 2: Extract new SCRAM hashes¶

SELECT usename, passwd FROM pg_shadow
WHERE usename IN ('postgres', 'recipicity_production_owner', 'recipicity_staging_owner');

Step 3: Update PgBouncer userlist¶

Create /tmp/pgbouncer-userlist.txt with format:

"username" "SCRAM-SHA-256$iterations:salt$StoredKey:ServerKey"

docker config rm pgbouncer-userlist
docker config create pgbouncer-userlist /tmp/pgbouncer-userlist.txt

Step 4: Update DATABASE_URL secrets¶

# Create new version of secrets
echo "postgresql://recipicity_production_owner:NEW_PASS@pgbouncer:6432/recipicity_production" \
  | docker secret create recipicity_production_database_url_v2 -

# Update stack file to reference _v2 secret name
# Then redeploy

Step 5: Redeploy¶

docker stack deploy -c /opt/swarm/stacks/data/stack.data.yml data
docker stack deploy -c /opt/swarm/stacks/recipicity-production/stack.recipicity-production.yml app-recipicity-production
docker stack deploy -c /opt/swarm/stacks/app-recipicity/stack.recipicity.yml app-recipicity-staging

Step 6: Verify¶

# Check services are healthy
docker service ls

# Check for auth errors
docker service logs app-recipicity-production_api --tail=50 | grep -i "auth\|password\|failed"

# Test database connection
docker exec -it $(docker ps -qf name=data_postgres) psql -U postgres -c "SELECT 1"

Redis Password¶

# Generate new password
NEW_PASS=$(openssl rand -base64 32)

# Update Redis config (inside container or via config)
docker exec -it $(docker ps -qf name=data_redis) redis-cli CONFIG SET requirepass "$NEW_PASS"

# Create new secret
echo "$NEW_PASS" | docker secret create redis_password_v2 -

# Update REDIS_URL secrets for applications
echo "redis://:${NEW_PASS}@data_redis:6379" | docker secret create recipicity_production_redis_url_v2 -

# Update stack files and redeploy

Admin User Password¶

# Generate secure password
NEW_PASS=$(openssl rand -base64 32)
echo "New admin password: $NEW_PASS"

# Update in database
docker exec -it $(docker ps -qf name=data_postgres) psql -U postgres -d recipicity_production -c \
  "UPDATE admin_users SET password_hash = crypt('$NEW_PASS', gen_salt('bf', 12)) WHERE email = 'admin@recipicity.com';"

# Test login
curl -X POST https://recipicity.com/api/admin/auth/login \
  -H "Content-Type: application/json" \
  -d "{\"email\":\"admin@recipicity.com\",\"password\":\"$NEW_PASS\"}"

Cloudflare API Token (DNS-01 Challenge)¶

Secret name: cf_dns_api_token Scope: jlwaller.com only (DNS Zone edit permissions) Used by: Traefik letsencrypt-dns certresolver for internal-only domains (apps.jlwaller.com, docs.apps.jlwaller.com)

Rotation¶

# 1. Generate new API token in Cloudflare dashboard
#    Zone: jlwaller.com -> DNS -> Edit permissions

# 2. Remove old secret (Traefik must be stopped first)
docker service scale edge_traefik=0
docker secret rm cf_dns_api_token

# 3. Create new secret
echo -n "NEW_TOKEN_HERE" | docker secret create cf_dns_api_token -

# 4. Restart Traefik
docker service scale edge_traefik=1
# Or redeploy:
docker stack deploy -c /opt/swarm/stacks/edge/stack.traefik.yml edge

# 5. Monitor cert renewal
docker exec $(docker ps -qf name=edge_traefik) tail -f /logs/traefik.log | grep letsencrypt-dns

Notes¶

This token is used exclusively for Let's Encrypt DNS-01 challenges
Traefik reads it via CF_DNS_API_TOKEN_FILE=/run/secrets/cf_dns_api_token
Certificates auto-renew 30 days before expiry (90-day certs)
No manual intervention needed for renewals
ACME data stored in traefik_letsencrypt volume at /letsencrypt/acme-dns.json

Best Practices¶

Generate passwords with openssl rand -base64 32 (minimum 32 characters)
Store new passwords in a secure password manager immediately
Never log or echo passwords to shell history
Use Docker secrets exclusively (never hardcode in stack files)
Schedule rotation every 90 days
Test all services after rotation before considering it complete