2.9 KiB
Encryption key rotation (production)
This runbook describes how to rotate ENCRYPTION_KEY without data loss. The application encrypts user PII, TOTP secrets, hot-wallet keys, and other sensitive columns at rest.
Prerequisites
- Maintenance window (API can stay up; rotation is I/O heavy on large tables).
- Current and new 32-byte keys (base64 or hex per your
ENCRYPTION_KEYformat). - Database backup before starting.
- Shell access to the API host or a job runner with
DATABASE_URLand keys.
Settings
| Variable | Purpose |
|---|---|
ENCRYPTION_KEY |
Active encryption key (after rotation: the new key) |
ENCRYPTION_KEY_PREVIOUS |
Previous key; required during rotation so decrypt() can read legacy ciphertext |
ENCRYPTION_KEY_ROTATION_REMIND_DAYS |
Optional; days before daily reminder (default 90) |
The API records encryption_key_rotated_at in system_settings when rotation completes successfully.
Rotation steps
-
Generate a new key (example):
node -e "console.log(require('crypto').randomBytes(32).toString('base64'))" -
Set environment on the rotation job (not production API yet):
export ENCRYPTION_KEY_PREVIOUS="<current-key>" export ENCRYPTION_KEY="<new-key>" export DATABASE_URL="<production-database-url>" -
Run the rotation script from
backend/:npm run rotate-encryption-keyThis re-encrypts all managed columns and updates
encryption_key_rotated_at. -
Deploy the new
ENCRYPTION_KEYto all API/worker processes. RemoveENCRYPTION_KEY_PREVIOUSfrom runtime env after verification. -
Verify:
- Login, 2FA, withdrawals, and admin user detail (email decrypt).
- Check logs for decrypt errors.
- Confirm
encryption_key_rotated_atinsystem_settings.
Scheduled reminder
On boot, the API runs a daily check (encryptionRotationReminderService). If the key is older than ENCRYPTION_KEY_ROTATION_REMIND_DAYS, it logs a warning and creates an admin notification.
Rotation itself is manual (script above). Do not auto-rotate keys without updating deployment secrets in the same change window.
Cron example (optional)
Run rotation from a secure CI job or cron only when keys are pre-staged:
# Example: quarterly rotation at 03:00 UTC on the 1st — adjust to your policy
0 3 1 */3 * cd /opt/games-app/backend && ENCRYPTION_KEY_PREVIOUS=... ENCRYPTION_KEY=... npm run rotate-encryption-key >> /var/log/encryption-rotation.log 2>&1
After the job succeeds, update the orchestrator secrets and redeploy before removing ENCRYPTION_KEY_PREVIOUS.
Rollback
If rotation fails mid-run:
- Stop the script.
- Restore
ENCRYPTION_KEYto the previous value across all services. - Restore database from backup if partial writes occurred.
- Investigate logs from
rotate-encryption-keybefore retrying.