Secrets management: Vault, KMS and secure rotation

Secrets management covers service passwords, tokens, API keys, certificates and cryptographic material used by applications. A secret committed to a repository, container layer or public pipeline can remain copied for years even after it disappears from the current code.

A secure platform does more than store values. It issues them to the correct identity for a short period, records use, rotates without downtime and supports rapid revocation after an incident.

Vault, KMS and secret managers serve different purposes

A KMS controls encryption keys and cryptographic operations. Applications can decrypt a data key without receiving the master key. A secret manager stores values such as database passwords or tokens. Vault-style systems can also generate dynamic, short-lived credentials.

Do not encrypt every value with one static key stored beside the application. Use envelope encryption, separate environment boundaries and policies based on workload identity.

Identity before secret retrieval

Applications should authenticate through platform mechanisms: cloud roles, Kubernetes ServiceAccounts, workload identity or certificates. Avoid a permanent “bootstrap secret” required to fetch all other secrets.

Each workload receives only the values it needs. A reporting service should not read production administration credentials, and a test pipeline does not need production keys.

Short TTL and dynamic credentials

Dynamic credentials are created on demand, bound to one service and expire automatically. A stolen token valid for 30 minutes has a smaller blast radius than a database password unchanged for five years.

For static secrets, define an owner, rotation deadline, consumers and emergency procedure. Rotation should be automated and testable without manual access to dozens of hosts.

Zero-downtime rotation

Use an overlap window:

1. create the new secret version;
2. expose it to authorised applications;
3. confirm every instance has switched;
4. revoke the old version;
5. monitor errors and retain a controlled rollback.

If consumers cache a secret for hours, changing the central store does not complete rotation. Use refresh signals or a short cache TTL.

CI/CD and container rules

Never place secrets in a Dockerfile, image layer, artefact, build log or committed .env file. CI should obtain a token through identity federation rather than a long-lived cloud key.

Secret scanning is necessary, but a discovered key must be revoked immediately. Removing it from Git history does not undo prior exposure. Connect this process with DevSecOps controls.

Monitoring and response

Log identity, secret name, operation, time and outcome—never the value. Alert on bulk retrieval, new environments, administrative access and attempts to read another tenant’s secret.

After a leak:

• revoke the secret and active sessions;
• identify systems that may have copied it;
• review use since the earliest possible exposure;
• rotate derived credentials;
• fix the root cause such as verbose logging or over-broad roles;
• preserve evidence for incident response.

Secrets-management checklist

• Central store instead of values in code.
• Workload identity without static bootstrap credentials.
• Least-privilege policy per application and environment.
• Encryption, versioning and audit trails.
• Short TTL or automated rotation.
• Tested overlap and rollback.
• Redaction in logs, traces and errors.
• Repository and image scanning.
• Emergency-revocation procedure.

A secure vault cannot compensate for a poor permission model. The strongest design combines workload identity, least privilege, short lifetime and measurable rotation.

---

Rotation without outage or false success

Safe rotation has an overlap phase: accept new and old secrets for a short controlled interval, migrate clients, then revoke the old one. Monitor use of both versions because generating a key does not mean applications moved. Emergency access must work when the primary vault or control plane is unavailable.

Inventory each secret with its owner, consumers, scope and maximum lifetime. Prefer dynamic credentials and workload identity over copied tokens. Logs must omit secret values but show who read one and for what purpose. Test revocation on a canary and scan repositories and images after rotation.

Sources: NIST SP 800-57, OWASP Secrets Management, NIST SP 800-190.