Cryptography, Key Management and Secrets

Introduction and Context

Cloud computing moves data across networks, stores it in shared infrastructure, and exposes services through programmable interfaces. These characteristics create substantial security benefits, but they also increase the importance of confidentiality, integrity, authenticity, and controlled access. Cryptography provides the technical foundation for many of these protections.

Cryptography is not simply the act of “encrypting everything.” Effective cloud protection requires selecting appropriate cryptographic controls, managing keys throughout their life cycle, protecting certificates and secrets, and understanding what encryption can and cannot achieve. A correctly selected algorithm is of little value if its key is exposed, if access permissions are excessive, or if encrypted data is decrypted unnecessarily in an insecure environment.

This chapter examines encryption in transit, at rest, and, where feasible, in use; distinguishes encryption from hashing and tokenization; and introduces practical approaches to key management, certificates, hardware security modules (HSMs), cloud key-management services (KMSs), and secret-management systems.

Learning Objectives

By the end of this chapter, you should be able to:

• Explain the purposes of confidentiality, integrity, authentication, and non-repudiation in cloud security.
• Distinguish symmetric encryption, asymmetric cryptography, hashing, digital signatures, and tokenization.
• Describe encryption in transit, at rest, and in use.
• Explain how certificates and public-key infrastructure support trusted communication.
• Describe key life-cycle management, rotation, revocation, backup, and destruction.
• Compare cloud KMSs, HSMs, and secrets-management services.
• Identify common implementation failures and propose appropriate controls.
• Evaluate governance, privacy, compliance, and operational issues surrounding cryptographic protection.

1. What Cryptography Protects

Cryptography transforms information so that only authorized parties, or parties able to verify a protected message, can use it. Four security objectives are especially important:

Cryptography does not replace identity and access management. Encryption may protect a database file, but an authorized application can still disclose the plaintext. Similarly, a valid certificate does not prove that an application is safe; it primarily binds a public key to an identity under a trust model.

Plaintext, ciphertext, keys, and algorithms

The original readable information is plaintext. Encryption transforms plaintext into ciphertext using an algorithm and a key. Decryption uses a key and an appropriate algorithm to recover plaintext. Modern security depends primarily on protecting keys, not on hiding algorithms. This principle is associated with Kerckhoffs’s principle: a system should remain secure even if its design is public, provided the secret keys remain protected.

A cryptographic key is not the same as a password. Keys should normally be generated using a cryptographically secure random-number generator, have an appropriate length and purpose, and be stored and used under controlled conditions. Passwords are human-created secrets and should generally be processed using password-specific, deliberately expensive functions rather than ordinary encryption or fast hashing.

2. Major Cryptographic Techniques

2.1 Symmetric encryption

Symmetric encryption uses the same secret key, or closely related secret material, for encryption and decryption. It is efficient and is therefore used for large volumes of data and for most data-plane encryption.

Examples include AES and ChaCha20. Modern authenticated encryption modes, such as AES-GCM and ChaCha20-Poly1305, provide confidentiality and integrity together. They produce an authentication tag that allows the recipient to detect alteration.

A nonce or initialization vector (IV) is often required. For many modes, the nonce need not be secret, but it must be unique under the same key. Reusing a nonce inappropriately, especially with counter-based authenticated encryption, can seriously compromise security. Systems should normally generate nonces through a reliable library rather than constructing them manually.

2.2 Asymmetric cryptography

Asymmetric cryptography uses a key pair: a public key and a private key. The public key may be distributed, whereas the private key must be protected. Asymmetric operations support encryption, key establishment, authentication, and digital signatures, but they are generally slower than symmetric encryption.

In practice, cloud protocols use hybrid cryptography. An asymmetric protocol authenticates parties and establishes a temporary symmetric session key; symmetric encryption then protects the data stream. This approach combines the scalability of symmetric encryption with the identity and key-establishment capabilities of public-key cryptography.

Examples include RSA, elliptic-curve cryptography (ECC), and elliptic-curve Diffie–Hellman (ECDH). The appropriate algorithm and parameters depend on current standards, platform support, threat models, and organizational policy. Deprecated algorithms and obsolete protocol versions should not be selected merely because they remain technically available.

2.3 Hashing

A cryptographic hash function maps input data of arbitrary length to a fixed-length digest. A secure hash should make it computationally infeasible to recover the input from the digest, find a different input with the same digest, or deliberately find two inputs with the same digest.

Hashing is one-way in normal use; it is not encryption. A hash does not use a reversible decryption key. Hashes are useful for integrity checks, content addressing, digital signatures, and password verification. For passwords, a salt—a unique random value stored with the password record—must be used, together with a password hashing or key-derivation function such as Argon2id, scrypt, or an appropriately configured PBKDF2 implementation. Fast general-purpose hashes are unsuitable by themselves for password storage because attackers can test guesses rapidly.

A hash alone does not authenticate its source. An attacker who can replace both a file and its publicly available hash can produce a new matching hash. A keyed-hash message authentication code (HMAC) or digital signature is required when the verifier must have confidence in the sender or in a secret-controlled process.

2.4 Message authentication codes and digital signatures

A MAC uses a shared secret to generate an integrity and authentication value. HMAC is a common MAC construction. Both sender and verifier possess the secret, so a MAC demonstrates possession of that shared secret but does not independently prove which party created a message.

A digital signature uses a private key to sign data and a corresponding public key to verify it. Signatures support integrity, origin authentication, and, when supported by appropriate evidence and governance, stronger attribution than a shared MAC. They do not automatically provide confidentiality.

2.5 Tokenization

Tokenization replaces a sensitive value with a token that has no exploitable mathematical relationship to the original value. A tokenization service retains a protected mapping between the token and the original value. For example, a payment-card number may be replaced by a token used by an application that does not need to handle the card number directly.

Tokenization differs from encryption. Encrypted data can be decrypted with a key; a token is resolved through a controlled tokenization system. Tokenization can reduce the scope of some compliance obligations, but it does not remove the need to secure the mapping service, access controls, logs, and recovery processes. Whether a token is considered personal or sensitive data depends on the legal and operational context.

3. Encryption in Transit, at Rest, and in Use

3.1 Encryption in transit

Encryption in transit protects data while it travels between clients, services, regions, administrators, and external systems. Transport Layer Security (TLS) is the dominant protocol for web and many service-to-service communications. Secure configuration includes current protocol versions, trusted certificate validation, strong cipher suites, protection of private keys, and correct hostname verification.

TLS normally provides server authentication through a certificate. Mutual TLS (mTLS) additionally requires the client to present a certificate, which can be useful for service-to-service identity. Encryption in transit should be applied to internal traffic as well as public traffic because internal networks are not automatically trusted in cloud environments.

Common failures include disabling certificate validation, accepting self-signed certificates without a controlled trust process, allowing obsolete protocols, terminating TLS without protecting the next network segment, and transmitting credentials through unencrypted administrative channels.

3.2 Encryption at rest

Encryption at rest protects stored data, including databases, object storage, block volumes, file systems, snapshots, logs, backups, and removable media. It reduces the consequences of stolen media, unauthorized storage access, or an improperly disposed disk. It does not prevent an authorized application, administrator, or compromised workload from reading data after decryption.

Cloud providers often offer default encryption using provider-managed keys. This may be appropriate for low-risk workloads, but organizations may require customer-managed keys for stronger separation of duties, independent rotation schedules, access auditing, or regulatory reasons. Encryption should be combined with least-privilege policies, network controls, data classification, retention limits, and monitoring.

Envelope encryption is widely used. A data-encryption key (DEK) encrypts the actual data. A key-encryption key (KEK), often held or controlled through a KMS, encrypts the DEK. The encrypted DEK is stored with the ciphertext, while the KEK remains protected by the key-management system. This avoids using a single high-value key directly on every data object and permits efficient data-key generation and rotation.

Here, P is plaintext, C is ciphertext, DEK is a data-encryption key, KEK is a key-encryption key, and E_DEK is the encrypted data key. The notation represents a conceptual envelope-encryption process; exact formats and authenticated metadata depend on the implementation.

3.3 Encryption in use

Encryption in use protects data while it is being processed. Traditional applications must usually decrypt data in memory, creating exposure to a compromised process, privileged administrator, debugging interface, or memory dump.

Approaches include:

• Confidential computing: hardware-backed trusted execution environments isolate selected computations and attest to the environment.
• Trusted execution environments (TEEs): protected hardware regions process data with restrictions on host visibility.
• Homomorphic encryption: permits selected computations on ciphertext, but remains computationally expensive and limited for many workloads.
• Secure multi-party computation: enables parties to compute jointly without directly revealing their inputs, often with significant performance and design costs.
• Application-level techniques: minimize plaintext exposure by filtering, masking, partitioning, or using privacy-preserving analytics.

Encryption in use is not a universal replacement for conventional encryption. It introduces performance, compatibility, attestation, key-provisioning, and operational challenges. It is most valuable when the threat model includes the host platform or when multiple parties need to process sensitive information without receiving unrestricted plaintext.

4. Certificates and Public-Key Infrastructure

A digital certificate binds an identity, such as a domain name or service identity, to a public key. A certificate authority (CA) signs the certificate. Clients validate a chain from the certificate through intermediate certificates to a trusted root CA. Validation commonly includes signature verification, validity dates, subject and alternative names, key-usage constraints, and revocation or status mechanisms where applicable.

Public-key infrastructure (PKI) includes policies, certificate authorities, registration processes, certificate issuance, renewal, revocation, repositories, and audit procedures. A certificate is not a secret; the corresponding private key is.

Certificate management should include:

1. Automated inventory of certificates and private keys.
2. Defined ownership and purpose for each certificate.
3. Automated renewal before expiration.
4. Secure private-key generation and storage.
5. Restricted issuance authority.
6. Monitoring for unexpected certificates and weak parameters.
7. Revocation or replacement when a private key is exposed.

Expired certificates can cause outages; compromised private keys can enable impersonation. Both operational reliability and security therefore depend on lifecycle automation.

5. Key Management as a Life-Cycle Discipline

Key management includes generation, distribution, storage, use, rotation, backup, recovery, revocation, archival, and destruction. The central principle is to separate duties: the people or services that use data keys should not automatically control the root keys that protect those data keys.

Key states and controls

A useful lifecycle is:

• Generation: create keys using an approved cryptographic random source and approved algorithm.
• Activation: permit use only after ownership, purpose, and policy are recorded.
• Distribution: deliver keys through authenticated and protected channels.
• Use: restrict operations, identities, services, regions, and contexts.
• Rotation: replace keys according to risk, policy, cryptographic guidance, or suspected compromise.
• Suspension or revocation: prevent further use when a key or certificate is no longer trustworthy.
• Archival: retain material only when necessary for lawful decryption or verification.
• Destruction: securely eliminate key material when retention is no longer justified.

Rotation does not necessarily re-encrypt every historical object immediately. In envelope encryption, a new KEK version may protect newly generated or rewrapped DEKs, while older versions remain available for permitted decryption. Organizations must define how long old versions remain usable and how they will re-encrypt data before destroying an old key.

Key rotation should be risk-based. Automatic periodic rotation is useful, but it cannot compensate for an exposed key. Suspected compromise requires immediate containment, access review, revocation or disabling where safe, replacement, investigation, and potentially re-encryption.

Access control and audit

Key permissions should distinguish operations such as encrypt, decrypt, generate data key, sign, verify, rotate, disable, schedule deletion, and administer policy. A workload may need to decrypt data but should not be able to delete the key or change its access policy. Conditions should restrict access by workload identity, account, environment, region, network context, and purpose where the platform supports them.

Key use must be logged. Logs should record the key identifier, requesting identity, operation, resource, time, result, and relevant context. Logging systems themselves require integrity protection and access controls. Excessive logging of plaintext, tokens, or secret values must be avoided.

6. KMSs, HSMs, and Secrets Managers

A cloud KMS provides managed cryptographic operations and key lifecycle functions through APIs, consoles, and policy systems. It commonly supports key generation, encryption and decryption, envelope encryption, rotation, access policies, audit logs, and integration with storage and databases.

An HSM is a specialized tamper-resistant device designed to generate, store, and use cryptographic keys within a protected boundary. HSMs can provide stronger control over key material, hardware-backed operations, and assurance for high-value keys. They may be dedicated, cloud-hosted, or integrated into a provider’s managed service. HSMs also introduce cost, availability, capacity, interoperability, and operational complexity.

A secrets manager stores and retrieves application secrets such as database passwords, API tokens, OAuth client secrets, and private configuration values. It commonly supports versioning, access policies, audit trails, and automatic rotation. A secrets manager is not identical to a KMS: a KMS primarily performs controlled cryptographic operations, while a secrets manager stores and distributes arbitrary secret values. They are often integrated.

7. Secrets Management

Secrets should not be embedded in source code, container images, machine images, public repositories, command histories, or ordinary configuration files. They should be retrieved at runtime by an authenticated workload identity, with the minimum permissions and scope necessary.

A sound secrets-management process includes:

• inventory and classification of secrets;
• strong workload identity rather than shared static credentials;
• short-lived credentials where practical;
• automatic rotation and tested recovery;
• separate secrets for development, testing, and production;
• redaction in logs and error messages;
• monitoring for anomalous retrieval and use;
• rapid revocation after suspected exposure;
• prevention of secret values appearing in backups, telemetry, or support tickets.

Secret rotation can break applications if clients cache values or if multiple systems depend on one credential. A safe design supports overlapping validity: issue a new credential, update consumers, verify operation, and then revoke the old credential. The exact process depends on the service and its ability to accept multiple active credentials.

8. Applied Case: Illustrative Online Health-Service Platform

Background

A fictional health-service platform stores appointment records, clinician notes, and billing data in cloud databases and object storage. Mobile clients connect through public APIs, while internal microservices communicate across several cloud networks. The organization must protect personal and health information and demonstrate controlled access to auditors.

Problem

An initial review finds that TLS is used at the public gateway but not consistently between internal services. Database backups use provider-default encryption, application credentials are stored in deployment files, and certificates are renewed manually. A database administrator can both alter key policies and access production data. The organization also has no reliable inventory of secrets.

Analysis

The primary risks are not merely weak algorithms. They include excessive privilege, poor lifecycle control, undiscovered credentials, certificate expiration, and inadequate separation of duties. Encryption at rest reduces exposure from storage theft but does not prevent an authorized database process from returning plaintext. Internal service traffic requires protection because a compromised workload could otherwise observe or modify requests.

Approach and solution

The organization adopts the following architecture:

1. TLS is required for external and internal service communication; mTLS is used for selected high-value service relationships.
2. Databases, object storage, snapshots, and backups use envelope encryption through a cloud KMS.
3. Separate KMS keys are assigned to production data domains, backup operations, and signing functions.
4. Application identities may request data-key operations only for defined resources. They cannot administer keys or change policies.
5. High-value signing keys are held in an HSM-backed service.
6. Passwords, API tokens, and connection credentials move to a secrets manager and are retrieved through workload identity.
7. Certificates are inventoried and renewed automatically, with alerts for anomalous issuance and approaching expiration.
8. Key-use, secret-retrieval, certificate, and administrative events are sent to an access-controlled monitoring system.
9. A tested incident process covers credential exposure, certificate compromise, key disabling, re-encryption, and customer notification where required.

Outcome

The platform gains stronger confidentiality and integrity controls, more defensible separation of duties, and improved operational reliability. The organization can demonstrate which identities used which keys and can rotate credentials without rebuilding every application. It also recognizes that encryption does not remove the need for database authorization, secure application design, data minimization, and incident response.

Learning points

The case illustrates four principles. First, cryptography must be designed with identity and access management. Second, key and certificate lifecycle automation is a security control, not merely an administrative convenience. Third, different materials require different services: data keys, root keys, application secrets, and certificates should not be treated as interchangeable. Fourth, the strongest protection is ineffective if recovery, rotation, logging, and operational ownership are undefined.

9. Governance, Ethics, and Regulation

Encryption decisions affect legal rights, organizational accountability, and public safety. Personal data should be protected according to its sensitivity, purpose, retention period, and applicable law. Data residency requirements may influence where keys and encrypted data are stored. However, residency of ciphertext does not automatically resolve legal questions if a provider or administrator can access keys elsewhere.

Key ownership and lawful access require explicit governance. Organizations should document who can authorize decryption, under what circumstances, how requests are recorded, and how emergency access is controlled. “Break-glass” access should be limited, time-bound, approved where possible, monitored, and reviewed afterward.

Strong encryption can protect individuals from criminals and unauthorized surveillance, but poor key recovery can make essential services unavailable. Conversely, undisclosed universal access mechanisms create high-value targets and weaken trust. Security governance should therefore assess proportionality, transparency, accountability, and the consequences of both access and non-access.

10. Challenges and Limitations

Cryptography has important limitations:

• It cannot correct incorrect authorization decisions.
• It cannot prevent disclosure by an authorized application or user.
• Lost keys can make data permanently inaccessible.
• Compromised endpoints may expose plaintext before encryption or after decryption.
• Poor randomness, nonce reuse, weak passwords, and incorrect certificate validation can defeat strong algorithms.
• Rotation can create outages if dependencies are unknown.
• HSMs and KMSs can become availability dependencies and may incur substantial cost.
• Encryption can complicate search, analytics, debugging, and data-loss prevention.
• Tokenization may centralize risk in the token vault.
• Advanced encryption-in-use technologies may have performance and portability constraints.

These limitations support a defense-in-depth strategy involving least privilege, secure software development, segmentation, monitoring, backup, incident response, and data minimization.

11. Best-Practice Design Checklist

A cloud team should be able to answer the following questions:

1. What data requires encryption, tokenization, or masking?
2. Is traffic encrypted and authenticated across every relevant network segment?
3. Which data is encrypted at rest, including backups and temporary copies?
4. Are encryption algorithms, protocol versions, and parameters approved and current?
5. Who owns each key, certificate, and secret?
6. Can workloads use a key without administering it?
7. How are keys generated, rotated, backed up, disabled, and destroyed?
8. How are exposed credentials and private keys revoked?
9. Are certificates and secrets inventoried and automatically renewed or rotated?
10. Are key use, secret retrieval, and administrative actions logged and reviewed?
11. Has recovery been tested, including loss of a region, service, or key version?
12. Does the design address data processed in memory and the security of endpoints?

Chapter Summary

Cryptography protects cloud information through confidentiality, integrity, authentication, and evidence of origin, but it must be integrated with identity, authorization, monitoring, and lifecycle governance. Symmetric encryption efficiently protects data, while asymmetric cryptography supports key establishment, certificates, and signatures. Hashing is a one-way integrity or verification technique; tokenization replaces sensitive data with a controlled reference and is not equivalent to encryption.

Encryption in transit protects network communication, encryption at rest protects stored representations, and encryption in use addresses exposure during processing through techniques such as confidential computing and, in specialized cases, homomorphic encryption. Envelope encryption separates data-encryption keys from key-encryption keys and is widely suited to cloud architecture.

Effective key management covers generation, controlled use, rotation, revocation, backup, recovery, and destruction. KMSs, HSMs, secrets managers, and certificate-management systems serve related but distinct purposes. Least privilege, separation of duties, automation, auditability, and tested recovery are essential. Finally, encryption cannot compensate for compromised endpoints, excessive permissions, weak software, or poor governance.

Key Terms

Cryptography: The discipline of protecting information through mathematical techniques.

Plaintext: Readable or original data before encryption.

Ciphertext: Data produced by encryption and intended to be unreadable without the required key.

Symmetric encryption: Encryption using a shared secret key for related encryption and decryption operations.

Asymmetric cryptography: Cryptography using mathematically related public and private keys.

Hash function: A one-way function that produces a fixed-length digest from input data.

Salt: A unique random value added to password input before password hashing.

HMAC: A keyed-hash message authentication code used for integrity and shared-secret authentication.

Digital signature: A private-key-generated value verified with a public key.

Tokenization: Replacement of sensitive data with a token resolved through a protected mapping service.

TLS: A protocol family for authenticated and encrypted network communication.

Certificate: A signed binding between an identity and a public key.

PKI: The policies, authorities, processes, and technologies supporting public-key certificates.

Data-encryption key: A key used to encrypt actual application data.

Key-encryption key: A key used to protect data-encryption keys.

Envelope encryption: A design in which encrypted data keys accompany ciphertext while a higher-level key remains protected.

KMS: A managed service for cryptographic keys and operations.

HSM: A hardware security module that protects cryptographic operations and key material.

Secrets manager: A controlled service for storing, delivering, auditing, and rotating application secrets.

Confidential computing: Protection of data during processing using hardware-backed trusted environments.

Review Questions

Short Answer Questions

1. What are the differences between encryption, hashing, and tokenization?
2. Why is a nonce important in many authenticated-encryption schemes?
3. What is the purpose of a certificate authority?
4. Define envelope encryption and identify its two principal key types.
5. Why should a workload be prevented from changing the policy of a key it uses?

Long Answer Questions

1. Explain how symmetric and asymmetric cryptography work together in a typical TLS connection.
2. Describe the complete lifecycle of a cloud encryption key, from generation to destruction.
3. Compare a cloud KMS, an HSM, and a secrets-management service.
4. Explain the differences among encryption in transit, at rest, and in use, including limitations of each.
5. Discuss how certificate automation and secret rotation improve both security and service reliability.

Analytical / Application Questions

1. A company encrypts database disks but allows every production administrator to decrypt backups and change key policies. Analyze the weaknesses and propose a revised separation-of-duties model.
2. A service’s API credential has been committed to a public repository. Develop an incident response sequence covering containment, rotation, investigation, monitoring, and prevention.
3. A hospital wants to perform analytics on highly sensitive records while reducing exposure to its cloud host. Compare conventional application controls, confidential computing, and homomorphic encryption, and recommend a proportionate approach.

Multiple Choice Questions with Answers

Question 1

Which technique is primarily designed to provide confidentiality for large volumes of cloud data?

A. Symmetric encryption

B. Cryptographic hashing

C. Digital signatures

D. Tokenization only

Correct Answer: A. Symmetric encryption

Question 2

What is the principal purpose of a salt in password storage?

A. To make a password reversible

B. To provide a unique random value for each password record

C. To replace the need for access control

D. To encrypt the database encryption key

Correct Answer: B. To provide a unique random value for each password record

Question 3

In envelope encryption, what normally encrypts the data-encryption key?

A. A certificate authority

B. A hash digest

C. A key-encryption key

D. A public log

Correct Answer: C. A key-encryption key

Question 4

Which statement about a digital signature is correct?

A. It provides confidentiality by itself

B. It uses a private key to create a value verifiable with a public key

C. It is identical to tokenization

D. It requires both parties to share one password

Correct Answer: B. It uses a private key to create a value verifiable with a public key

Question 5

Why is certificate hostname validation important in TLS?

A. It confirms that the certificate is associated with the intended service identity

B. It compresses the network traffic

C. It rotates the server’s private key automatically

D. It converts ciphertext into a token

Correct Answer: A. It confirms that the certificate is associated with the intended service identity

Question 6

Which service is most directly intended for storing application passwords and API tokens?

A. A secrets manager

B. A certificate authority only

C. A public DNS resolver

D. A hash function

Correct Answer: A. A secrets manager

Question 7

Which is a key limitation of encryption at rest?

A. It cannot protect any stored data

B. It prevents all authorized applications from reading plaintext

C. It does not prevent an authorized or compromised application from accessing decrypted data

D. It eliminates the need for backups

Correct Answer: C. It does not prevent an authorized or compromised application from accessing decrypted data

Question 8

What is a major operational risk of manual certificate renewal?

A. It guarantees stronger encryption

B. It can cause service outages when certificates expire

C. It removes the need for private keys

D. It prevents certificate issuance

Correct Answer: B. It can cause service outages when certificates expire

Question 9

Which characteristic best distinguishes an HSM from an ordinary secrets file?

A. An HSM provides a hardware-protected boundary for key operations

B. An HSM stores only public web pages

C. An HSM is a password hashing algorithm

D. An HSM eliminates the need for authorization

Correct Answer: A. An HSM provides a hardware-protected boundary for key operations

Question 10

What is a suitable response when a private key is suspected to be compromised?

A. Continue using it until its planned expiration

B. Publish the private key for transparency

C. Contain use, revoke or disable it where appropriate, replace it, and investigate

D. Convert it into a hash without changing access

Correct Answer: C. Contain use, revoke or disable it where appropriate, replace it, and investigate