Symmetric Encryption and Confidentiality

Introduction and Context

Confidentiality is the security property that prevents unauthorised parties from learning information. Symmetric encryption is one of the principal technologies used to achieve it. It is called symmetric because the same secret key, or closely related secret material, is used for encryption and decryption.

Symmetric cryptography is important because it is substantially faster than public-key cryptography and is suitable for protecting large volumes of data. It is used in disk encryption, database protection, virtual private networks, wireless security, web sessions, backups, and messaging systems. However, encryption alone does not automatically provide integrity, authenticity, replay protection, or secure key distribution. A sound design must therefore select an appropriate cipher, mode of operation, nonce or initialization vector, key-management process, and authentication mechanism.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish block ciphers from stream ciphers.
2. Explain the structure and security goals of AES.
3. Describe common modes of operation and their limitations.
4. Explain initialization vectors, nonces, padding, and authentication tags.
5. Select suitable methods for data at rest and data in transit.
6. Identify common implementation failures and recommend professional safeguards.

1. Symmetric Encryption Fundamentals

A symmetric encryption system consists of a plaintext message, a secret key, an encryption algorithm, and a decryption algorithm. In abstract form:

Here, P is plaintext, C is ciphertext, K is the secret key, E_K is encryption under key K, and D_K is decryption under that key. The equations assume that the correct key and any required auxiliary values, such as a nonce, are available and that the ciphertext has not been corrupted.

A secure modern cipher should exhibit computational confusion and diffusion. Confusion makes the relationship between the key and ciphertext difficult to analyse; diffusion spreads the influence of each plaintext bit across many ciphertext bits. A practical cipher should also resist known attacks, have a well-studied design, and be implemented without avoidable side channels.

Confidentiality is not the same as complete security

Encryption protects the content from observers who do not possess the key. It does not necessarily prove who created the message, detect modification, prevent replay, conceal metadata, or protect endpoint devices. For example, encrypted traffic may still reveal timing, volume, destination, and communication patterns. A stolen decryption key can also expose all data protected by that key unless additional key separation and rotation are used.

Important terminology

• Plaintext: Original readable or structured data.
• Ciphertext: Transformed data intended to conceal plaintext.
• Key: Secret value controlling cryptographic transformation.
• Key space: Set of all possible keys.
• Block: Fixed-size unit processed by a block cipher.
• Nonce: Value intended to be used once with a particular key.
• Initialization vector (IV): Auxiliary value that initializes a mode of operation; it may or may not require uniqueness, unpredictability, or secrecy depending on the mode.
• Authentication tag: Short value used to detect unauthorised modification.
• Key separation: Use of distinct keys for distinct purposes.

2. Block Ciphers and Stream Ciphers

Block ciphers

A block cipher transforms a fixed-size block of plaintext into a block of ciphertext under a key. AES, for example, has a block size of 128 bits and supports keys of 128, 192, or 256 bits. A block cipher by itself handles only one block at a time. A mode of operation specifies how multiple blocks are processed and how auxiliary values are used.

Block ciphers are not normally used by encrypting every block independently. If identical plaintext blocks always produced identical ciphertext blocks, patterns would remain visible. Modes solve this problem by linking blocks, incorporating counters, or combining encryption with authentication.

Stream ciphers

A stream cipher generates a pseudorandom keystream and combines it with plaintext, usually using exclusive OR (XOR):

Here, P_i, C_i, and S_i are corresponding plaintext, ciphertext, and keystream units; ⊕ denotes XOR. The equations rely on the property that XORing the same value twice restores the original value. The keystream must be unpredictable and, critically, must never be reused with the same key and nonce.

Modern stream-like constructions include ChaCha20. AES in counter mode also behaves like a stream cipher because it encrypts successive counter values to create a keystream. Stream ciphers do not require padding, which is useful for data of arbitrary length.

Comparison

A block or stream cipher provides confidentiality only if used correctly. In current practice, authenticated encryption is preferred over unauthenticated encryption.

3. AES: The Standard Block Cipher

The Advanced Encryption Standard (AES) is a widely deployed symmetric block cipher standardised by NIST. AES always has a 128-bit block size. Its supported key lengths are 128, 192, and 256 bits, corresponding to 10, 12, and 14 transformation rounds respectively.

AES represents a 128-bit block as a 4 × 4 array of bytes called the state. Each round, except the final round, applies four main transformations:

1. SubBytes: Each byte is replaced using a nonlinear substitution table, the S-box.
2. ShiftRows: Rows of the state are cyclically shifted.
3. MixColumns: Bytes in each column are combined through a defined operation over a finite field.
4. AddRoundKey: The state is XORed with a round key derived from the original key.

The final round omits MixColumns. A key-expansion procedure derives round keys from the supplied key. Decryption applies inverse transformations in the appropriate order.

AES security depends on the key remaining secret, correct implementation, and suitable mode selection. AES-256 is not automatically the best choice for every situation: AES-128 already provides a very large key space and may be faster on some systems. Organisational policy, long-term confidentiality requirements, hardware support, and regulatory expectations should guide the choice.

AES is not a password-hashing algorithm. Passwords should not be used directly as AES keys. A password-derived key must be produced by a password-based key-derivation function such as Argon2id, scrypt, or PBKDF2, with a unique salt and appropriate work factor.

4. Modes of Operation

A mode of operation defines how a block cipher protects a message longer than one block. The following modes illustrate why mode choice matters.

Electronic Codebook (ECB)

ECB encrypts each block independently. Formally, for block index i, C_i = E_K(P_i). Identical plaintext blocks produce identical ciphertext blocks under the same key. This leaks patterns and makes ECB unsuitable for structured data such as images, records, and documents. ECB should not be used for general application data.

Cipher Block Chaining (CBC)

CBC combines each plaintext block with the previous ciphertext block before encryption:

For the first block, an IV acts as C₀. Decryption is:

CBC requires padding when the message length is not an exact multiple of the block size. The IV must be unpredictable for many security goals and must not be reused inappropriately. CBC encryption does not provide integrity. An attacker may alter ciphertext and cause controlled changes in decrypted plaintext, and careless error handling can create padding-oracle attacks. CBC should therefore be used only in carefully designed, authenticated constructions; new systems generally prefer an AEAD mode.

Counter (CTR) mode

CTR encrypts a distinct counter input to produce a keystream:

where T_i is a unique counter block for each encrypted block under key K. CTR needs no padding and supports parallel processing and random access. However, reusing the same counter sequence with the same key is catastrophic. If two plaintexts use the same keystream, then:

This follows because the repeated keystream cancels under XOR. CTR also provides no authentication and should normally be combined with a secure MAC using encrypt-then-MAC, or replaced by an AEAD construction.

Galois/Counter Mode (GCM)

GCM combines counter-mode confidentiality with authentication based on a polynomial hash in a finite field. It produces ciphertext and an authentication tag. GCM is an AEAD mode: authenticated encryption with associated data. Associated data is authenticated but not encrypted, such as a protocol version, record number, or message header.

A crucial GCM rule is that a nonce must never repeat with the same key. The commonly used nonce length is 96 bits, but the implementation and protocol must define nonce construction clearly. A repeated nonce can reveal relationships between plaintexts and seriously weaken authentication.

CCM and other AEAD designs

CCM combines counter-mode encryption with CBC-MAC and is used in several constrained and wireless environments. ChaCha20-Poly1305 is not AES-based, but it is a widely used AEAD alternative, especially where hardware AES acceleration is unavailable. The practical principle is to use a standard, reviewed AEAD implementation rather than designing a new combination.

5. Initialization Vectors, Nonces, and Counters

IV and nonce requirements are mode-specific and must not be treated as interchangeable labels. An IV may need to be unpredictable, unique, or merely distinct according to the construction. A nonce generally emphasises uniqueness, meaning “number used once.” Neither normally needs to be secret.

A secure message format commonly stores the nonce or IV alongside the ciphertext because it is not secret:

The symbol ∥ means concatenation. The key identifier helps the recipient select the correct key; it is metadata, not a secret. The version field supports controlled protocol evolution. All fields that influence interpretation should either be authenticated as associated data or protected by another integrity mechanism.

Nonce generation deserves engineering attention. Random nonces require an adequately secure random generator and an acceptably low collision probability. Deterministic counters can avoid random collisions but must persist safely across restarts, devices, and failover events. A distributed service must ensure that two nodes cannot generate the same nonce under the same key.

6. Padding and Message Length

Block ciphers process complete blocks, while applications often produce arbitrary-length messages. Padding adds structured bytes so that the total length is a multiple of the block size. PKCS#7-style padding for a block size of b bytes adds p bytes, where:

Here, ℓ is the original message length in bytes. If the message already has a length divisible by b, a full block of padding is added. Each padding byte has the numeric value p. For example, with a 16-byte block size and a 29-byte message, p = 16 − (29 mod 16) = 3, so three bytes of value 0x03 are appended.

Padding must be validated strictly and errors must not be exposed through distinguishable timing or messages. In an authenticated design, authentication should be checked before padding is interpreted. AEAD modes such as GCM and ChaCha20-Poly1305 generally do not require block padding because they support arbitrary-length plaintext.

7. Authenticated Encryption

Confidentiality without integrity is unsafe for most modern applications. An attacker who cannot read a message may still alter it. Authenticated encryption provides:

• confidentiality of plaintext;
• integrity of ciphertext and authenticated data;
• authenticity evidence under the shared key;
• detection of tampering before plaintext is accepted.

An AEAD interface can be represented as:

Here, K is the key, N is a unique nonce, P is plaintext, A is associated data, C is ciphertext, and T is the authentication tag. Decryption must return plaintext only after successful tag verification. If verification fails, the application should reject the complete record and avoid revealing whether the failure arose from the ciphertext, tag, nonce, or formatting.

Associated data is useful when some information must remain visible for routing or processing but must not be alterable. Examples include a user identifier, record number, protocol version, or algorithm identifier. It is authenticated, not encrypted.

Encrypt-then-MAC

Before widespread AEAD, a robust composition was encrypt-then-MAC: encrypt the plaintext, then calculate a MAC over the ciphertext and relevant metadata. The receiver verifies the MAC before decrypting or processing the message. A secure implementation needs independent keys for encryption and MAC, usually derived from a master key through a key-derivation process. “Encrypt-and-MAC” and “MAC-then-encrypt” have different security properties and should not be substituted without analysis.

8. Key Management and Secure Implementation

The effective security of symmetric encryption is frequently limited by key management rather than by the cipher. A professional lifecycle includes generation, distribution, storage, use, rotation, backup, revocation, and destruction.

Key-management practices

• Generate keys with an operating-system or hardware-backed cryptographically secure random generator.
• Do not embed production keys in source code, mobile applications, scripts, or public repositories.
• Protect high-value keys in a hardware security module, trusted platform module, or managed key-management service where appropriate.
• Use key-encryption keys to protect data-encryption keys and separate keys by tenant, environment, purpose, and sensitivity.
• Assign key identifiers and maintain controlled rotation procedures.
• Plan recovery and revocation before an incident occurs.
• Limit plaintext key exposure in memory and logs.
• Destroy obsolete keys securely, while preserving authorised recovery keys where retention obligations require it.

Keys should be random, not memorable. A human password requires a password-based key-derivation function; a machine-generated encryption key does not. Salts prevent identical passwords from producing identical derived keys and make precomputed attacks more difficult. Salts need not be secret but should be unique.

Implementations should use established libraries, constant-time primitives where relevant, secure defaults, authenticated APIs, and automatic nonce management where the library can guarantee uniqueness. Developers should not implement AES, GCM, padding, or random-number generation themselves for production systems.

9. Selecting Methods for Data at Rest and in Transit

Data at rest

Data at rest includes files, databases, backups, removable media, and device storage. Full-disk encryption is valuable against loss or theft of a powered-off device, but it may not protect data after an attacker gains access to an unlocked operating system. File- or database-level encryption can provide finer control and support separation between applications and storage administrators.

Recommended choices include:

• authenticated encryption for individual objects, records, or files;
• a standard disk-encryption scheme for device or volume protection;
• envelope encryption for large data sets, using a data-encryption key protected by a key-encryption key;
• authenticated metadata, including object identifiers, versions, and tenant identifiers;
• separate keys and access policies for backup copies.

The appropriate granularity depends on threat model, performance, search requirements, recovery processes, and regulatory obligations. Encrypting a database column may protect stored values but can affect indexing, equality searches, reporting, and key rotation.

Data in transit

Data in transit includes web requests, service-to-service communication, messaging, and remote administration. Applications should normally use a well-configured protocol such as TLS rather than adding ad hoc encryption to an already established secure channel. TLS combines symmetric session encryption with authenticated key exchange and certificates. After a session key is established, symmetric AEAD protects records efficiently.

For direct application protocols, use an authenticated channel or a standard AEAD construction with explicit nonce and replay design. Certificate validation, endpoint authentication, algorithm negotiation, forward secrecy, and secure downgrade handling are as important as the symmetric cipher itself.

10. Worked Example: Designing an Encrypted Record

Suppose a records service stores an invoice. The service chooses an AEAD library, a 256-bit data-encryption key, and a unique 96-bit nonce. The visible metadata includes a tenant identifier and schema version. The design is:

The stored object contains the tenant identifier, schema version, nonce, ciphertext, tag, and key identifier. During retrieval, the service obtains the key using the key identifier, reconstructs exactly the same associated data, and calls decryption. If the tag verifies, it parses the invoice; otherwise, it rejects the record.

This design protects the invoice contents and detects alteration of the invoice, tenant identifier, schema version, nonce-related inputs, and ciphertext, assuming all these fields are included correctly in the authenticated interface. It does not hide the tenant identifier or object size, and it does not protect an endpoint that already has the key and plaintext.

11. Illustrative Case Study: Secure Cloud Backups

Background

A medium-sized healthcare organisation stores encrypted backups in cloud storage. An earlier system used AES-CBC with a fixed IV embedded in application configuration. It also logged decryption errors and had one encryption key shared across all customers.

Problem

The fixed IV could reveal relationships between first blocks, CBC did not detect malicious modification, error messages could support padding-oracle analysis, and a single compromised key could expose every customer’s backup. Key rotation was also difficult because objects lacked key identifiers.

Analysis

The organisation’s requirements were confidentiality, tamper detection, tenant isolation, operational recovery, and auditable rotation. Full-disk encryption at the cloud provider was insufficient because the organisation also needed protection against storage-layer access and accidental corruption.

Approach and solution

The redesigned service used envelope encryption. Each backup object received a randomly generated data-encryption key. The object was encrypted with an approved AEAD construction, using a nonce generated and tracked by the encryption library. Tenant identifier, object identifier, format version, and key identifier were authenticated as associated data. The data-encryption key was itself encrypted under a tenant-scoped key-encryption key held in a managed key service. Decryption errors were consolidated into one failure response, and audit logs recorded key usage without recording plaintext or keys.

Outcome

The new design detected tampering, reduced the impact of a single key compromise, supported staged rotation, and improved incident investigation. It did not eliminate all risk: compromised application credentials could still request decryption, and metadata remained visible to the storage platform.

Learning points

1. A strong cipher cannot compensate for unsafe mode and IV practices.
2. Authentication must cover metadata that influences interpretation.
3. Key scope and lifecycle determine the blast radius of compromise.
4. Security design must include operational recovery, logging, and access control.

12. Limitations, Governance, and Contemporary Practice

Encryption can create governance challenges. Organisations must determine who may decrypt data, how lawful access requests are handled, how keys are escrowed, and how retention and deletion policies interact with backups. Excessive centralisation of keys creates a high-value target, while uncontrolled key fragmentation can make recovery impossible. Access should follow least privilege, with separation of duties and auditable approvals for sensitive operations.

Regulatory requirements differ by jurisdiction and sector, but common expectations include documented risk assessment, access control, incident response, secure disposal, and evidence that cryptographic algorithms and implementations are maintained. Compliance is not proof of security; it must be integrated with threat modelling and testing.

Contemporary systems increasingly use AEAD by default, hardware acceleration for AES, memory-safe cryptographic libraries, automatic key rotation, and confidential-computing technologies. Post-quantum cryptography primarily affects public-key key exchange and signatures, but symmetric systems should still use accepted key sizes and monitor standards guidance. Larger symmetric keys may be selected for long-term protection, but implementation quality and key management remain essential.

Best-Practice Checklist

Before approving a symmetric-encryption design, ask:

• Is a standard, reviewed algorithm used?
• Is authenticated encryption used where feasible?
• Is every nonce unique for the relevant key and mode?
• Are IV and nonce requirements documented precisely?
• Are keys generated, stored, rotated, and revoked securely?
• Are encryption and authentication purposes separated when separate primitives are used?
• Is associated data authenticated?
• Are failures handled without padding oracles and sensitive diagnostics?
• Does the design address backups, logs, caches, and temporary files?
• Are key scope, recovery, auditability, and regulatory obligations defined?

Chapter Summary

Symmetric encryption efficiently protects large quantities of information, but its security depends on more than selecting AES. Block ciphers process fixed-size blocks and require modes of operation for longer messages; stream ciphers and counter modes generate keystreams and demand strict nonce uniqueness. AES uses substitution, permutation, mixing, and round keys, but AES-ECB is unsuitable for general data and unauthenticated CBC or CTR can permit undetected manipulation. IVs and nonces are mode-specific auxiliary values and are usually public, while their uniqueness or unpredictability is essential. Padding makes arbitrary-length data compatible with some block modes, but AEAD modes generally avoid padding and combine confidentiality with integrity and authenticated associated data. Secure systems use standard libraries, strong key lifecycle management, tenant- and purpose-specific keys, careful failure handling, and protocol-level protections. Data at rest commonly benefits from envelope and storage encryption, while data in transit should use a standard authenticated channel such as TLS.

Key Terms

Symmetric encryption: Encryption and decryption using the same secret key or related secret material.

Plaintext: Original data before encryption.

Ciphertext: Encrypted data intended to conceal plaintext.

Block cipher: Algorithm that transforms fixed-size blocks under a key.

Stream cipher: Algorithm that combines plaintext with a generated keystream.

AES: Advanced Encryption Standard, a 128-bit-block symmetric cipher.

Mode of operation: Method for applying a block cipher to messages longer than one block.

ECB: Mode that encrypts blocks independently and leaks repeated-block patterns.

CBC: Mode that chains each plaintext block with the preceding ciphertext block.

CTR: Mode that encrypts counter inputs to produce a keystream.

AEAD: Authenticated encryption with associated data.

Nonce: Value intended for one-time use with a particular key.

Initialization vector: Auxiliary input used to initialise a cryptographic mode.

Padding: Extra structured data added to complete a block.

Authentication tag: Value used to detect unauthorised modification.

Associated data: Data authenticated by AEAD but left unencrypted.

Key derivation: Process of producing cryptographic keys from secret input.

Envelope encryption: Protection of a data key by encrypting it under another key.

Key rotation: Controlled replacement of cryptographic keys.

Review Questions

5 Short Answer Questions

1. What is the principal difference between a block cipher and a stream cipher?
2. Why is AES-ECB unsuitable for encrypting ordinary structured data?
3. What is the difference between confidentiality and integrity?
4. Why must a GCM nonce not be reused with the same key?
5. What role does associated data play in an AEAD scheme?

5 Long Answer Questions

1. Explain AES’s principal round transformations and discuss how AES becomes suitable for messages longer than one block.
2. Compare CBC, CTR, and GCM with respect to padding, parallelism, authentication, and nonce or IV requirements.
3. Describe a secure key-management lifecycle for a cloud storage service.
4. Explain why encryption without authentication can permit dangerous message modification.
5. Discuss the differences between protecting data at rest and protecting data in transit.

3 Analytical / Application Questions

1. A developer proposes AES-CTR with a randomly generated nonce but does not store the nonce with the ciphertext. Analyse the design and recommend a corrected message format.
2. A database uses one AES key for every customer and never rotates it. Evaluate the security and operational risks, then propose an improved architecture.
3. A service receives an AEAD record whose tag is invalid. Explain what the service should do and why detailed error messages may create security risk.

Multiple Choice Questions with Answers

Question 1

Which property is provided directly by symmetric encryption?

A. Confidentiality

B. Non-repudiation

C. Automatic key distribution

D. Guaranteed availability

Correct Answer: A. Confidentiality

Question 2

What is the AES block size?

A. 64 bits

B. 128 bits

C. 192 bits

D. 256 bits

Correct Answer: B. 128 bits

Question 3

Why should ECB generally not be used for structured data?

A. It requires public-key certificates

B. It cannot process any plaintext

C. Identical plaintext blocks produce identical ciphertext blocks

D. It always requires a password

Correct Answer: C. Identical plaintext blocks produce identical ciphertext blocks

Question 4

What is the critical requirement for a GCM nonce?

A. It must be kept permanently secret

B. It must never repeat with the same key

C. It must always be exactly 32 bytes

D. It must contain printable characters

Correct Answer: B. It must never repeat with the same key

Question 5

Which construction provides confidentiality and integrity together?

A. AES-ECB

B. Unauthenticated AES-CTR

C. AEAD

D. Plain hashing

Correct Answer: C. AEAD

Question 6

For a 16-byte block size and a 29-byte message, how many PKCS#7-style padding bytes are added?

A. 1

B. 2

C. 3

D. 13

Correct Answer: C. 3

Question 7

What does associated data in AEAD provide?

A. Encryption without authentication

B. Authentication without encryption

C. Password recovery

D. Nonce generation

Correct Answer: B. Authentication without encryption

Question 8

Which practice best protects against nonce collisions in a distributed encryption service?

A. Let each node choose counters independently

B. Use a coordinated, persistent nonce-allocation design

C. Reuse the nonce after a restart

D. Hide the nonce from the recipient

Correct Answer: B. Use a coordinated, persistent nonce-allocation design

Question 9

What is envelope encryption?

A. Encrypting a data key under a key-encryption key

B. Encrypting only network headers

C. Hashing a password twice

D. Using ECB for backup files

Correct Answer: A. Encrypting a data key under a key-encryption key

Question 10

What should an AEAD receiver do when tag verification fails?

A. Process the plaintext anyway

B. Retry with a different key automatically

C. Reject the record without releasing plaintext

D. Ignore the tag if the ciphertext is readable

Correct Answer: C. Reject the record without releasing plaintext