Hashing, Message Authentication and Password Protection

Introduction

Cybersecurity systems must answer three different questions: Has data been altered? Did it come from the claimed source? Can a user prove knowledge of a password without exposing the password itself? Encryption, hashing, checksums, digital fingerprints, message authentication codes (MACs), salts and password-based key derivation functions address related but distinct problems.

A common design error is to treat a hash as encryption or to store passwords as ordinary cryptographic hashes. A secure design selects a primitive according to its security goal, threat model and operational constraints. This chapter develops those distinctions and applies them to file integrity, API authentication and password storage.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish encryption, hashing, checksums and digital fingerprints;
• explain preimage resistance, second-preimage resistance and collision resistance;
• select modern hash functions for appropriate uses;
• calculate and interpret an HMAC at a conceptual level;
• explain the purpose of salts, peppers and password-based key derivation functions;
• evaluate password-storage and integrity-verification designs;
• identify implementation, governance and ethical risks.

1. Cryptographic Hash Functions

A cryptographic hash function maps an input of arbitrary length to a fixed-length output, called a digest or hash value:

Here, m is the message or input, H is the hash function and h is the fixed-length digest. A hash function is deterministic: the same input produces the same output. It is not intended to be reversible, and unlike encryption it has no decryption key.

Important security properties are:

• Preimage resistance: Given a digest h, it should be computationally infeasible to find any message m such that H(m) = h.
• Second-preimage resistance: Given a particular message m₁, it should be infeasible to find a different message m₂ with the same digest.
• Collision resistance: It should be infeasible to find any two distinct messages m₁ and m₂ such that H(m₁) = H(m₂).
• Avalanche effect: A small input change should produce an apparently unrelated digest.
• Efficiency: Legitimate computation should be practical, while attacks should require infeasible resources for the intended security level.

For an ideal n-bit hash, a generic collision attack requires approximately 2^n/2 hash evaluations because of the birthday bound. This is an approximation for generic attacks, not a guarantee against weaknesses in a particular algorithm. A generic preimage attack requires approximately 2ⁿ evaluations. For example, SHA-256 has a 256-bit output and an ideal collision-work estimate of about 2¹²⁸, although real security also depends on cryptanalysis, implementation and the value of the protected asset.

Common Hash Algorithms

A hash alone does not prove who created a message. If an attacker can replace both a file and its published digest, the digest provides little assurance. Authentic integrity requires a trusted distribution channel, a digital signature, or a keyed MAC.

2. Encryption, Hashing, Checksums and Digital Fingerprints

Encryption is a reversible transformation controlled by a key. Its principal objective is confidentiality, although authenticated encryption can also provide integrity and authenticity. A ciphertext should be decrypted only by an authorized holder of the key.

Hashing is normally one-way and keyless. It supports integrity comparison, indexing, content addressing and cryptographic constructions such as digital signatures and HMAC. It does not hide the input: an attacker can hash guesses and compare outputs.

Checksums are error-detection values designed mainly to detect accidental corruption. CRCs and ordinary checksums are fast and useful for transmission or storage errors, but they are not designed to resist a deliberate attacker. A malicious actor can usually modify data and recompute the checksum.

Digital fingerprints is an informal term for a compact digest used to identify or compare data. A fingerprint may be cryptographic, such as SHA-256, or non-cryptographic, such as a perceptual image hash. The security claim depends on the underlying method and context.

3. Message Authentication Codes

A message authentication code is a keyed authenticator. A sender and receiver share a secret key K. The sender computes a tag:

The receiver recomputes the tag over the received message and accepts only if the tags match. A secure MAC provides message integrity and authentication to parties that possess the shared secret. It does not provide non-repudiation: either key holder could have generated a valid tag.

HMAC

HMAC combines a cryptographic hash with a secret key. For a hash function with block size B, inner pad ipad and outer pad opad, the construction is:

K' is the key normalized to the hash block size, ⊕ denotes exclusive OR, and ∥ denotes concatenation. If the supplied key is longer than the block size, it is first hashed; if shorter, it is padded according to the HMAC specification. The formula assumes a correctly implemented approved hash and standard HMAC processing.

HMAC is preferable to naïvely computing H(K ∥ m), because HMAC was specifically designed to avoid structural problems associated with hash constructions, including length-extension concerns for some Merkle–Damgård hashes. HMAC-SHA-256 is widely supported.

Tags must be compared safely. A comparison routine that stops at the first unequal byte may reveal timing information. A constant-time comparison should be used where an attacker can make repeated observations. Tags should normally be transmitted in a fixed, adequate length; truncation reduces security and must be justified by the threat model.

MACs and Digital Signatures

A MAC uses one shared secret and is efficient for mutually trusted systems. A digital signature uses a private key to sign and a public key to verify. Signatures support public verification and stronger attribution properties, but require public-key infrastructure and are generally more computationally expensive. Neither a MAC nor a signature encrypts the message.

Authenticated Encryption

Modern applications commonly use authenticated-encryption schemes such as AES-GCM or ChaCha20-Poly1305. These produce ciphertext and an authentication tag, protecting confidentiality and integrity together. Developers should use a vetted authenticated-encryption API rather than combining encryption and authentication ad hoc. Nonces or initialization vectors must follow the algorithm's requirements; nonce reuse can be catastrophic for some schemes.

4. Integrity Verification in Practice

A software publisher may provide a file and a signed digest. The user computes the digest locally and verifies the publisher's signature on the digest or release metadata. The sequence is:

1. obtain the file and trusted verification key through an authenticated channel;
2. compute the specified digest over exactly the specified bytes;
3. verify the signature or authenticated MAC on the digest or metadata;
4. compare the computed value and the authenticated value;
5. reject the file if any step fails.

A displayed hash copied from the same compromised website as the download is not a strong trust anchor. Secure update systems therefore authenticate metadata, use key rotation and protect signing keys, often with offline or hardware-backed controls.

Canonicalization is another practical concern. Two systems may hash different byte representations of what users perceive as the same document. Encoding, line endings, compression, field ordering and whitespace must be defined before hashing structured data.

5. Password Protection

Passwords have low and uneven entropy. Users reuse passwords, select predictable phrases and are vulnerable to phishing. A database compromise therefore permits offline guessing if password verification data is weakly protected.

A password-storage system should not store plaintext passwords, reversible encrypted passwords or fast general-purpose hashes such as SHA-256 alone. Instead, it stores a password-hashing record generated by a deliberately expensive, memory-hard or otherwise resource-intensive function.

Salts

A salt is a unique, randomly generated non-secret value stored with each password record. Conceptually:

P is the password, S is the salt, r is the derived result and parameters specify the cost configuration. A salt ensures that identical passwords receive different records and prevents effective use of precomputed rainbow tables. It does not make a weak password strong and does not need to be confidential.

A fresh, cryptographically random salt should be generated for every password. The salt must be long enough to make accidental repetition negligible; modern password-hashing libraries generally generate suitable salts automatically. The salt, algorithm identifier and cost parameters are stored alongside the derived value.

Password-Hashing Functions and PBKDFs

Password-based key derivation functions (PBKDFs) transform a password and salt into a key or verifier while increasing guessing cost. Common choices include Argon2id, scrypt, bcrypt and PBKDF2-HMAC. Argon2id is often preferred for new systems when available because it combines memory-hard processing with resistance to different classes of parallel attacks.

A simplified work model is:

where G is the attacker's number of guesses and t_guess is the average time per guess under the attack conditions. This is an operational approximation, not a universal law: attackers may use GPUs, specialized hardware and parallelism, while defenders can rate-limit online attempts. Memory cost, parallelism and implementation details also affect the real cost.

The chosen parameters should make legitimate authentication acceptable on the organization's supported hardware while making large-scale offline guessing expensive. Benchmark on production-like systems, monitor authentication latency and establish a rehash policy as hardware improves.

Peppering and Secret Separation

A pepper is an additional secret value used in password verification but not stored in the password database. It may be held in a secrets manager or hardware security module. A pepper can reduce the value of a database-only breach, but it introduces key-management and availability risks. It must not be treated as a substitute for salts or a strong password KDF.

A conceptual record might contain:

algorithm = Argon2id
parameters = memory, iterations, parallelism
salt = unique random value
verifier = Argon2id(password, salt, parameters)

A pepper, if used, is integrated according to the library's documented construction rather than through improvised concatenation.

Verification and Account Recovery

During login, the server retrieves the algorithm and parameters, recomputes the verifier using the submitted password and stored salt, and performs a constant-time comparison. It should not reveal whether the username or password was the incorrect component. Rate limiting, multi-factor authentication, breached-password screening and monitoring reduce online attack risk.

Password reset links should be random, single-use, short-lived and stored or compared securely. Support personnel should never need to know a user's password. Recovery processes are part of password security because attackers frequently target recovery rather than the KDF.

6. Worked Application: Designing an API and User Authentication Service

Background

Consider an illustrative online student-services platform. It provides a web login and an internal API used by a mobile application and a payment service.

Problem

An initial design stores SHA-256(password) and sends API requests with a client identifier but no integrity protection. An attacker obtains the database and can alter API parameters in transit. Identical passwords also have identical database values.

Analysis

The password design enables fast offline guessing and exposes password equality. SHA-256 is intentionally fast, which is useful for file hashing but harmful for password verification. The API design lacks a mechanism to detect modification or authenticate the request. Encryption of the transport channel alone does not replace application-level authorization or protect a compromised endpoint.

Approach and Solution

The platform adopts these controls:

1. Passwords are processed by Argon2id with a unique random salt and documented memory, time and parallelism parameters. The record stores the algorithm version, parameters, salt and verifier.
2. The service benchmarks the KDF, rehashes successful logins when parameters become outdated and blocks known breached passwords.
3. API clients use TLS and authenticated tokens with narrowly scoped permissions and short lifetimes.
4. For a server-to-server integration requiring shared-key authentication, each request includes a method, path, timestamp, nonce and body representation in a canonical format. The sender computes an HMAC over that canonical string. The receiver checks the timestamp window, rejects reused nonces and verifies the HMAC before authorization.
5. Keys are generated randomly, separated by environment and rotated through a secrets-management system. Logs exclude passwords, raw tokens and secret keys.

Outcome

A database copy no longer directly reveals password equality or permits cheap SHA-256 guessing. HMAC verification detects alteration of authenticated request fields, while timestamps and nonces reduce replay risk. TLS protects the channel, and authorization controls determine what an authenticated client may do.

Learning Points

The case illustrates a layered design. Password hashing addresses offline verification risk; HMAC addresses shared-key message authenticity; TLS protects communication; authorization limits actions. No individual primitive solves all four problems.

7. Limitations, Governance and Ethical Considerations

Cryptography does not compensate for weak key management, insecure endpoints, poor access control or social engineering. A mathematically strong hash can be undermined by an attacker replacing the trusted reference value. A secure password KDF cannot protect a password typed into a phishing page. A valid HMAC proves possession of a key, not that the underlying business action is authorized.

Organizations should document algorithms, parameters, key ownership, rotation, retention and incident-response procedures. Security testing should include offline password-cracking simulations using authorized test data, replay testing, malformed input, timing analysis and key-compromise scenarios. Testing must respect privacy, authorization and data-protection law.

Credential databases are high-value personal-data assets. Collection should be minimized, access should be logged and restricted, and breach plans should include forced resets, token revocation and communication obligations. Passwords should never be used as general-purpose encryption keys without a proper KDF, and secrets should not be placed in source code or logs.

8. Contemporary Developments and Best Practices

Passkeys and public-key authenticators reduce dependence on reusable passwords by keeping a private key on the user's device and using public-key verification. They do not eliminate the need for hashing elsewhere, but they can reduce phishing and password-reuse risk.

Useful current practices include:

• use SHA-256, SHA-3 or another approved cryptographic hash for general integrity purposes;
• use HMAC or authenticated encryption when authenticity is required;
• use Argon2id, scrypt, bcrypt or PBKDF2-HMAC through maintained libraries for passwords;
• generate a unique random salt per password;
• benchmark and periodically review KDF parameters;
• use constant-time verification where appropriate;
• authenticate reference hashes, not merely publish them beside downloadable files;
• prevent replay with nonces, timestamps or sequence numbers when the protocol requires it;
• separate keys by purpose, environment and tenant where appropriate;
• prefer well-reviewed standard constructions over custom cryptography.

Chapter Summary

Encryption is reversible and primarily protects confidentiality, whereas hashing produces a fixed-length, non-reversible digest. Checksums detect accidental errors but generally do not resist malicious alteration. A cryptographic hash provides important integrity properties but does not authenticate its source by itself. HMAC combines a shared secret with a hash to provide message integrity and origin authentication; digital signatures provide public verification and different attribution properties. Authenticated encryption combines confidentiality and integrity.

Passwords require special treatment because they are guessable and attackers can perform offline trials. Unique salts prevent identical passwords from producing identical records and defeat precomputed tables. Password-hashing functions such as Argon2id deliberately increase the cost of guessing. Peppers, rate limiting, multi-factor authentication, secure recovery and careful key management provide additional layers. Effective designs match each primitive to its security goal and protect the surrounding operational process.

Key Terms

Cryptographic hash function: A function that maps arbitrary input to a fixed-length digest with security properties against inversion and collisions.

Digest: The fixed-length output produced by a hash function.

Preimage resistance: Difficulty of finding an input that produces a specified digest.

Collision resistance: Difficulty of finding two different inputs with the same digest.

Checksum: An error-detection value mainly intended to identify accidental corruption.

Digital fingerprint: A compact value used to identify or compare data.

Message authentication code: A keyed value used to verify message integrity and origin to parties sharing the key.

HMAC: A standardized MAC construction based on a cryptographic hash function.

Encryption: Reversible transformation using a key to protect information, usually confidentiality.

Authenticated encryption: Encryption that also provides integrity and authenticity through an authentication tag.

Salt: A unique, non-secret random value combined with a password before password derivation.

Password-based key derivation function: A function that derives a key or verifier from a password while imposing computational cost.

Pepper: An additional secret used in password protection and kept separately from the password database.

Argon2id: A modern memory-hard password-hashing and key-derivation algorithm.

Preimage attack: An attempt to find an input corresponding to a known digest.

Birthday bound: The approximate collision-search complexity of 2^n/2 for an ideal n-bit hash.

Nonce: A value intended for one-time use in a protocol.

Replay attack: Reuse of a previously valid message or authentication exchange.

Constant-time comparison: A comparison designed to limit timing leakage about secret values.

Canonicalization: Converting data into a defined representation before hashing or authentication.

Review Questions

Short Answer Questions

1. Why is a cryptographic hash not considered encryption?
2. What security problem does a unique password salt address?
3. What two primary properties does an HMAC provide?
4. Why is SHA-256 alone unsuitable for storing passwords?
5. What is the difference between a checksum and a cryptographic hash?

Long Answer Questions

1. Compare encryption, hashing, checksums, MACs and digital signatures in terms of keys, reversibility, authenticity and typical uses.
2. Explain preimage resistance, second-preimage resistance and collision resistance, including the significance of the birthday bound.
3. Describe HMAC processing conceptually and explain why a naïve keyed hash construction should not replace HMAC.
4. Design a secure password-storage system, covering salts, KDF parameters, verification, rehashing, recovery and breach response.
5. Explain how authenticated encryption, TLS and application-level authorization provide distinct security functions.

Analytical / Application Questions

1. A company publishes a software download and an unsigned SHA-256 digest on the same web server. Evaluate the protection offered if the server is compromised, and propose a stronger distribution design.
2. An API uses HMAC but authenticates only the request body, not the HTTP method, path, timestamp or nonce. Analyze possible manipulation and replay risks and recommend a canonical request design.
3. A breached database contains identical SHA-256 values for many users and the organization wants to add a single global salt. Evaluate the proposal and recommend a migration strategy using a modern password KDF.

Multiple Choice Questions with Answers

Question 1

Which property is primarily provided by a cryptographic hash function?

A. Reversible confidentiality

B. Fixed-length representation with one-way integrity support

C. Public-key non-repudiation

D. Guaranteed authentication of the data source

Correct Answer: B. Fixed-length representation with one-way integrity support

Question 2

For an ideal n-bit hash function, the approximate generic collision-search work is:

A. 2ⁿ

B. 2²ⁿ

C. 2ⁿ⁄²

D. n²

Correct Answer: C. 2ⁿ⁄²

Question 3

What is the principal purpose of a unique password salt?

A. To keep the password database secret from administrators

B. To make identical passwords produce different derived values

C. To provide public verification of a password

D. To replace the need for a password KDF

Correct Answer: B. To make identical passwords produce different derived values

Question 4

Which construction uses a shared secret and a hash function to authenticate a message?

A. HMAC

B. CRC

C. Plain SHA-256

D. A digital fingerprint

Correct Answer: A. HMAC

Question 5

Why is SHA-256 alone generally unsuitable for password storage?

A. It cannot process text passwords

B. It is deliberately fast, enabling rapid offline guessing

C. It always produces variable-length output

D. It requires a public key

Correct Answer: B. It is deliberately fast, enabling rapid offline guessing

Question 6

What does a checksum primarily detect?

A. Accidental corruption

B. The identity of a sender

C. Password reuse

D. Non-repudiation

Correct Answer: A. Accidental corruption

Question 7

Which statement about a MAC is correct?

A. Anyone with the message can verify it without a key

B. It provides confidentiality automatically

C. It provides integrity and authentication to parties sharing a secret

D. It proves which one of two key holders created the message

Correct Answer: C. It provides integrity and authentication to parties sharing a secret

Question 8

Which algorithm is specifically designed for password hashing and key derivation?

A. CRC-32

B. MD5

C. Argon2id

D. RSA encryption

Correct Answer: C. Argon2id

Question 9

What is a replay-prevention nonce intended to do?

A. Compress a message

B. Ensure a previously valid protocol message cannot simply be reused

C. Make a hash reversible

D. Replace authorization checks

Correct Answer: B. Ensure a previously valid protocol message cannot simply be reused

Question 10

Which design best authenticates a downloadable software release?

A. Publish an unsigned hash on the same server as the download

B. Use a CRC and omit key protection

C. Publish authenticated release metadata and verify the digest through a trusted signature key

D. Encrypt the filename with a user password

Correct Answer: C. Publish authenticated release metadata and verify the digest through a trusted signature key