Hashing, Message Authentication and Password Security

Introduction and Context

Cryptography is often associated with encryption, but confidentiality is only one security objective. Modern systems must also determine whether data has been altered, whether a message came from an authorised source, and whether a password can be stored without exposing the original secret. Hash functions, message authentication codes, salts and password-based key-derivation functions address these requirements.

A cryptographic hash converts an input of arbitrary length into a fixed-length digest. Hashing is normally one-way and is used for integrity checking, digital signatures, indexing and password-related constructions. A message authentication code (MAC) combines a message with a secret key to provide integrity and origin authentication to parties that share that key. Password storage requires a different approach: passwords should not be encrypted for routine storage, because a stolen decryption key could reveal every password. Instead, password-derived values should be generated using a deliberately expensive password-hashing or key-derivation function, with a unique salt for each account.

These mechanisms are related but not interchangeable. A plain hash does not prove who created a message. A MAC does not provide confidentiality. A password hash is not simply an ordinary hash applied once to a password. Correct selection and implementation are therefore essential cybersecurity skills.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the properties and applications of cryptographic hash functions.
2. Distinguish hashing, encryption, digital signatures and message authentication codes.
3. Explain how HMAC provides integrity and authentication.
4. Select and use salts and password-based key-derivation functions appropriately.
5. Evaluate password-storage designs and identify insecure practices.
6. Apply these concepts to practical system and incident-response scenarios.

1. Cryptographic Hash Functions

A cryptographic hash function is a deterministic algorithm that maps an input message of arbitrary length to a fixed-length output called a digest, hash or message digest. In general notation:

Here, m is the input message, H is the hash function and h is the resulting digest. The output length is fixed for a particular algorithm; for SHA-256, it is 256 bits, or 32 bytes, regardless of whether the input is one byte or several gigabytes.

Required security properties

A modern cryptographic hash should provide the following properties:

• Preimage resistance: Given a digest h, it should be computationally infeasible to find any message m such that H(m) = h.
• Second-preimage resistance: Given a particular message m₁, it should be infeasible to find a different message m₂ with H(m₁) = H(m₂)
• Collision resistance: It should be infeasible to find any two distinct messages with the same digest. A collision is a pair m₁ ≠ m₂ for which H(m₁) = H(m₂).
• Avalanche effect: A small change in the input should produce a substantially different, apparently unpredictable digest.
• Efficiency: The function should be practical to compute for legitimate users and systems.

“Infeasible” does not mean mathematically impossible. It means that the required computational resources, time or cost are beyond a realistic adversary under the relevant threat model.

Collision probability and the birthday effect

For an ideal hash with an n-bit output, an attacker searching for a collision may expect success after roughly 2^n/2 trials, rather than 2ⁿ trials. This is the birthday effect. It is one reason that collision resistance is weaker than preimage resistance for the same output length.

This approximation applies to idealised random-like hashes and large search spaces; it is not a prediction of the exact effort against every implementation. Practical attacks may exploit structural weaknesses, poor protocols or inadequate key management rather than brute force alone.

Common hash algorithms

MD5 and SHA-1 are historically important but should not be selected for new security-sensitive designs because practical collision attacks exist. SHA-2, including SHA-256 and SHA-512, remains widely deployed. SHA-3 uses a different internal construction and is standardised for contemporary use. BLAKE2 and BLAKE3 are also efficient modern designs, although the correct choice depends on standards, libraries and interoperability requirements.

A hash algorithm is not automatically suitable for password storage merely because it is cryptographically strong. General-purpose hashes are designed to be fast; password protection requires controlled computational expense.

2. Integrity, Authentication and Confidentiality

Hashing is not encryption

Encryption transforms plaintext into ciphertext using a key and is designed to be reversible by an authorised party. Hashing produces a digest and is intended to be computationally infeasible to reverse. A digest can help detect accidental or unauthorised modification, but it does not conceal the original data.

For example, a software publisher may publish a SHA-256 digest of an installer. A user downloads the installer, computes its digest and compares the result. Matching values provide evidence that the downloaded bytes are unchanged relative to the reference value. However, if an attacker can alter both the installer and the published digest, an unkeyed hash alone cannot identify the attack. The reference digest must come through a trusted channel, such as a digitally signed release record.

Hashes and digital signatures

A digital signature normally signs a message digest using a private key. Anyone with the corresponding public key can verify the signature. This provides integrity, authentication of the signing key and non-repudiation in an operational or legal sense, subject to key protection and governance. A plain hash provides none of these identity assurances.

Message authentication codes

A MAC is a keyed construction that produces a tag:

where K is a secret key shared by the communicating parties, m is the message and t is the authentication tag. The recipient recomputes the tag using the same key and compares it using a constant-time comparison operation.

A secure MAC provides:

• message integrity;
• authentication to parties that possess the shared key; and
• resistance to forgery when the key remains secret.

It does not normally provide confidentiality, and it does not allow the recipient to prove to an independent third party which of the two key holders created the message, because both holders can generate valid tags.

3. HMAC

HMAC is a widely used MAC construction based on a cryptographic hash function. Its standard form is:

In this expression, H is a hash function, K′ is the key adjusted to the hash block size, ipad and opad are fixed inner and outer padding constants, ⊕ denotes bitwise exclusive OR, and ∥ denotes concatenation. If the original key is longer than the hash block size, it is first hashed; if shorter, it is padded according to the HMAC specification.

The nested construction is important. Simply appending a secret to a message and hashing it, for example H(m ∥ K), is not a general substitute for HMAC and may be unsafe in protocols using certain hash constructions.

HMAC operation

1. The sender and receiver establish a secret key using a secure key-management process.
2. The sender calculates an HMAC over the exact message or canonical representation.
3. The sender transmits the message and tag.
4. The receiver independently calculates the expected tag.
5. The receiver performs a constant-time comparison and rejects the message if the tags differ.

HMAC authenticates the data covered by the tag. It does not automatically prevent replay. A protocol should therefore include a nonce, timestamp, sequence number or unique transaction identifier, and the receiver must enforce its validity.

HMAC applications

HMAC is used in API request signing, webhook verification, secure cookies, token integrity and parts of network-security protocols. A common API design includes the HTTP method, path, selected headers, body digest, timestamp and nonce in a canonical string before calculating the HMAC. Canonicalisation matters: if the sender and receiver construct different byte sequences from the same apparent request, verification will fail.

The key must be protected as carefully as any other authentication credential. Logging an HMAC key, embedding it in public client software or reusing one key across unrelated services increases the impact of compromise.

4. Salts and Password Security

Passwords are low-entropy human secrets. Attackers can guess common passwords, use dictionaries, exploit leaked password lists and try combinations offline. The central password-storage principle is:

Store a password-derived verifier, not the plaintext password and not a reversibly encrypted copy used merely as a password record.

Salts

A salt is a random, unique value generated for a password record. It is not required to be secret and is normally stored alongside the derived password value. The salt changes the computation for each account:

A unique salt prevents identical passwords from producing identical stored values and defeats precomputed rainbow tables. It does not, by itself, make a weak password strong or prevent offline guessing. An attacker who steals a database can also obtain the salts and attempt guesses one password at a time.

Salts should be generated with a cryptographically secure random-number generator and should be long enough for the selected password-hashing scheme. They should not be reused across accounts or generated from predictable information such as a username.

Password-based key derivation

A password-based key-derivation function (KDF) transforms a password and salt into a derived value, often while consuming configurable time and memory. The cost should be high enough to slow attackers but acceptable for legitimate authentication.

Important password-oriented algorithms include:

• Argon2id: A modern memory-hard design that combines resistance to GPU attacks with configurable time and memory costs. It is a strong default when supported by a maintained library.
• scrypt: A memory-hard KDF designed to make large-scale parallel guessing more expensive.
• bcrypt: A mature adaptive password-hashing function with a cost parameter. It remains useful, although its memory use is more limited than newer memory-hard designs.
• PBKDF2: An iterated construction based on a pseudorandom function, commonly HMAC. It is widely standardised and useful where compliance or platform support requires it, but its computational profile may be more favourable to specialised hardware than memory-hard alternatives.

A generic iteration relationship for an iterative construction can be expressed conceptually as:

where c is the configured iteration count. This is an engineering approximation, not a universal security equation. Actual cost depends on the algorithm, hardware, memory, parallelism and implementation. Password KDF parameters must therefore be benchmarked on the production environment and reviewed periodically.

Verification process

For registration or password change:

1. Validate password policy and risk controls.
2. Generate a fresh random salt.
3. Select approved KDF parameters.
4. Compute the derived verifier.
5. Store the algorithm identifier, parameters, salt and verifier.
6. Never store the plaintext password in logs, analytics or routine application data.

For login:

1. Retrieve the stored algorithm, parameters, salt and verifier.
2. Apply the same KDF to the supplied password.
3. Compare the result using a constant-time comparison.
4. If valid, authenticate the user and record security-relevant events without recording the password.
5. If parameters are obsolete, rehash after successful authentication, subject to safe account handling.

A typical record is conceptually:

algorithm | version | cost parameters | salt | derived verifier

The exact encoding should follow the selected library or standard rather than being invented by an application team.

Peppering

A pepper is an additional secret kept separately from the password database, usually in a secrets manager, hardware security module or protected application configuration. It can increase resistance if only the database is stolen, but it introduces key-management and availability requirements. A pepper is not a replacement for a unique salt, strong KDF and good passwords.

Password policy and authentication controls

Length is generally more valuable than arbitrary complexity rules. Systems should permit long passphrases, avoid unnecessary composition restrictions, screen against known breached passwords and support password managers. Rate limiting, progressive delays, multifactor authentication and monitoring reduce online guessing; they do not solve the problem of an offline database attack.

Password reset processes are part of password security. A reset token should be random, short-lived, single-use and bound to the appropriate account or transaction. Security questions based on public information should not be treated as strong authentication.

5. Worked Application: Verifying an API Request

Assume a service and a client share a secret HMAC key. The request contains a timestamp, HTTP method, path and body. They construct the canonical string:

POST\n/v1/payments\n1700000000\n{"amount":2500,"currency":"GBP"}

The client computes an HMAC-SHA-256 tag over the exact UTF-8 bytes of this string and sends the request, timestamp and tag. The server rejects the request if:

• the timestamp lies outside the permitted time window;
• the nonce has already been used;
• the canonical form is invalid;
• the HMAC comparison fails; or
• the client is not authorised for the requested operation.

The HMAC establishes that the request was generated by someone possessing the shared key and that the covered contents were not changed. The timestamp and nonce address replay. TLS should still be used to protect transport confidentiality and reduce exposure of metadata. HMAC and TLS solve different, complementary problems.

6. Illustrative Case Study: Legacy Password Database Migration

Background

A fictional online-learning provider, Northbridge Learning, discovers that its legacy application stores passwords using a single unsalted SHA-1 digest. The database is accessible to a compromised reporting account.

Problem

Because SHA-1 is fast and the passwords are unsalted, an attacker can use common-password lists and precomputed data to recover many credentials offline. Recovered passwords may also unlock users’ email or financial accounts because of password reuse.

Analysis

The organisation first preserves evidence and revokes the compromised database credentials. It determines which records were exposed, checks authentication logs and informs affected users according to applicable breach-reporting obligations. It does not attempt to “decrypt” the SHA-1 values: hashes are not encryption, and the practical attack is guessing candidate passwords and comparing their digests.

Approach and solution

Northbridge introduces a maintained Argon2id implementation with a unique random salt per account and parameters benchmarked to an acceptable login delay. The record stores the algorithm version, salt, parameters and verifier. During the migration period, a successful login against a legacy record triggers a password rehash. Accounts that do not log in are required to complete a secure reset after a defined deadline.

The provider also invalidates active sessions, adds rate limiting and breached-password screening, encourages multifactor authentication, separates secrets from the database and improves monitoring. Password reset tokens are made single-use and time-limited. Staff are trained not to place credentials in support tickets or logs.

Outcome

The new design limits the value of a database copy: each password guess must be processed through an expensive KDF, and identical passwords no longer reveal matching records. It does not guarantee safety if users choose weak passwords, if the application leaks passwords during login, or if an attacker compromises the KDF service and its secret-management environment.

Learning points

1. A fast general-purpose hash is unsuitable for password storage.
2. Salts prevent precomputation and cross-account equality analysis but are not secret.
3. KDF parameters must be selected, benchmarked and upgraded.
4. Password security includes reset, session, monitoring and incident-response processes.
5. Cryptographic controls must be combined with access control and user education.

7. Limitations, Governance and Professional Practice

Cryptographic mechanisms cannot compensate for poor key management, vulnerable application logic or incorrect data boundaries. Developers must specify exactly what is authenticated, encode data consistently, reject malformed inputs and use trusted libraries. Homemade cryptographic algorithms and ad hoc constructions should be avoided.

Use constant-time comparison for MAC tags and derived password values where practical. Constant-time techniques reduce timing leakage; they do not eliminate all side channels, so secure libraries and system-level controls remain important. Error messages should not reveal whether an account exists or which part of authentication failed, while audit records should contain enough information for investigation without storing secrets.

Organisations should maintain an approved cryptographic standard covering algorithms, minimum versions, key lifetimes, random-number generation, secrets storage and deprecation. Requirements may arise from privacy law, sector regulation, contractual controls and standards such as NIST guidance, ISO/IEC 27001 programmes or payment-security frameworks. Compliance does not replace threat modelling.

Artificial intelligence can assist defenders by identifying abnormal login patterns, grouping credential-stuffing attempts and prioritising exposed secrets. However, AI systems must not receive plaintext passwords or unprotected keys. Generated code must be reviewed because an apparently plausible implementation may use an ordinary hash, omit a salt, compare tags unsafely or authenticate an incomplete message.

Best-Practice Checklist

• Use SHA-256, SHA-3 or another approved modern hash for integrity and non-password applications.
• Do not use MD5 or SHA-1 for new security-sensitive designs.
• Use HMAC or an authenticated-encryption scheme rather than an unkeyed hash for message authentication.
• Authenticate the complete, canonical message and address replay separately.
• Use a unique, unpredictable salt for every password record.
• Prefer Argon2id, or use scrypt, bcrypt or PBKDF2 when justified by compatibility or standards.
• Store algorithm identifiers and parameters with the verifier.
• Benchmark and periodically increase KDF costs.
• Use a secret pepper only with a sound secrets-management design.
• Never log plaintext passwords, password-derived secrets or long-lived HMAC keys.
• Support password managers, long passphrases, breached-password screening and multifactor authentication.
• Use maintained, tested cryptographic libraries and obtain independent review for security-critical code.

Chapter Summary

Cryptographic hashes produce fixed-length digests and support integrity, but an unkeyed hash does not authenticate a sender. Hash security includes preimage, second-preimage and collision resistance; collision attacks benefit from the birthday effect. Encryption is reversible with a key, whereas hashing is designed as a one-way transformation.

HMAC combines a secret key with a cryptographic hash to provide message integrity and shared-key authentication. It does not provide confidentiality or automatically prevent replay, so protocols require timestamps, nonces or sequence numbers where replay matters. Correct canonicalisation and constant-time tag comparison are important implementation practices.

Password storage requires a unique salt and an intentionally expensive password-oriented KDF such as Argon2id, scrypt, bcrypt or PBKDF2. Salts are public and defeat precomputation, while peppers are optional additional secrets that require careful management. Secure password practice also includes long passphrases, breached-password checks, rate limiting, multifactor authentication, secure reset procedures, monitoring and regular parameter upgrades. The overall lesson is that cryptographic primitives must be selected according to their security purpose and integrated with sound governance and operational controls.

Key Terms

Cryptographic hash: A deterministic function that maps arbitrary-length input to a fixed-length digest.

Digest: The fixed-length output produced by a hash function.

Preimage resistance: Difficulty of finding an input that produces a specified digest.

Second-preimage resistance: Difficulty of finding a different input with the same digest as a specified input.

Collision resistance: Difficulty of finding any two different inputs with the same digest.

Avalanche effect: Large unpredictable output change caused by a small input change.

Hashing: One-way transformation used for integrity and other applications.

Encryption: Reversible transformation that protects confidentiality using a key.

Message authentication code: A keyed tag providing integrity and shared-key authentication.

HMAC: A standard hash-based message authentication code construction.

Salt: A unique random value added to password processing and stored with the verifier.

Password KDF: A function that derives a value from a password using configurable cost.

Verifier: A stored password-derived value used to check a supplied password.

Pepper: An additional secret used in password processing and stored separately from the database.

Offline attack: Guessing performed against stolen data without interacting with the live service.

Replay attack: Reuse of a previously valid message or authentication exchange.

Canonicalisation: Converting equivalent data into one agreed byte representation.

Constant-time comparison: A comparison designed to reduce timing information leakage.

Memory-hard function: A function designed to require substantial memory as well as computation.

Credential stuffing: Automated use of leaked username-password pairs against other services.

Review Questions

Short Answer Questions

1. What is the difference between preimage resistance and collision resistance?
2. Why does a plain hash not authenticate the source of a message?
3. What security services does HMAC provide, and which major service does it not provide?
4. Why should a password salt be unique even though it is not secret?
5. Name two password-oriented KDFs and explain why a general-purpose hash is unsuitable for password storage.

Long Answer Questions

1. Explain the differences among hashing, encryption, HMAC and digital signatures, including their security purposes and key requirements.
2. Describe the complete password registration and login process using a salted, adaptive KDF.
3. Explain the construction and operational use of HMAC, including canonicalisation, constant-time comparison and replay protection.
4. Compare Argon2id, scrypt, bcrypt and PBKDF2 in terms of design purpose, configurability and deployment considerations.
5. Discuss the organisational and governance controls required to support secure password storage beyond the cryptographic algorithm itself.

Analytical / Application Questions

1. A company stores unsalted SHA-256 password hashes and publishes a downloadable database backup. Analyse the attack opportunities and propose a staged migration plan.
2. An API verifies an HMAC correctly but accepts any timestamp and allows a valid request to be submitted repeatedly. Identify the vulnerability and design an appropriate replay-prevention mechanism.
3. A development team proposes storing passwords with AES encryption so that the application can recover them when users forget them. Evaluate the proposal and recommend a safer account-recovery design.

Multiple Choice Questions with Answers

Question 1

Which property makes it difficult to find an input that produces a specified hash digest?

A. Collision resistance

B. Preimage resistance

C. Replay resistance

D. Forward secrecy

Correct Answer: B. Preimage resistance

Question 2

What is the primary purpose of a unique salt in password storage?

A. To keep the password database confidential

B. To make the password recoverable by administrators

C. To prevent precomputation and identical passwords producing identical values

D. To replace the need for an expensive KDF

Correct Answer: C. To prevent precomputation and identical passwords producing identical values

Question 3

Which mechanism provides integrity and shared-secret authentication but normally not confidentiality?

A. HMAC

B. Plain SHA-256

C. Symmetric encryption alone

D. A random salt

Correct Answer: A. HMAC

Question 4

Why is a single application of SHA-256 generally unsuitable for password storage?

A. It cannot produce a fixed-length output

B. It is designed to be fast, which helps attackers make many guesses

C. It requires a public key

D. It always reveals the plaintext

Correct Answer: B. It is designed to be fast, which helps attackers make many guesses

Question 5

Which algorithm is specifically designed as a modern memory-hard password-hashing option?

A. MD5

B. RSA

C. Argon2id

D. CRC32

Correct Answer: C. Argon2id

Question 6

What additional control is needed to prevent replay of a correctly authenticated API request?

A. A larger hash digest only

B. A timestamp, nonce or sequence number with validation

C. Removal of the HMAC key

D. Publishing the salt

Correct Answer: B. A timestamp, nonce or sequence number with validation

Question 7

In an HMAC expression, what does the symbol ∥ represent?

A. Bitwise exclusive OR

B. Modular division

C. Concatenation

D. Random permutation

Correct Answer: C. Concatenation

Question 8

Which statement about a password salt is correct?

A. It must be kept secret in a hardware security module

B. It should normally be unique and stored with the derived verifier

C. It should be the username for ease of recovery

D. It eliminates offline guessing

Correct Answer: B. It should normally be unique and stored with the derived verifier

Question 9

Why should MAC tags be compared using a constant-time comparison where practical?

A. To increase the tag length

B. To reduce timing leakage during verification

C. To encrypt the message automatically

D. To generate a new salt

Correct Answer: B. To reduce timing leakage during verification

Question 10

A digital signature differs from HMAC primarily because a digital signature uses:

A. A private/public key pair and can be verified with a public key

B. No cryptographic key

C. Only a password salt

D. A reversible hash operation

Correct Answer: A. A private/public key pair and can be verified with a public key