Cryptography, PKI and Secure Communications

Introduction and Context

Networked systems routinely carry credentials, financial transactions, personal information, software updates and administrative commands across networks that may be monitored or manipulated. Network security therefore requires more than access control. It requires mechanisms that protect confidentiality, integrity, authentication and non-repudiation.

Cryptography provides the mathematical and procedural tools for these goals. Public key infrastructure (PKI) applies those tools through certificates, certification authorities and trust policies. Secure communication technologies—including TLS, SSH, IPsec and virtual private networks (VPNs)—use combinations of cryptographic techniques to protect real network traffic.

Cryptography does not automatically make a system secure. Weak passwords, stolen private keys, incorrect certificate validation, obsolete algorithms, insecure endpoints and poor key management can defeat strong mathematics. This chapter consequently treats cryptography as part of a complete security architecture.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish encryption, hashing, message authentication and digital signatures.
2. Explain symmetric and asymmetric cryptography and select suitable uses for each.
3. Describe certificate operation, trust chains and PKI components.
4. Explain key generation, storage, rotation, revocation and recovery.
5. Distinguish the purposes and operating contexts of TLS, SSH, IPsec and VPNs.
6. Identify common implementation failures and recommend secure practices.

1. Security Services and Cryptographic Building Blocks

1.1 The four principal security objectives

• Confidentiality prevents unauthorised disclosure. Encryption is the principal cryptographic control.
• Integrity provides assurance that data has not been altered. Hashes, message authentication codes and signatures support this objective.
• Authentication establishes the identity of a user, device, service or message origin.
• Non-repudiation provides evidence that can help demonstrate who signed data, although legal effect depends on policy, identity proofing, key protection and applicable law. A digital signature is not, by itself, proof that a particular human intentionally approved an action.

Availability is also a fundamental security objective, but cryptography generally supports it indirectly—for example, by preventing unauthorised modification of routing or configuration data.

1.2 Symmetric encryption

Symmetric encryption uses the same secret key, or computationally related secret keys, for encryption and decryption. A simplified model is:

Here, P is plaintext, C is ciphertext, K is a secret key, and Enc and Dec are encryption and decryption algorithms. The relationship assumes a correct algorithm, a secret key, and correct use of the algorithm's mode and parameters.

Modern examples include AES and ChaCha20. AES commonly uses 128-, 192- or 256-bit keys. ChaCha20 uses a 256-bit key. Key length alone does not determine security: nonce reuse, weak random numbers or a vulnerable mode can compromise an otherwise strong cipher.

Block ciphers process fixed-size blocks and require a mode of operation for longer messages. Authenticated encryption with associated data (AEAD), such as AES-GCM or ChaCha20-Poly1305, provides confidentiality and an authentication tag. The associated data is authenticated but not encrypted, making it useful for selected headers. A nonce must be unique for a given key; accidental nonce reuse in some AEAD modes can seriously compromise confidentiality and integrity.

1.3 Asymmetric encryption and key agreement

Asymmetric cryptography uses a public key and a mathematically related private key. The public key may be distributed, while the private key must remain secret. Public key algorithms support key agreement, signatures and, in some systems, encryption.

Common families include RSA, elliptic-curve cryptography (ECC) and elliptic-curve Diffie–Hellman (ECDH). Diffie–Hellman key exchange allows two parties to derive a shared secret over an observable network. It does not, by itself, authenticate either party; an attacker can perform a man-in-the-middle attack unless authentication is added.

Asymmetric operations are generally slower and require larger computational resources than symmetric encryption. Secure protocols therefore use asymmetric cryptography to authenticate parties or establish a short-lived symmetric key, then use symmetric AEAD for bulk data.

1.4 Hash functions

A cryptographic hash function maps input of arbitrary length to a fixed-length digest:

M is the message, H is the hash function and h is the digest. A secure hash should make it computationally difficult to find a preimage, a second message with the same digest, or any two colliding messages. SHA-256 and SHA-3 are widely used modern hash families. MD5 and SHA-1 should not be selected for collision-sensitive security applications.

A hash is not encryption. It is not reversible, and it does not authenticate the sender unless combined with a secret or a digital signature. Passwords should not normally be stored as ordinary hashes; password-specific functions such as Argon2id, scrypt or bcrypt should be used with a unique salt and suitable cost parameters.

1.5 Message authentication codes

A message authentication code (MAC) combines a message with a shared secret:

T is an authentication tag. A recipient possessing K recalculates the tag and compares it using a safe comparison method. HMAC is a widely deployed construction based on a hash function. A MAC provides integrity and origin authentication to parties sharing the key, but it does not provide non-repudiation because either party could have generated the tag.

1.6 Digital signatures

A digital signature is generated with a private key and verified using the corresponding public key. In practice, the message is normally hashed before signing:

sk is the signing private key, pk is the public verification key, s is the signature, and M is the message. The verifier also needs confidence that the public key belongs to the claimed signer. Certificates and PKI address this identity-binding problem.

Signatures provide integrity and evidence of key-based origin. They do not conceal the message. Secure signature schemes include RSA-PSS and modern elliptic-curve schemes such as Ed25519, subject to platform and protocol support.

2. Key Management: The Operational Centre of Cryptography

Key management covers the complete life cycle of cryptographic keys:

1. Generation: use an approved cryptographically secure random-number generator.
2. Distribution: deliver keys through a protected channel or establish them using authenticated key agreement.
3. Storage: protect private keys with access controls, encryption at rest, hardware security modules (HSMs), smart cards or platform key stores.
4. Use: restrict algorithms, purposes and identities; avoid reusing one key for unrelated functions.
5. Rotation: replace keys according to risk, cryptoperiod, compromise indicators and policy.
6. Backup and recovery: protect backups at least as carefully as live keys and test restoration.
7. Revocation and destruction: invalidate compromised or expired keys and securely erase them when no longer required.

A key hierarchy reduces exposure. For example, a root key may protect key-encryption keys, which protect data-encryption keys. Separation of duties can require two authorised administrators to approve sensitive operations. Key inventories should record owner, purpose, algorithm, creation time, expiry, location and status.

A lost private key can make encrypted data unrecoverable; a stolen private key can permit impersonation. Recovery planning must therefore balance confidentiality against business continuity. Escrow may be appropriate in tightly governed environments, but it introduces an additional high-value target and must be legally and technically controlled.

3. Public Key Infrastructure

3.1 Purpose and components

PKI is a governance and technology framework for binding identities to public keys. Its principal components are:

• Certificate authority (CA): issues and signs certificates.
• Root CA: a trust anchor, normally kept offline or strongly protected.
• Intermediate CA: issues end-entity certificates under a root or another intermediate.
• Registration authority (RA): performs or coordinates identity validation.
• End entity: a server, device, user or service holding a certificate and private key.
• Relying party: a system that decides whether to trust a certificate.
• Certificate repository and status service: distributes certificates and provides revocation information.

A certificate is a signed data structure containing an identity, a public key, validity limits, issuer information, permitted uses and other constraints. In the widely used X.509 model, the CA signature allows a relying party to detect unauthorised certificate modification, but trust still depends on validating the chain and the identity claims.

3.2 Certificate validation

A client should validate at least:

• the signature on each certificate in the chain;
• that the chain terminates at a locally trusted root;
• validity dates, including reliable system time;
• the intended hostname or subject identity;
• key usage and extended key usage;
• basic constraints and path length constraints;
• revocation status where required by policy; and
• algorithm and key-strength requirements.

Hostname validation is especially important. A certificate issued for example.org should not be accepted for an unrelated hostname merely because the certificate is signed by a trusted CA.

Certificate revocation may be communicated through certificate revocation lists (CRLs) or the Online Certificate Status Protocol (OCSP). Revocation checking can be incomplete because clients may be offline, status services may be unavailable and some implementations apply soft-fail behaviour. Short certificate lifetimes, automated renewal and effective incident response are therefore also important.

3.3 Trust models and limitations

The public Web PKI uses a hierarchical trust model in which operating systems and browsers distribute trusted root certificates. Organisations may operate private PKIs for internal devices, employees and services. Other models include a web of trust and certificate pinning, each with different administration and failure characteristics.

A CA validates according to its stated policy; it does not guarantee that a website is honest, that its server is free of malware or that a user will make a safe decision. A trusted certificate proves control of an identity under the CA's validation rules, not overall trustworthiness.

Figure: A relying party validates a certificate chain from the server certificate to a trusted root, while also checking identity and usage constraints.

4. Secure Communication Protocols

4.1 TLS

Transport Layer Security (TLS) protects application traffic, especially HTTP. A modern TLS connection negotiates protocol parameters, authenticates the server with a certificate, establishes shared key material and then protects records using authenticated encryption. TLS 1.3 is preferred where supported; obsolete SSL and early TLS versions should be disabled according to current organisational policy.

TLS normally provides server authentication. Mutual TLS (mTLS) additionally requires a client certificate, useful for service-to-service authentication, device identity and controlled enterprise access. TLS protects data in transit between endpoints, but it does not protect data after decryption inside a server or client, nor does it correct compromised endpoints.

4.2 SSH

Secure Shell (SSH) provides encrypted remote login, command execution, file transfer and tunnelling. Server host keys authenticate the server; user authentication may use public keys, certificates, passwords or multi-factor mechanisms. Administrators should verify host keys, protect private keys with passphrases or hardware-backed storage, disable obsolete algorithms and restrict accounts and commands. SSH port forwarding can securely carry another application, but it can also bypass network controls if poorly governed.

4.3 IPsec

IPsec operates mainly at the network layer. Its Authentication Header (AH) provides integrity and origin authentication in limited contexts, while Encapsulating Security Payload (ESP) can provide confidentiality, integrity and authentication. Internet Key Exchange (IKE), commonly IKEv2, negotiates security associations and keys.

Transport mode protects the payload of an IP packet; tunnel mode encapsulates the original packet, commonly for gateway-to-gateway or remote-access VPNs. IPsec is often transparent to applications but can be more complex to troubleshoot across address translation, routing and policy boundaries.

4.4 VPN technologies

A VPN creates a protected logical path across an untrusted or shared network. The term describes a security function, not one single protocol. IPsec VPNs are common for site-to-site links; TLS-based VPNs are common for remote access; WireGuard is a modern VPN protocol using a compact design and contemporary cryptography. A VPN protects traffic between defined endpoints, not automatically all traffic from malware, insecure applications or a compromised device.

5. Worked Example: Choosing Controls for an Online Service

An organisation operates an online student portal. Students use browsers, administrators manage Linux servers, and two internal services exchange records.

Analysis: Browser traffic needs confidentiality and server authentication; administrator sessions need verified server identity and strong user authentication; service-to-service traffic needs machine authentication and restricted authorisation. A single shared password would provide poor accountability and create a large compromise domain.

Design: The portal uses TLS with a publicly trusted certificate and strict hostname validation. Administrators use SSH with individual keys, MFA and a restricted management network. Internal services use mTLS with certificates issued by a private CA. Sensitive records are encrypted using AEAD, with keys held by a managed key service. Logs record certificate identity, administrator identity and authorisation decisions.

Operational controls: The private CA has an offline root and protected intermediate CA. Certificates have automated renewal, inventory and alerting for impending expiry. Private keys are non-exportable where possible. Revocation and emergency replacement procedures are tested.

Result: Each communication path receives a control matched to its threat and operational context. TLS does not replace authorisation; SSH does not replace patching; and the VPN used by administrators does not make every internal service trustworthy.

6. Illustrative Case Study: Certificate Compromise

This is a fictional illustrative case.

Background

A regional college uses a private PKI for laptops and internal web applications. A web server's private key is accidentally copied to a public code repository.

Problem

An attacker can impersonate the server to clients that trust the corresponding certificate. Simply deleting the repository copy does not remove already downloaded copies.

Analysis

The security team identifies the certificate serial number, affected systems, key exposure period and possible logs of fraudulent use. It determines that the key must be treated as compromised even though misuse has not been proven.

Approach and solution

The team revokes the certificate where supported, issues a replacement certificate with a newly generated key, removes the exposed secret from repositories and access logs, rotates related credentials, and checks for unauthorised certificates. It also adds secret-scanning controls, limits private-key export, improves permissions and trains developers.

Outcome

The immediate impersonation risk is reduced, but the event demonstrates that revocation is not a substitute for prevention. Certificate inventories, short lifetimes, automated deployment and tested incident procedures reduce future exposure.

Learning points

Private keys are high-value credentials. Certificate identity, key protection and deployment security must be managed together. A CA cannot protect an organisation when private keys are copied into uncontrolled locations.

7. Threats, Limitations and Governance

Common failures include using obsolete algorithms, accepting any certificate without validation, disabling hostname checks to “fix” an error, reusing nonces, generating predictable keys, storing private keys in source code, relying on passwords alone, failing to rotate keys, and assuming encryption provides anonymity or endpoint security.

Cryptographic choices should follow current standards and risk assessments rather than informal algorithm lists. Organisations should maintain an approved cryptographic policy, inventory cryptographic dependencies, plan for algorithm deprecation and monitor vendor advisories. Key material and personal data may be subject to privacy, financial, export-control or electronic-signature regulation. Evidence from signatures must be retained according to legal and organisational requirements.

Quantum computing is a contemporary planning issue. A sufficiently capable cryptographically relevant quantum computer could threaten widely used public-key systems based on factoring or discrete logarithms. Symmetric systems are affected differently, and migration planning should prioritise cryptographic inventories, algorithm agility and new post-quantum standards. Organisations should not replace proven controls hastily, but they should identify long-lived confidential data and systems that cannot be upgraded quickly.

Artificial intelligence can assist certificate inventory, anomaly detection and code scanning, but generated recommendations require expert review. An AI system must not be given private keys or trusted authority to approve certificates without strong controls, logging and human accountability.

8. Best-Practice Checklist

• Prefer TLS 1.3 and current, authenticated encryption suites where compatible.
• Validate certificate chain, hostname, dates, constraints and intended use.
• Use unique, strong keys generated by approved random sources.
• Protect private keys with least privilege, encryption and hardware-backed controls where justified.
• Use AEAD and never reuse a nonce with the same key.
• Use password-specific hashing with salts and appropriate work factors.
• Separate keys by purpose, environment and tenant.
• Automate certificate renewal and monitor expiry.
• Maintain revocation, compromise-response and recovery procedures.
• Use individual administrator identities, MFA and audited SSH access.
• Treat VPN access as a network control, not as proof that users or devices are trusted.
• Test cryptographic implementations, configurations and disaster recovery.

Chapter Summary

Cryptography supplies mechanisms for confidentiality, integrity and authentication. Symmetric encryption is efficient for bulk data, while asymmetric cryptography supports key agreement and signatures. Hashes provide fixed-length integrity representations but do not authenticate messages alone; MACs authenticate parties sharing a secret; digital signatures provide public verification but require a trustworthy identity-to-key binding.

PKI binds identities to public keys through certificates, CAs and validation policies. Trust depends on correct chain building, hostname checking, validity and usage constraints, private-key protection and operational governance. TLS protects application transport, SSH protects administration, IPsec protects network-layer traffic, and VPNs create controlled protected paths. None of these technologies eliminates endpoint compromise, poor authorisation or weak operations. Effective security combines suitable algorithms with disciplined key management, monitoring, incident response and continual modernisation.

Key Terms

Cryptography: The study and application of techniques for protecting information.

Symmetric encryption: Encryption using a shared secret key.

Asymmetric cryptography: Cryptography using related public and private keys.

Plaintext: Readable input before encryption.

Ciphertext: Encrypted output.

Hash function: A function producing a fixed-length digest from arbitrary input.

Salt: Random data added to a password before password hashing.

MAC: A keyed tag providing message integrity and shared-key authentication.

Digital signature: A private-key-generated value verified with a public key.

Nonce: A value intended for limited or single use in a cryptographic operation.

AEAD: Authenticated encryption with associated data.

Certificate: A signed binding between an identity and a public key.

Certificate authority: An entity that issues and signs certificates.

PKI: The people, policies, processes and technologies managing public-key trust.

Root CA: A trust anchor at the top of a certificate hierarchy.

TLS: A protocol for protecting application-layer communications.

SSH: A protocol for secure remote access and tunnelling.

IPsec: A suite for protecting IP-layer communications.

VPN: A protected logical network path across an untrusted network.

HSM: Hardware designed to protect and operate on cryptographic keys.

Review Questions

Short Answer Questions

1. What are the differences between encryption and hashing?
2. Why must an ECDH exchange be authenticated?
3. What is the purpose of a certificate authority?
4. State two checks a TLS client should perform on a server certificate.
5. Why is private-key protection central to PKI security?

Long Answer Questions

1. Compare symmetric encryption, MACs and digital signatures in terms of keys, confidentiality, integrity and authentication.
2. Explain the certificate chain from a root CA to an end-entity certificate and describe how a client validates it.
3. Describe the key life cycle and explain the risks associated with poor rotation, backup and destruction.
4. Compare TLS, SSH and IPsec by layer, typical use and authentication method.
5. Explain why a VPN does not automatically make an organisation's internal network trustworthy.

Analytical / Application Questions

1. A company disables hostname validation because its application reports certificate errors. Analyse the security consequences and propose a safer remedy.
2. Design a cryptographic architecture for a small online banking service, including public access, administrator access, service-to-service traffic and key recovery.
3. A developer commits a production private key to a public repository. Construct an incident-response plan covering containment, certificate replacement, investigation and preventive improvements.

Multiple Choice Questions with Answers

Question 1

Which mechanism provides confidentiality and integrity using a shared secret key?

A. Plain hash

B. MAC alone

C. Authenticated encryption

D. Certificate chain

Correct Answer: C. Authenticated encryption

Question 2

What is the primary purpose of a digital signature?

A. To hide the message contents

B. To provide verifiable integrity and key-based origin

C. To compress a message

D. To distribute a symmetric key without authentication

Correct Answer: B. To provide verifiable integrity and key-based origin

Question 3

Which item is normally a trust anchor for certificate validation?

A. An end-entity server certificate

B. A session nonce

C. A root CA certificate

D. A password hash

Correct Answer: C. A root CA certificate

Question 4

Why must an AEAD nonce generally be unique for a given key?

A. Nonce reuse can compromise security

B. It increases certificate validity

C. It makes public keys secret

D. It removes the need for authentication

Correct Answer: A. Nonce reuse can compromise security

Question 5

Which protocol is primarily designed for secure remote administration?

A. SSH

B. DNS

C. ARP

D. FTP

Correct Answer: A. SSH

Question 6

What does mTLS add to ordinary server-authenticated TLS?

A. Client certificate authentication

B. IP packet fragmentation

C. Password hashing only

D. Removal of encryption

Correct Answer: A. Client certificate authentication

Question 7

Which statement about a hash function is correct?

A. It is normally reversible with a public key

B. It provides confidentiality by itself

C. It maps input to a fixed-length digest

D. It always authenticates the sender

Correct Answer: C. It maps input to a fixed-length digest

Question 8

Which technology primarily operates at the network layer?

A. TLS

B. IPsec

C. SSH

D. HMAC

Correct Answer: B. IPsec

Question 9

A stolen private key most directly creates which risk?

A. Improved availability

B. Unauthorised impersonation or decryption, depending on its use

C. Automatic certificate renewal

D. Stronger password storage

Correct Answer: B. Unauthorised impersonation or decryption, depending on its use

Question 10

What is the safest response when a server private key is exposed publicly?

A. Only delete the public repository copy

B. Ignore it unless misuse is proven

C. Treat it as compromised and replace the key and certificate

D. Disable all hashing permanently

Correct Answer: C. Treat it as compromised and replace the key and certificate