7. Caching, Content Delivery and Edge Computing

Introduction: performance as a distributed-systems problem

A web application rarely serves every request by executing its complete business logic, querying its primary database, and returning data from a single data centre. Such an architecture may work for a small user base, but latency, infrastructure cost, and failure risk increase rapidly as traffic grows and users become geographically distributed. Caching and content delivery address this problem by placing frequently required data closer to the point of use and by avoiding repeated computation.

Caching is not merely a performance optimisation. It changes the behaviour of a system. A cached response can be older than the source of truth; an invalidation failure can expose incorrect information; and a popular object can overload both the cache and its backing service. Consequently, cache design belongs within system architecture, data governance, reliability engineering, and security—not only within application tuning.

This chapter examines caching at several layers: browsers, reverse proxies, application memory, distributed cache clusters, databases, and content delivery networks (CDNs). It then considers cache consistency, eviction and invalidation, hot keys, stampedes, observability, and edge computing. The central design question is not simply “How can the system be made faster?” but “Which data may be reused, for how long, by whom, and with what consequences if it is stale?”

Learning objectives

After studying this chapter, a learner should be able to:

1. Explain the purpose and trade-offs of browser, application, database, distributed, CDN, and edge caching.

2. Select suitable cache-aside, read-through, write-through, write-back, and write-around patterns.

3. Design eviction, expiration, invalidation, and cache-consistency strategies.

4. Diagnose hot keys, cache stampedes, penetration, and avalanche failures.

5. Compare CDN and edge-computing architectures for global low-latency delivery.

6. Define cache metrics, operational controls, security safeguards, and governance policies.

7. Evaluate caching decisions in realistic application and organisational contexts.

A cache is a deliberate approximation of authoritative data

A cache is a temporary store containing a representation of data that can be obtained, or reconstructed, from another source. The authoritative source may be a database, an object store, an API, or a computation. The cache attempts to reduce one or more of the following:

• response latency;
• load on an origin service or database;
• bandwidth consumption;
• computational cost; and
• geographical distance between users and content.

The basic sequence is a cache lookup. A hit returns a usable cached representation; a miss causes the system to obtain the data from the origin and commonly place it in the cache. The value of caching depends on the probability of a hit, the cost of a miss, the cost of maintaining the cache, and the business tolerance for staleness.

A useful approximation for average latency is:

`Average latency ≈ hit ratio × cache latency + miss ratio × origin latency`

This simplified expression does not include eviction overhead, network variability, or failures, but it illustrates why a high hit ratio is valuable only when the cached response is correct and the cache itself is reliable. A cache that returns invalid data is not a successful cache.

Important terminology

A cache key identifies an item in the cache. It should include every request attribute that materially changes the response, such as resource identifier, language, tenant, device class, or permission context. A value is the stored representation. TTL (time to live) limits how long an item remains fresh. Freshness is the period during which a value may be served without revalidation. Stale data is older than the intended freshness boundary, although some systems deliberately serve stale data during an origin failure.

A cache hit occurs when an acceptable value is available; a cache miss occurs when it is absent, expired, or unusable. Hit ratio and miss ratio measure effectiveness, but should be interpreted alongside latency and correctness. Eviction removes an item because of space pressure or policy. Invalidation removes or marks an item unusable because the underlying data changed. Expiration is time-based; invalidation is usually event- or version-based.

A hot key is disproportionately accessed, for example a globally popular product or a breaking-news article. Cache penetration occurs when repeated requests for absent or uncacheable objects reach the origin. A cache stampede occurs when many requests simultaneously discover that an item is missing or expired and all rebuild it. A cache avalanche is a broad loss or simultaneous expiration of many cached objects, producing a large origin surge.

The layered caching architecture

Caching is usually layered. Each layer has a distinct owner, scope, and consistency model; adding a layer does not remove the design obligations of the others.

Browser and client caching

Browsers cache static files and HTTP responses according to headers such as `Cache-Control`, `Expires`, `ETag`, and `Last-Modified`. A directive such as `max-age` specifies a freshness lifetime. `no-store` instructs a compliant client not to retain a response, while `no-cache` generally means that a stored response must be revalidated before reuse; it does not necessarily mean “do not cache.”

Versioned asset names are a strong technique for static resources. A file such as `app.8f31.js` can receive a long immutable lifetime because a new build produces a new name. Without versioning, a browser may continue using an old JavaScript or CSS file after deployment. Personal, confidential, or permission-sensitive responses require stricter controls, especially where shared devices or browser history create privacy risks.

Reverse proxies and CDNs

A reverse proxy sits between clients and origin servers. It can terminate TLS, compress responses, apply rate limits, and cache HTTP objects. A CDN distributes such proxy nodes across many points of presence. Static images, style sheets, videos, software packages, and suitably designed API responses are common CDN candidates.

A CDN reduces distance and origin traffic, but it does not automatically make dynamic content safe to cache. The cache key, request method, cookies, authorization headers, query parameters, and response headers must be considered. A poorly configured CDN can serve one user’s personalised response to another user. CDN invalidation also varies by provider and may not be instantaneous, so deployments should combine controlled TTLs with versioned assets where possible.

Application-memory caching

An application process may retain frequently used objects in memory. This is extremely fast and simple, but each process has a separate cache. A load balancer may direct successive requests to different instances, producing low effective reuse. Application-local caches are therefore suitable for small, stable, process-specific data such as configuration, parsed templates, or metadata, but less suitable for shared rapidly changing state.

Memory limits must be explicit. An unbounded cache can cause garbage-collection pressure or process termination. Entries should have size limits, TTLs, and an eviction policy. In a multi-instance service, local caching may be combined with a shared distributed cache, with careful attention to stale copies.

Distributed caches

A distributed cache such as a Redis- or Memcached-style service provides shared access across application instances. Data may be partitioned across nodes using consistent hashing or another distribution method. Replication can improve availability, but introduces additional complexity and may be asynchronous. A distributed cache is not automatically a durable database: it may lose data during failure, and its operational guarantees must be explicitly understood.

Keys should be namespaced, for example `catalog:product:v2:481`, to support ownership, migration, and controlled invalidation. Serialization formats, maximum value size, compression, connection pooling, timeouts, and failure behaviour should be standardised. A service must decide whether cache unavailability causes a graceful origin fallback, a degraded response, or a failure. Indefinite blocking on a failed cache can be worse than bypassing it.

Database and storage caching

Databases use buffer pools, query plans, indexes, and operating-system caches to reduce disk access. These internal mechanisms should not be confused with application response caching. A query result cache may be effective for repeated read-only queries, but broad invalidation and changing query parameters often reduce its usefulness. Proper indexing, query design, connection management, and data modelling remain necessary; caching cannot repair an inefficient or incorrect query indefinitely.

Object storage and file systems also cache blocks and metadata. At this layer, locality of reference is important: recently or frequently accessed blocks are more likely to be useful. Database caching is generally invisible to application code, while application and CDN caches require explicit policies and testing.

Choosing a caching pattern

The appropriate pattern depends on the relationship between reads, writes, freshness, and failure tolerance.

Cache-aside (lazy loading) leaves the application responsible for reading and populating the cache. On a miss, it reads the source, stores the result, and returns it. Writes update the source and then invalidate or update the cache. The pattern is flexible and common, but concurrent misses and invalidation races require design attention.

Read-through makes the cache responsible for loading a missing value from the origin. The application interacts primarily with the cache. This can standardise loading and reduce application code, but the cache layer becomes more complex and may not support every data-access pattern.

Write-through writes to the cache and synchronously propagates the write to the source. It provides a relatively predictable cache state, at the cost of write latency and dependence on both systems. Write-back acknowledges the cache write and persists to the source later. It offers fast writes but risks data loss and ordering problems, so it is appropriate only where durability and recovery have been carefully addressed.

Write-around writes directly to the source and does not populate the cache. It avoids filling the cache with data that may never be read, but the next read incurs a miss. A common hybrid is to invalidate on write and use cache-aside on read.

For HTTP content, conditional requests provide an economical consistency mechanism. A client or CDN sends an entity tag using `If-None-Match`; if unchanged, the origin returns `304 Not Modified` without the full representation. This preserves validation while reducing transfer cost.

Eviction, expiration, and invalidation

A finite cache requires a policy for deciding what to remove. Least recently used (LRU) evicts items that have not been accessed for the longest time. Least frequently used (LFU) favours items with high access frequency. First-in-first-out (FIFO) removes the oldest insertion, while random or approximate policies can be computationally inexpensive. Modern systems often use approximate algorithms because exact tracking of recency and frequency is costly.

No single policy is universally best. LRU performs well when recent access predicts future access. LFU can protect highly popular objects but may retain historically popular items after demand changes. A practical design may combine TTL with an approximate LRU or LFU policy. Item size should also be considered: evicting one very large object may release more memory than evicting several small ones.

Expiration controls staleness, but identical TTLs can cause synchronized expiry. Jitter—adding a small random variation to TTL—reduces simultaneous rebuilds. A stale-while-revalidate approach serves a recently expired response while one request refreshes it in the background. Stale-if-error can preserve availability during a temporary origin failure, provided the business accepts old content.

Invalidation is harder than insertion because a changed source record may appear in multiple caches and representations. Common approaches include:

1. TTL-based invalidation, simple but potentially stale.

2. Explicit deletion, immediate in principle but vulnerable to missed events.

3. Versioned keys, which avoid deleting old data by changing the namespace.

4. Event-driven invalidation, in which a domain event identifies affected keys.

5. Tag or surrogate-key invalidation, useful when one update affects many rendered objects.

A robust system defines what happens when invalidation fails. Retries, durable event queues, reconciliation jobs, and monitoring can reduce the risk. For critical data, a cache should be treated as an optimisation whose failure cannot violate the system’s fundamental correctness rules.

Consistency and concurrency hazards

Caching creates a temporal separation between a value and its source. Strong consistency may be required for account balances, inventory reservations, access permissions, or legal records. Eventual consistency may be acceptable for recommendations, view counts, search suggestions, or product descriptions, provided users understand the delay.

A frequent race occurs when a stale reader repopulates the cache after a newer writer has invalidated it. Version numbers, timestamps, compare-and-set operations, or write ordering can prevent an older result from overwriting a newer one. Cache keys should sometimes include a data version, allowing a new version to become active without deleting every old key.

For stampede prevention, the system may use request coalescing or a single-flight lock: one request rebuilds the item while other requests wait briefly or receive stale data. Locks need an expiry, bounded wait time, and failure handling. A distributed lock is not a universal solution; network partitions and lock expiry can produce duplicate work. Probabilistic early refresh, in which popular entries are refreshed before expiration, can smooth load.

A negative cache stores the fact that an object does not exist. A short negative TTL protects the origin against repeated requests for invalid identifiers, but it must be used cautiously: a resource created soon afterward should not remain invisible for too long. Rate limiting and input validation complement negative caching.

CDN and edge computing: from delivery to execution

A CDN primarily delivers cached content from geographically distributed nodes. Edge computing extends this idea by executing code, transformations, authentication checks, routing logic, or lightweight data processing near users. Edge functions may personalise a response, select an origin, resize an image, reject abusive traffic, or implement an experiment without sending every request to a central region.

The edge is valuable when latency, bandwidth, or regulatory locality matters. However, edge environments commonly impose limits on execution time, memory, package size, statefulness, and observability. Stateful transactions, complex joins, and long-running workflows usually remain better suited to regional services or central data systems. A sound architecture partitions work: perform latency-sensitive, stateless, or easily replicated operations at the edge, while retaining authoritative state in controlled services.

Edge caching raises data-residency and privacy questions. A response containing personal information may be replicated into multiple jurisdictions. Cache keys must not expose tokens or sensitive identifiers, and encryption must cover transport and, where appropriate, stored content. Organisations should define retention, deletion, audit, and incident-response procedures for edge nodes and provider platforms.

Observability and operational governance

Cache monitoring should extend beyond aggregate hit ratio. Important measures include hit ratio by endpoint and region, p50/p95/p99 cache latency, origin requests caused by misses, eviction rate, memory utilisation, stale-response rate, invalidation delay, stampede events, and error rate. A high hit ratio can conceal severe performance problems if hits are slow or concentrated in one region.

Tracing should identify cache lookup, origin fetch, deserialization, and invalidation activity. Logs should record key classes rather than sensitive raw keys. Dashboards should distinguish a normal miss from a cache outage. Capacity tests should simulate cold starts, regional failure, popular-object surges, invalidation storms, and degraded origin performance.

Governance requires ownership. Each cached dataset should have a responsible team, classification, freshness target, maximum permitted staleness, invalidation mechanism, capacity budget, and fallback behaviour. Security review should test cache poisoning, unkeyed request attributes, host-header manipulation, cross-tenant leakage, and accidental caching of authenticated responses. A cache key must represent the dimensions that determine authorisation as well as content.

An illustrative case: a global learning platform

Background

Consider LearnSphere, a fictional online learning platform serving recorded lectures, course pages, quizzes, and learner dashboards from a primary regional deployment. A new international enrolment campaign produces a sharp increase in traffic. Users in distant regions report slow video start times and intermittent failures on course pages.

Situation and analysis

The engineering team discovers that large lecture files are served directly from object storage, while course-page requests repeatedly query the same database records. A single cache expiration time causes thousands of course pages to refresh simultaneously each hour. The learner dashboard includes personalised enrolment and progress data, but a reverse-proxy rule caches responses without consistently varying on the authentication context. This creates a serious confidentiality risk.

The team classifies content by freshness and sensitivity. Public lecture media and versioned static assets are highly cacheable. Public course metadata may tolerate a short stale period. Progress, grades, and enrolment data must not be shared through a public cache and require user- or tenant-specific handling.

Strategy and solution

LearnSphere places a CDN in front of lecture media and immutable assets. Each deployment generates content-hashed filenames, allowing long browser and CDN lifetimes. Course pages use a CDN only for the public portion; personalised fragments are generated by the application. A distributed cache using cache-aside stores course metadata with namespace-qualified keys and a moderate TTL with random jitter.

Course updates emit events containing course identifiers and versions. Consumers invalidate relevant metadata keys and CDN surrogate tags. If an event is delayed, the short TTL bounds staleness. A single-flight mechanism prevents many requests from rebuilding the same popular course entry. Negative caching protects the database from repeated invalid course identifiers.

The dashboard response is marked private and is not stored in a shared CDN cache. Its application cache key includes tenant and learner identity where caching is justified, and sensitive values have short lifetimes. Origin fallback is bounded by timeouts and circuit breakers so that a cache outage does not create unlimited database traffic. Regional CDN nodes absorb media demand, while authoritative enrolment and grading remain in the central transactional services.

Outcome and learning points

The architecture reduces repeated origin reads and improves global media delivery without weakening the correctness of grades or access control. More importantly, performance and security are treated together. The case demonstrates that content classification should precede cache configuration, that deployment versioning is often safer than mass deletion, and that a cache policy must specify both normal operation and failure behaviour.

Challenges, limitations, and professional judgement

Caching can increase total system complexity. Developers must reason about two states—the source and the cached representation—instead of one. Invalidation bugs are difficult to reproduce because they depend on timing, geography, and concurrency. Memory and CDN costs may also exceed expectations when keys have excessive cardinality, responses are large, or personalisation prevents reuse.

Common mistakes include caching by URL while ignoring headers or tenant identity; using one TTL for every data type; treating a distributed cache as durable storage; allowing unbounded values; deleting keys without considering concurrent repopulation; and measuring only hit ratio. Another mistake is adding caching before establishing a performance baseline. Profiling should first identify whether latency arises from network distance, database work, serialisation, locking, or application logic.

Caching may also amplify an outage. If a cache fails open, an origin may be overwhelmed; if it fails closed, users may receive errors despite a healthy origin. Capacity planning must include the cold-cache scenario. Operational runbooks should provide controlled bypass, emergency TTL changes, origin protection, and rollback procedures.

Emerging directions

Modern platforms increasingly combine CDN caching, programmable edge functions, regional data services, and observability tools into a single delivery architecture. Intelligent prefetching and adaptive TTLs use traffic patterns to refresh likely-demanded content, although predictive approaches must be evaluated for cost, bias, and privacy. AI-assisted systems can help detect anomalous cache misses, identify hot keys, and recommend policies, but automated decisions should remain bounded by explicit freshness and security rules.

The broader trend is toward computation moving closer to users while authoritative state remains deliberately placed. This is not a universal migration to the edge. It is a selective distribution of work according to latency sensitivity, data sovereignty, consistency requirements, and operational capability.

Best-practice design sequence

A practical design review can follow this sequence:

1. Classify each response by sensitivity, volatility, size, and geographic demand.

2. Identify the authoritative source and define the maximum acceptable staleness.

3. Choose the cache layer and pattern that match the access and write behaviour.

4. Design complete keys, including tenant, language, version, and authorisation dimensions where relevant.

5. Select TTL, eviction, invalidation, and stampede controls together.

6. Define cache failure, origin fallback, and cold-start behaviour.

7. Instrument hits, misses, latency, memory, invalidation delay, and correctness signals.

8. Test security, regional failure, synchronized expiration, hot keys, and deployment rollback.

9. Assign ownership and document the policy for every important cached dataset.

Chapter Summary

Caching improves performance by reusing data and computation, but it introduces temporal separation from the authoritative source. Browser caches, application memory, distributed caches, database mechanisms, reverse proxies, CDNs, and edge runtimes operate at different scopes and require different controls. Cache-aside, read-through, write-through, write-back, and write-around patterns represent distinct trade-offs between latency, durability, complexity, and consistency.

Effective design combines TTL and eviction policies with explicit invalidation, versioned keys, jitter, stale-while-revalidate, negative caching, and request coalescing where appropriate. Hot keys, stampedes, penetration, avalanche failures, cache poisoning, and cross-tenant leakage require both technical and operational safeguards. CDNs distribute content, while edge computing also executes selected logic near users; neither removes the need for an authoritative source and clear governance. Performance measurements must include latency, origin load, staleness, invalidation delay, and failure behaviour, not hit ratio alone.

Key Terms

Cache: Temporary storage of reusable data or representations.

Cache-aside: A pattern in which the application reads and populates the cache explicitly.

Cache hit: A request successfully served from an acceptable cached value.

Cache key: Identifier used to locate a cached object.

Cache miss: A request for which no usable cached value is available.

Cache penetration: Repeated requests for absent or uncacheable data reaching the origin.

Cache stampede: Concurrent rebuilding of the same missing or expired item.

Cache avalanche: A large origin surge caused by simultaneous cache loss or expiration.

CDN: A geographically distributed network that delivers content from nearby proxy nodes.

Edge computing: Execution of selected processing near users or data sources.

Eviction: Removal of cache entries because of capacity or policy.

Eventual consistency: A model in which replicas may temporarily differ but converge later.

Hot key: A cache key receiving unusually concentrated demand.

Invalidation: Making a cached value unusable because its source has changed.

Negative caching: Temporarily caching the fact that a requested object does not exist.

Origin: The authoritative service or store from which cached data is obtained.

Stale-while-revalidate: Serving an expired value while refreshing it asynchronously.

TTL: Time-to-live controlling the permitted freshness lifetime of an entry.

Write-through: A pattern that synchronously writes changes through the cache to the source.

Review Questions

Short Answer Questions

1. Distinguish cache expiration from cache invalidation.

2. Why are versioned asset names useful for browser and CDN caching?

3. Define a hot key and state one method for managing it.

4. What is the principal difference between a CDN and edge computing?

5. Why should a cache key include tenant or authorisation dimensions when relevant?

Long Answer Questions

1. Explain the layered caching architecture and discuss the role and limitations of each major layer.

2. Compare cache-aside, read-through, write-through, write-back, and write-around patterns.

3. Discuss eviction, TTL, invalidation, versioning, and stale-while-revalidate as a combined policy problem.

4. Analyse the consistency and concurrency hazards introduced by caching in a distributed web application.

5. Explain how observability, governance, and security should be incorporated into a caching strategy.

Analytical / Application Questions

1. A product catalogue has frequent reads, occasional updates, and users in many countries. Design a caching architecture and justify its layers, TTL, invalidation method, and failure behaviour.

2. At every hour, a service experiences a sudden database surge even though its cache hit ratio is normally high. Diagnose likely causes and propose a technically justified response.

3. A CDN is accidentally serving one customer’s dashboard data to another customer. Analyse the probable cache-key and response-policy failures and specify corrective controls.

22. Multiple Choice Questions with Answers

Question 1

Which caching pattern requires the application to load data into the cache after a miss?

A. Cache-aside

B. Write-back

C. Write-around

D. FIFO caching

Correct Answer: A. Cache-aside

Question 2

Which design most directly reduces synchronized expiration of many entries?

A. Using identical short TTLs

B. Adding random TTL jitter

C. Disabling eviction

D. Increasing database indexes

Correct Answer: B. Adding random TTL jitter

Question 3

A cache key for a personalised response must include the user or tenant context primarily to prevent:

A. Disk fragmentation

B. Database deadlocks

C. Cross-user data leakage

D. Video compression

Correct Answer: C. Cross-user data leakage

Question 4

What is the principal purpose of a CDN?

A. To replace all transactional databases

B. To execute long-running business workflows

C. To distribute content closer to users

D. To guarantee strong consistency for every response

Correct Answer: C. To distribute content closer to users

Question 5

Which technique allows one request to rebuild an expired popular item while others avoid duplicate rebuilding?

A. Request coalescing or single-flight control

B. Increasing key cardinality

C. Removing all TTLs

D. Disabling origin timeouts

Correct Answer: A. Request coalescing or single-flight control

Question 6

Why are content-hashed filenames effective for static assets?

A. They make personalised data public

B. They permit long lifetimes while new versions receive new keys

C. They eliminate the need for TLS

D. They force every request to the origin

Correct Answer: B. They permit long lifetimes while new versions receive new keys

Question 7

Which situation is most suitable for stale-if-error behaviour?

A. Serving an old account balance during a failed transaction

B. Serving an old public article during a brief origin outage

C. Serving another user’s private dashboard

D. Serving an expired authorisation decision indefinitely

Correct Answer: B. Serving an old public article during a brief origin outage

Question 8

What is cache penetration?

A. A cache node being geographically close to users

B. Repeated requests for nonexistent objects reaching the origin

C. A write being synchronously persisted

D. A CDN compressing a response

Correct Answer: B. Repeated requests for nonexistent objects reaching the origin

Question 9

Which metric is insufficient by itself for evaluating cache effectiveness?

A. Origin load

B. Invalidation delay

C. Hit ratio

D. p95 latency

Correct Answer: C. Hit ratio

Question 10

Which workload is generally least appropriate for execution at a stateless edge function?

A. Routing a request to a nearby origin

B. Resizing a public image

C. Applying a lightweight header rule

D. Running a long, stateful transactional workflow

Correct Answer: D. Running a long, stateful transactional workflow