Performance, Caching, and Scalability

Introduction

A backend system is successful only when it delivers correct results within acceptable time, under expected demand, and with reasonable infrastructure cost. Functional correctness answers whether an application produces the right result; performance engineering asks how quickly, consistently, and efficiently it does so. In Node.js and Django applications, performance is influenced by application code, database design, network behavior, serialization, operating-system resources, deployment architecture, and traffic patterns.

Performance work should not begin with arbitrary optimization. A disciplined engineer first defines service objectives, measures the system, identifies the dominant constraint, applies a targeted intervention, and measures again. This chapter presents that cycle together with caching, compression, pagination, query optimization, rate limiting, load balancing, horizontal scaling, content-delivery networks (CDNs), and capacity planning.

Learning Objectives

By the end of this chapter, you should be able to:

1. Define and interpret latency, throughput, utilization, error rate, and availability metrics.
2. Profile Node.js, Django, and database workloads without confusing symptoms with causes.
3. Design cache strategies, compression policies, and pagination schemes.
4. Optimize database queries and recognize common ORM performance problems.
5. Explain rate limiting, load balancing, horizontal scaling, and CDN delivery.
6. Use queueing relationships and capacity-planning calculations responsibly.
7. Evaluate trade-offs involving consistency, cost, reliability, security, and maintainability.

1. Performance as a Multi-Dimensional Property

Performance is not a single number. The most useful dimensions are:

• Latency: elapsed time between a request and its response. It is commonly reported using percentiles, such as p50, p95, and p99, rather than only an average.
• Throughput: completed requests, jobs, or bytes per unit of time.
• Concurrency: the number of operations in progress at a given moment.
• Utilization: the proportion of a resource’s capacity being consumed, such as CPU, memory, database connections, or network bandwidth.
• Error rate: the proportion of requests that fail, often separated into client errors, server errors, timeouts, and dependency failures.
• Availability: the proportion of time a service is usable according to an explicit measurement policy.
• Efficiency: useful work delivered per unit of compute, memory, storage, or monetary cost.

An average latency can conceal serious user experience problems. If most requests take 100 ms but a small fraction take 8 seconds, the average may appear acceptable while p99 is unacceptable. Percentiles describe distribution behavior: p95 is the value at or below which approximately 95% of observations fall. Percentiles do not add linearly across services, and a percentile of a sample is an estimate rather than a guarantee.

Service-level objectives

A service-level indicator (SLI) is a measured property, such as successful request proportion or p95 latency. A service-level objective (SLO) is a target for that indicator, for example, “99.9% of authenticated API requests succeed within 500 ms during a calendar month.” A service-level agreement (SLA) is a contractual commitment that may include remedies. These concepts should not be conflated.

A practical performance budget may specify a maximum response time, an error budget, a maximum database load, and a cost ceiling. Product requirements should identify which requests are interactive, which can be asynchronous, and which require strong consistency.

2. A Measurement-First Engineering Method

2.1 Establish a baseline

Before changing code, record a representative baseline. Include request rate, response-size distribution, latency percentiles, status codes, CPU and memory use, garbage-collection activity, database query count and duration, cache hit ratio, connection-pool usage, and dependency latency. Measurements should be tagged by endpoint, method, tenant or region where appropriate, and release version.

Observability has three complementary forms:

• Metrics aggregate behavior over time and support alerting.
• Logs record discrete events and diagnostic context.
• Traces follow a request through application code and dependencies.

A correlation or trace identifier allows an engineer to connect an HTTP request, database statements, cache operations, and downstream calls. Logs must avoid passwords, access tokens, payment data, and unnecessary personal information.

2.2 Profiling application code

A profiler samples or instruments execution to show where time and allocations are spent. CPU profiles can expose expensive loops, regular-expression misuse, JSON transformation, synchronous filesystem calls, and excessive serialization. Memory profiles can reveal retained objects, unbounded collections, large response buffers, and event listeners that are never removed.

Node.js uses an event loop for JavaScript execution and asynchronous I/O coordination. A long synchronous function blocks other requests on the same process. Common causes include synchronous filesystem or cryptographic functions, large compression operations, CPU-heavy parsing, and inefficient algorithms. CPU-intensive work may be moved to worker threads, a job queue, or a separate service, but the communication and operational costs must be considered.

In Django, profiling should distinguish view execution, template rendering, middleware, serialization, ORM work, and external services. Development tools that show SQL queries are useful for diagnosis but should not automatically be enabled in production because of overhead and possible data exposure.

A benchmark must resemble production. Synthetic tests should specify request mix, payload sizes, authentication behavior, warm-up period, concurrency, data volume, cache state, and dependency behavior. A single “requests per second” result without these conditions is difficult to interpret.

2.3 Finding bottlenecks

A bottleneck is a resource or operation that limits system progress. CPU saturation, database locks, slow disk, exhausted connection pools, network bandwidth, event-loop blocking, and a slow third-party API can all be bottlenecks. Improving a non-bottleneck may produce no meaningful result.

A useful diagnostic sequence is:

1. Confirm the user-visible symptom and affected endpoint or workflow.
2. Compare latency percentiles and error rates with the SLO.
3. Use traces to divide total time among application, database, cache, and remote calls.
4. Check saturation indicators and queue lengths.
5. Inspect representative slow queries and execution plans.
6. Form one testable hypothesis.
7. Change one major variable, then repeat the measurement.

3. Quantitative Models for Capacity and Latency

3.1 Little’s Law

For a stable system, Little’s Law relates average concurrency, throughput, and average time in the system:

Here, L is average number of items in the system (requests), λ is average throughput in requests per unit time, and W is average time in the system in the same time unit. The assumptions are approximate stability over the observation interval and a well-defined system boundary. It applies to queues and workflows, not only HTTP servers.

For example, if an API completes 200 requests per second and the average response time is 0.25 seconds, estimated average in-flight requests are:

This does not state the p95 latency, maximum concurrency, or required number of servers. It is an average relationship and should not be used as a substitute for load testing.

3.2 Utilization and headroom

If a server has a measured sustainable capacity of 800 requests per second and expected peak demand is 560 requests per second, the demand-to-capacity ratio is:

Here ρ is a dimensionless utilization ratio. The calculation assumes that both values use the same workload mix and that “capacity” is defined by an SLO, not merely by the point at which the server stops responding. Operating close to saturation can cause disproportionately large queueing delays, so capacity planning normally reserves headroom for bursts, failures, deployments, and traffic growth.

3.3 Tail latency and fan-out

A request that performs several downstream operations may have latency determined by the slowest operation, especially when calls run in parallel. If a request depends on many independent services, the probability that at least one is slow increases with the number of dependencies. This is one reason to reduce unnecessary fan-out, set deadlines, use bulk operations, and degrade gracefully.

4. Caching: Speed Through Reuse

A cache stores a reusable representation so that a later request can avoid expensive computation, database access, or network transfer. Common layers include browser caches, CDN caches, reverse proxies, application memory, Redis or Memcached, and database buffer caches.

A cache is beneficial when the cost of a cache lookup plus occasional miss processing is lower than recomputing the result. The main design questions are:

• What is the cache key?
• How long may the value remain stale?
• How is invalidation performed?
• What happens on a miss or cache outage?
• Can one tenant or user receive another user’s data?
• What is the memory limit and eviction policy?

A useful hit ratio is:

where H ranges from 0 to 1 for a defined interval and key space. Hit ratio alone is insufficient: a cache can have a high hit ratio while returning stale or low-value data, or while its misses are extremely expensive.

4.1 Cache patterns

Cache-aside is widely used. The application reads the cache first; on a miss it reads the database, returns the result, and writes it to the cache. On an update, the application changes the source of truth and deletes or refreshes the relevant cache entry. Cache-aside is simple but can produce stale reads and a stampede when many requests miss simultaneously.

Read-through caching delegates loading to the cache layer. Write-through caching writes the cache and source of truth as part of an update path. Write-back caching delays persistence and therefore requires strong recovery controls; it is unsuitable when data loss is unacceptable.

Expiration can be fixed time-to-live (TTL), sliding expiration, or explicit invalidation. TTL reduces indefinite staleness but does not guarantee freshness during the TTL. Explicit invalidation can be precise but is harder to implement across services.

A cache stampede occurs when many requests simultaneously regenerate an expired value. Mitigations include adding randomized TTL jitter, single-flight locking so one request refreshes a key, background refresh, stale-while-revalidate behavior, and bounded fallbacks. Negative caching—temporarily caching a confirmed “not found” result—can reduce repeated misses, but short TTLs are prudent because the resource may subsequently be created.

4.2 Correctness and security

Never construct a cache key that omits a dimension affecting authorization or representation. A response varying by user, locale, permissions, currency, or feature flag must include those dimensions or be excluded from shared caching. Sensitive responses generally require private caching rules. Cache poisoning can occur when untrusted input changes a key or response interpretation. Cache invalidation should be observable and tested as a correctness mechanism, not treated as a minor optimization.

4.3 Node.js and Django examples

In Node.js, a Redis client can support cache-aside for expensive product summaries. The application should use bounded timeouts, handle Redis failure without taking down the primary request path when safe, and serialize values with explicit versioning. In Django, per-view caching, template-fragment caching, and low-level cache APIs can be combined with database queries. Cache keys should incorporate deployment or schema versions when representation changes.

5. Compression and Efficient Representation

Compression reduces bytes transmitted at the cost of CPU time and sometimes latency. It is most useful for text-based JSON, HTML, CSS, and JavaScript. Already compressed formats such as JPEG, PNG, MP4, and ZIP usually gain little and may become larger.

Modern HTTP services commonly negotiate Brotli or gzip through Accept-Encoding and respond with Content-Encoding. A shared cache must respect the Vary: Accept-Encoding response directive or otherwise risk serving an encoding a client cannot decode. Compression should be disabled or carefully controlled for secrets embedded with attacker-controlled text because compression-size differences can contribute to side-channel attacks such as BREACH-like attacks.

Best practice includes setting a minimum response size, compressing text rather than binary media, avoiding compression bomb risks, measuring CPU and bandwidth effects, and using streaming where appropriate. Minification, efficient JSON fields, appropriate media formats, and HTTP/2 or HTTP/3 multiplexing complement compression; none eliminates the need to measure.

6. Pagination and Data-Access Efficiency

Returning an entire collection creates latency, memory, serialization, and bandwidth problems. Pagination bounds response size and gives the client a controllable traversal mechanism.

Offset pagination uses a page number and size, conceptually LIMIT size OFFSET offset. It is simple and supports jumping to a page, but large offsets may require the database to scan and discard many rows. Inserts or deletes during traversal can also cause duplicates or omissions.

Cursor or keyset pagination uses a stable ordering and a continuation token. For example, with descending order by (created_at, id), the next query selects rows lexicographically after the last returned key. The unique identifier breaks timestamp ties. Keyset pagination is efficient for long feeds but does not naturally support arbitrary page jumps.

A cursor should be opaque, validated, bounded in lifetime where appropriate, and protected against tampering. The API should enforce a maximum page size and return a continuation token only when more data exists. Pagination does not solve an unindexed query: the ordering and filter columns still need suitable indexes.

7. Database Query Optimization

Databases often dominate backend latency because they perform disk access, locking, joins, sorting, and aggregation. Optimization starts with the actual query and its execution plan, not assumptions based only on ORM code.

7.1 The N+1 query problem

An ORM query that loads a list of objects and then retrieves a related object for each item creates one query for the list plus one query per item. With 500 items, this may produce 501 queries. Django’s select_related is appropriate for many-to-one or one-to-one joins; prefetch_related performs additional controlled queries for many-to-many or reverse relations. In Node.js, query builders and data mappers offer analogous eager-loading or batching facilities. A deliberate number of set-based queries is usually preferable to repeated round trips.

7.2 Indexes and plans

An index accelerates selected predicates and orderings but consumes storage and slows writes. A composite index should reflect common filtering and ordering patterns. Column order matters: an index on (tenant_id, status, created_at) is not interchangeable with (status, tenant_id, created_at) for every query. Selectivity, cardinality, write frequency, and query workload should guide decisions.

Use EXPLAIN or an equivalent execution-plan tool to inspect scans, index use, join algorithms, estimated versus actual row counts, sorts, and temporary operations. A query may ignore an index because the predicate matches a large fraction of rows, a function prevents index use, statistics are stale, or implicit type conversion occurs. Do not add indexes solely because a column appears in a table.

7.3 Connection pools and transactions

A connection pool limits concurrent database sessions and amortizes connection setup. An excessively large pool can overload the database; an excessively small pool causes application-side waiting. Pool size must be considered across all application instances, because total possible connections are approximately the sum of each instance’s configured maximum, subject to infrastructure behavior.

Transactions should be short and scoped to the required consistency boundary. Holding a transaction open while calling a remote service increases lock duration and failure exposure. Isolation level affects phenomena such as dirty reads, non-repeatable reads, and phantom reads; choose it according to correctness requirements rather than speed alone.

8. Rate Limiting and Resilience

Rate limiting protects a service from accidental overload, abusive clients, and unfair resource consumption. It can be applied per IP address, authenticated account, API key, tenant, endpoint, or costly operation. Identity selection must account for proxies and shared networks; an IP-only limit can unfairly group many legitimate users.

The token bucket algorithm maintains tokens at rate r up to capacity B. A request consuming c tokens is allowed when at least c tokens are available; otherwise it is delayed or rejected. If the bucket is empty, the maximum burst is limited by the tokens accumulated, while long-term average consumption is bounded by the refill rate. Implementations need a reliable clock, atomic updates in distributed deployments, and clear response semantics such as HTTP 429 with Retry-After when appropriate.

Other strategies include fixed windows, sliding windows, and leaky buckets. Fixed windows are simple but allow boundary bursts. Distributed rate limiting requires shared state or a deliberate per-instance policy. Limits should be documented and monitored; emergency limits must not silently block health checks, security operations, or critical administrative workflows.

Timeouts, bounded retries with exponential backoff and jitter, circuit breakers, bulkheads, and idempotency keys complement rate limiting. Retrying non-idempotent operations without safeguards can create duplicate charges or records. A retry budget should be smaller than the failure amplification it is intended to solve.

9. Load Balancing and Horizontal Scaling

A load balancer distributes requests among healthy backend instances. Algorithms include round robin, weighted distribution, least connections, and latency-aware routing. Health checks should test meaningful readiness, not merely whether a process accepts TCP connections. A readiness check may fail while an instance is draining, lacks database connectivity, or has not loaded required configuration.

Horizontal scaling adds instances; vertical scaling adds resources to an instance. Horizontal scaling improves capacity and fault isolation but introduces coordination, consistency, deployment, and observability challenges. To scale horizontally, application instances should ideally be stateless: sessions belong in a shared store or signed client mechanism, uploaded files use shared object storage, and background jobs use a durable queue.

Sticky sessions can simplify some legacy designs but reduce balancing flexibility and complicate failover. If used, their failure behavior must be explicit. Distributed locks and scheduled jobs require coordination that cannot be assumed merely because several instances share code.

A reverse proxy can terminate TLS, route traffic, enforce request limits, serve static assets, and provide connection management. It must correctly forward scheme, host, and client-IP information, with trusted-proxy configuration to prevent spoofing.

10. CDNs and Edge Delivery

A CDN places cacheable content near users. It is highly effective for versioned static assets, images, fonts, downloads, and selected public API responses. Benefits include lower geographic latency, reduced origin bandwidth, and protection against some traffic bursts. It does not automatically accelerate uncached, personalized, or highly dynamic requests.

Cache-control directives communicate policy. Long-lived caching is safest for immutable, content-hashed assets such as app.4f8c.js. Dynamic content requires careful Cache-Control, Vary, authorization, and invalidation rules. Purging a CDN is not instantaneous in every system, so versioned URLs are often more reliable than emergency invalidation.

CDNs also introduce governance questions: where user data is processed, which jurisdictions receive traffic, how logs are retained, and how access-controlled content is protected. Signed URLs, origin access controls, TLS, and least-privilege credentials help protect private objects.

11. Capacity Planning

Capacity planning connects expected demand to tested service capacity. A basic process is:

1. Forecast request volume, payload size, user activity, and batch work.
2. Define peak, sustained, and burst demand separately.
3. Load-test a production-like workload against an explicit SLO.
4. Measure per-instance sustainable capacity and resource saturation.
5. Add headroom for growth, failures, deployments, and uncertainty.
6. Validate database, cache, queue, network, and third-party limits.
7. Establish scaling triggers and revisit assumptions.

Suppose a tested instance sustains 250 requests per second while meeting the p95 latency objective. Forecast peak demand is 900 requests per second. With a planning headroom factor of 1.30, planned capacity is:

The minimum instance count by capacity is:

Here N is an integer instance count and ⌈x⌉ means the ceiling of x. This is a planning estimate, not a guarantee. It assumes the tested workload remains representative, instances scale approximately independently, and the load balancer and dependencies can support the result. A further failure requirement may require more instances; for example, “N+1” means retaining capacity after one instance fails, but the exact policy depends on architecture and risk.

A useful cost model includes compute, database, cache, storage, egress, observability, and operational labor. The cheapest architecture at average demand may be unsafe at peak demand. Autoscaling should use leading indicators such as queue depth or request concurrency when possible, not only CPU, because CPU may remain low while a database or remote dependency is saturated.

12. Applied Case: Illustrative Online Learning Platform

Background

Consider an illustrative university learning platform built with Django, PostgreSQL, Redis, and a JavaScript API gateway. During enrollment periods, students report slow course pages and occasional timeouts. The platform normally handles 120 requests per second but reaches 600 requests per second during registration.

Problem and analysis

Tracing shows that course-page p95 latency rises from 450 ms to 3.8 seconds. Database CPU reaches 88%, while application CPU remains below 55%. Query logs reveal 1 + N queries for instructors and modules, and a feed endpoint uses large offsets. A cache dashboard shows only a 42% hit ratio because keys omit the course version and expire simultaneously at the top of each hour. Static JavaScript is served from the application servers, consuming bandwidth.

The evidence indicates that the primary bottlenecks are database round trips, inefficient pagination, synchronized cache misses, and origin delivery of static assets—not insufficient application CPU.

Approach and solution

The team first establishes a p95 target of 800 ms for course pages and creates a representative load test. It then:

1. Uses controlled eager loading for course, instructor, and module relationships.
2. Adds and verifies indexes for tenant/course filters and stable ordering using execution plans.
3. Replaces offset pagination for the activity feed with a signed keyset cursor using (created_at, id).
4. Implements cache-aside for public course metadata, with versioned keys, randomized TTL jitter, and single-flight refresh.
5. Moves versioned static assets to a CDN and enables Brotli for JavaScript and CSS.
6. Adds per-user and per-IP rate limits to expensive search operations.
7. Runs multiple Django instances behind a load balancer and moves session state to a shared store.
8. Sets database pool limits based on the total number of application instances.
9. Adds alerts for p95 latency, error rate, database saturation, cache hit ratio, queue depth, and connection wait time.

Outcome

In the next load test, course-page p95 falls to 620 ms at 600 requests per second, database CPU falls to 63%, and origin bandwidth decreases substantially. The team does not claim that the system is permanently solved: it records assumptions, repeats the test after data growth, and schedules a failure test for the cache and database. The key outcome is a measured improvement against an explicit objective rather than an unverified collection of optimizations.

Learning points

The case demonstrates that performance is end-to-end. A faster application process cannot compensate for inefficient queries. Caching requires correct keys and invalidation. Pagination is a data-access design decision. CDN adoption helps static delivery but does not remove the need to scale the origin. Finally, capacity decisions should be based on tested SLO-compliant capacity, not a theoretical maximum.

13. Challenges, Governance, and Contemporary Practice

Performance improvements can conflict with consistency, privacy, fairness, and cost. A stale cache may display an old permission or course status. Aggressive rate limits may disadvantage users behind shared networks or with assistive technologies that generate more requests. Detailed traces may expose personal data. Autoscaling can increase cloud expenditure rapidly during an attack or runaway retry loop.

Use data minimization, access controls, retention limits, encryption, and redaction in observability systems. Treat performance logs as potentially sensitive operational data. Document cache freshness, deletion behavior, regional processing, and recovery responsibilities. Conduct load tests only with authorization and synthetic or appropriately protected data.

Contemporary systems increasingly use OpenTelemetry-compatible traces and metrics, service meshes, edge functions, asynchronous workflows, and automated performance regression testing. Artificial intelligence can assist with anomaly detection, query-plan explanation, and forecasting, but it should not autonomously change production limits or indexes without review. AI-generated diagnoses remain hypotheses requiring measurements, security review, and rollback plans.

Best-Practice Checklist

• Define latency, availability, throughput, and error objectives before optimizing.
• Measure p50, p95, and p99, not averages alone.
• Profile production-like workloads and inspect traces across dependencies.
• Fix the bottleneck, then verify the result with a controlled comparison.
• Bound response size with pagination and enforce maximum page limits.
• Use indexes based on actual query plans and write/read trade-offs.
• Design cache keys around representation, tenant, identity, and authorization.
• Plan for cache misses, stampedes, stale data, and cache outages.
• Compress suitable text responses and use immutable, versioned assets.
• Keep instances stateless where practical and externalize durable state.
• Apply timeouts, rate limits, bounded retries, and idempotency controls.
• Test capacity at SLO compliance, including dependency and failure scenarios.
• Protect telemetry and performance data as sensitive operational information.

Chapter Summary

Performance engineering is an empirical discipline concerned with latency, throughput, concurrency, utilization, errors, availability, and cost. Profiling and distributed tracing identify where time and resources are actually spent. Little’s Law provides a useful relationship among average concurrency, throughput, and response time, while capacity calculations translate tested per-instance performance into deployment estimates with explicit assumptions.

Caching reduces repeated work but introduces freshness, invalidation, stampede, authorization, and consistency concerns. Compression, efficient representations, pagination, query planning, indexes, and ORM discipline reduce computation and data transfer. Rate limiting protects shared resources; timeouts, retries, circuit breakers, and idempotency improve resilience. Load balancers and horizontal scaling increase capacity when state and coordination are designed appropriately. CDNs move suitable content closer to users and reduce origin load. Effective capacity planning combines realistic forecasts, SLO-based load tests, dependency limits, headroom, cost analysis, and failure planning.

Key Terms

Performance: The speed, capacity, efficiency, and reliability with which a system delivers useful work.

Latency: Elapsed time between an operation’s initiation and completion.

Throughput: Completed work per unit of time.

Percentile: A distribution value below which a specified proportion of observations falls.

SLO: A target for a measured service-level indicator.

Profiler: A tool that measures where execution time or memory is consumed.

Bottleneck: A limiting resource or operation that constrains system performance.

Cache: A storage layer containing reusable data or representations.

Cache-aside: A pattern in which application code reads and populates the cache around the source of truth.

TTL: Time-to-live; the maximum intended lifetime of a cache entry.

Cache stampede: A surge of regeneration work caused by simultaneous cache misses.

Pagination: Dividing a large result set into bounded portions.

Keyset pagination: Pagination based on an ordered continuation key rather than a large offset.

N+1 query problem: Excessive repeated queries caused by fetching related data individually.

Index: A database structure that accelerates selected access patterns at storage and write cost.

Rate limiting: Restricting request consumption over time or by identity.

Load balancer: A component that distributes traffic among service instances.

Horizontal scaling: Increasing capacity by adding service instances.

CDN: A geographically distributed network that serves cacheable content near users.

Capacity planning: Estimating resources needed to meet demand and service objectives.

Review Questions

Short Answer Questions

1. Why are p95 and p99 latency often more useful than average latency?
2. State Little’s Law and define its three variables.
3. What is the cache-aside pattern?
4. Distinguish offset pagination from keyset pagination.
5. Why can an excessively large database connection pool reduce reliability?

Long Answer Questions

1. Explain a measurement-first workflow for diagnosing a slow Django or Node.js endpoint.
2. Compare cache-aside, read-through, write-through, and write-back caching, including their risks.
3. Describe how indexes, execution plans, eager loading, and transaction scope affect database performance.
4. Explain how rate limiting, timeouts, retries, circuit breakers, and idempotency work together.
5. Discuss the architectural requirements and trade-offs of horizontal scaling behind a load balancer.

Analytical / Application Questions

1. An endpoint completes 300 requests per second with an average latency of 0.4 seconds. Estimate average in-flight requests using Little’s Law, then explain why this does not reveal p99 latency.
2. A service instance sustains 180 requests per second under its SLO. Forecast peak demand is 700 requests per second and the organization requires 25% planning headroom. Calculate the minimum capacity-based instance count and identify two reasons the deployed count might need to be higher.
3. A personalized API response is accidentally served from a shared CDN cache to different users. Analyze the likely cache-key or response-policy error and propose controls to prevent recurrence.

Multiple Choice Questions with Answers

Question 1

Which metric best describes the latency experienced by the slowest approximately 1% of requests?

A. p50 latency

B. p90 latency

C. p99 latency

D. Average throughput

Correct Answer: C. p99 latency

Question 2

An API processes 100 requests per second with an average response time of 0.2 seconds. According to Little’s Law, what is the average number of in-flight requests?

A. 5

B. 20

C. 50

D. 500

Correct Answer: B. 20

Question 3

Which technique most directly reduces a cache stampede when one popular entry expires?

A. Removing all cache expiration

B. Single-flight refresh or request coalescing

C. Increasing database page size only

D. Disabling observability

Correct Answer: B. Single-flight refresh or request coalescing

Question 4

Which pagination method is generally more efficient for traversing very large, continuously changing feeds?

A. Keyset pagination with a stable ordering

B. Unbounded response pagination

C. Large offset pagination without an index

D. Random page selection

Correct Answer: A. Keyset pagination with a stable ordering

Question 5

What is the primary purpose of an execution plan?

A. To encrypt database backups

B. To show how the database intends to execute a query

C. To generate API documentation

D. To compress query results

Correct Answer: B. To show how the database intends to execute a query

Question 6

Which Node.js behavior can block unrelated requests handled by the same process?

A. An asynchronous network call using a promise

B. A synchronous CPU-intensive operation on the event loop

C. A cache hit in an external Redis service

D. A response served by a CDN

Correct Answer: B. A synchronous CPU-intensive operation on the event loop

Question 7

What is a key requirement for safely caching personalized responses in a shared cache?

A. Ignore authorization when constructing cache keys

B. Use a key and policy that distinguish the relevant user or access context

C. Set every response to public caching

D. Disable response validation

Correct Answer: B. Use a key and policy that distinguish the relevant user or access context

Question 8

A tested instance supports 200 requests per second under its SLO. Planned demand is 1,000 requests per second. Ignoring failure reserve, what is the minimum capacity-based instance count?

A. 2

B. 4

C. 5

D. 10

Correct Answer: C. 5

Question 9

Which asset is most suitable for long-lived CDN caching when its URL contains a content hash?

A. A user’s account balance

B. A permission-sensitive dashboard response

C. An immutable versioned JavaScript bundle

D. A rapidly changing inventory count

Correct Answer: C. An immutable versioned JavaScript bundle

Question 10

Why can a fixed-window rate limiter permit a burst near a window boundary?

A. Requests are counted in two adjacent windows

B. It automatically increases database capacity

C. It disables authentication at the boundary

D. It converts all requests into cache hits

Correct Answer: A. Requests are counted in two adjacent windows