Distributed Application Engineering and DevOps

Introduction and Context

Distributed applications are software systems whose components execute on multiple networked computers and cooperate through communication protocols. Their engineering differs fundamentally from developing a single-process application. Networks introduce latency, partial failure, message loss, duplication, reordering, inconsistent views of data, and operational dependence on infrastructure. Consequently, a distributed application is not complete when its code compiles; it must also be deployable, observable, recoverable, secure, testable under failure, and changeable without unacceptable service interruption.

Distributed application engineering combines software architecture, domain modelling, API and event design, reliability engineering, infrastructure automation, and DevOps practices. DevOps is not merely a collection of tools. It is an organisational and technical approach that integrates development and operations, shortens feedback cycles, automates repeatable work, and makes system behaviour visible. In postgraduate practice, the central question is not “Which tool should be used?” but “Which engineering controls reduce risk while preserving delivery speed and system fitness?”

This chapter presents an end-to-end approach: decompose a business domain, define service and event boundaries, manage configuration and infrastructure as code, establish CI/CD pipelines, use progressive delivery, and test distributed behaviour through simulation, fault injection, chaos engineering, and reproducible environments.

Learning Objectives

By the end of this chapter, students should be able to:

1. Apply domain-driven design to identify bounded contexts and service boundaries.

2. Design robust synchronous APIs and asynchronous events.

3. Distinguish application configuration, secrets, and infrastructure state.

4. Explain infrastructure as code and evaluate its risks and controls.

5. Design CI/CD pipelines with quality, security, and deployment gates.

6. Compare blue-green, canary, rolling, and feature-flag-based delivery.

7. Construct testing strategies for distributed systems, including simulation and fault injection.

8. Explain chaos engineering and the role of reproducible environments.

9. Evaluate governance, security, ethical, and operational implications of automation.

1. Domain-Driven Decomposition

1.1 From technical layers to business capabilities

A common early design mistake is to divide a system into services such as “database service,” “validation service,” and “notification service” solely according to technical functions. Such decomposition often produces excessive coupling: a business transaction must coordinate many services, and changes in one data model propagate widely.

Domain-driven design (DDD) begins with the business domain and its language. Domain experts and engineers identify business capabilities, rules, entities, value objects, aggregates, domain events, and bounded contexts. A bounded context is a boundary within which a domain model and its terminology have a consistent meaning. For example, the word “customer” may mean a person with a billing relationship in one context and a recipient of marketing communications in another.

A business capability describes what an organisation does, such as order fulfilment, payment processing, inventory management, or identity management. A service boundary is a candidate implementation boundary around a capability or bounded context. It is not automatically a separate deployment. A modular monolith may provide strong boundaries without network overhead, while a distributed service may be justified where independent scaling, isolation, team autonomy, or deployment independence is valuable.

1.2 Strategic and tactical DDD

Strategic DDD identifies bounded contexts and relationships among them. A context map may describe relationships such as:

• Customer–supplier: one context provides data or functionality on which another depends.
• Conformist: a downstream context adopts the upstream model rather than translating it.
• Anti-corruption layer: translation protects one model from an unsuitable external model.
• Published language: a formally defined shared contract, often an API or event schema.

Tactical DDD supplies modelling constructs. An entity has identity that persists through state changes. A value object is defined by its attributes and is normally immutable. An aggregate is a consistency boundary containing an aggregate root through which modifications are controlled. Transactions should generally enforce invariants within one aggregate; cross-aggregate workflows are often coordinated using events or sagas rather than a distributed transaction.

A useful design test is: “Which invariants must be true immediately, and which can become true eventually?” If an order must not be confirmed without an authorised payment, the relevant consistency boundary and workflow must explicitly address that rule. If a recommendation index may lag behind an order, eventual consistency may be acceptable.

1.3 Decomposition criteria and trade-offs

Service boundaries should be evaluated using several criteria:

• Cohesion: related rules and data change together.
• Coupling: changes and runtime interactions across boundaries are limited.
• Data ownership: one context is authoritative for a business fact.
• Consistency requirements: strongly coupled invariants remain close together.
• Team and deployment autonomy: teams can release independently.
• Operational cost: each service adds monitoring, security, deployment, and failure modes.
• Scalability and isolation: independently variable workloads may justify separation.

The “microservices equals maturity” assumption is incorrect. A distributed architecture can increase latency, operational burden, and debugging difficulty. A well-modularised monolith is often a better starting point when domain boundaries are uncertain or the organisation lacks operational capability.

2. API and Event Design

2.1 Synchronous API design

An API is a contract between a provider and its consumers. A sound contract specifies resource or operation semantics, data types, validation, error meanings, authentication, authorisation, compatibility policy, rate limits, and observability requirements.

For HTTP APIs, resource-oriented design commonly uses methods such as GET, POST, PUT, PATCH, and DELETE. The semantics must be explicit. A GET should not produce business side effects. A retryable operation should be designed with idempotency in mind. Idempotency means that repeating the same request has the same intended effect as processing it once, even if the server receives duplicates. An idempotency key can allow a server to associate retries with a previously completed command.

Error responses should distinguish client errors, authentication and authorisation failures, missing resources, conflicts, validation failures, and transient server failures. Consumers should not be required to parse human-readable error strings to make control decisions. Contracts should be machine-readable where feasible, using approaches such as OpenAPI, schema validation, and consumer-driven contract tests.

Backward compatibility is a social and technical property. Adding an optional response field is usually safer than renaming or removing a field. A provider should not change the meaning of an existing field without a migration plan. Versioning may be implemented through URLs, headers, media types, or compatible evolution; no method is universally superior. Version proliferation is itself a maintenance cost.

2.2 Events and asynchronous communication

An event records that something happened, for example, `OrderPlaced` or `PaymentCaptured`. An event is different from a command: a command requests an action, whereas an event communicates a fact. Events may be published to a broker and consumed independently by multiple services.

Important event-design properties include:

• a stable event name and explicit semantic meaning;
• a unique event identifier;
• the aggregate or business entity identifier;
• occurrence time and, where needed, sequence information;
• a schema version or compatibility policy;
• correlation and causation identifiers for tracing workflows;
• sufficient data for consumers without exposing unnecessary sensitive data.

At-least-once delivery is common because it is usually safer to retry than to risk losing an event. It implies that consumers must tolerate duplicate delivery. Consumers can use idempotent updates, deduplication records, or naturally repeatable operations. Exactly-once claims must be treated carefully: a broker may provide exactly-once processing within a restricted boundary, but external side effects such as email or payment remain difficult to make globally exactly once.

Ordering is not free. A system may guarantee order only per partition, aggregate, or key. Consumers should avoid assuming global order unless the infrastructure and design explicitly provide it. Eventual consistency means that different projections may temporarily disagree while converging after successful processing. User interfaces and business processes should communicate this reality rather than silently presenting stale information as authoritative.

2.3 Reliable publication and workflow coordination

The dual-write problem occurs when a service updates its database and publishes an event as two separate actions. A crash between actions can create a committed business change with no event, or an event without the corresponding change. The transactional outbox pattern addresses this by writing the business change and an outbox record in the same local database transaction. A publisher later reads the outbox and sends the event, marking it as dispatched. Consumers still need idempotency because publication or acknowledgement may be retried.

A saga coordinates a long-running business transaction through a sequence of local transactions. In a choreography-based saga, services react to one another’s events. In an orchestration-based saga, a coordinator directs the steps. Compensation is a business action that reverses or offsets a completed step; it is not necessarily a technical rollback. For example, a refund may compensate a payment, but it cannot erase the historical fact that a payment occurred.

3. Configuration, Secrets, and Environment Management

Configuration includes values that vary between environments or deployments, such as database endpoints, feature settings, timeouts, and resource limits. Configuration should be externalised from application binaries so that the same artifact can be promoted through environments. However, externalisation does not mean that every value should be editable without control.

Secrets—passwords, private keys, tokens, and certificates—require stricter handling. They should be stored in a dedicated secret-management system, encrypted in transit and at rest, access-controlled by identity, rotated, audited, and excluded from source repositories and ordinary logs. Configuration values should be classified according to confidentiality, integrity, availability, and change impact.

A practical model has three layers:

1. Code defaults: safe, non-sensitive defaults that support local development.

2. Environment configuration: deployment-specific values supplied through controlled mechanisms.

3. Runtime overrides or feature controls: narrowly governed settings used for operations and experimentation.

Configuration schemas should be validated at startup or deployment time. Invalid configuration should fail clearly rather than causing obscure runtime behaviour. Configuration changes should be versioned, reviewed, and observable. Feature flags require ownership, expiry dates, access control, and cleanup; otherwise they become permanent hidden branches.

4. Infrastructure as Code

Infrastructure as code (IaC) represents infrastructure resources and their relationships in declarative or programmatic definitions. Compute instances, networks, databases, policies, queues, and observability components can therefore be reviewed, tested, versioned, and reproduced.

Declarative IaC states the desired end condition, while an engine determines the operations required to reach it. Imperative scripts specify procedural steps. Declarative approaches often improve repeatability, but they do not eliminate provider differences, state drift, failed partial operations, or unsafe changes.

A controlled IaC workflow normally includes:

1. source-controlled definitions and reusable modules;

2. formatting, static analysis, policy checks, and security scanning;

3. a preview or plan reviewed before application;

4. protected state storage with locking and access control;

5. staged application in isolated environments;

6. drift detection and documented exception handling;

7. backups and recovery procedures for state and data.

Infrastructure state is sensitive operational information. It may contain identifiers, network details, or references to secrets. State must therefore be protected like production metadata. IaC should not be confused with data migration: destroying and recreating infrastructure may be acceptable for an ephemeral test environment but dangerous for a production database.

5. Continuous Integration and Continuous Delivery

Continuous integration (CI) means that changes are integrated frequently and automatically verified. Continuous delivery means that software is maintained in a releasable state and can be deployed on demand. Continuous deployment goes further by automatically deploying changes that pass defined controls.

A robust pipeline is a risk-control system rather than a linear collection of commands. A representative sequence is:

• source checkout and dependency verification;
• formatting and static analysis;
• unit tests;
• build and artifact creation;
• software composition and vulnerability scanning;
• API, integration, and contract tests;
• container or package signing;
• deployment to an ephemeral or integration environment;
• smoke and end-to-end tests;
• approval or automated policy evaluation;
• progressive production delivery;
• post-deployment verification and rollback or roll-forward.

Builds should be reproducible: the same source, dependency lock information, build instructions, and controlled inputs should produce equivalent artifacts. Pinning dependencies reduces accidental change, but a process is still needed to update vulnerable dependencies. Supply-chain security includes protected source control, least-privilege build identities, isolated runners, provenance records, signed artifacts, and verification before deployment.

Test suites must be balanced. Unit tests are fast and isolate logic. Integration tests verify real component interactions. Contract tests detect incompatible API or event changes. End-to-end tests validate important user journeys but are slower and more fragile. A test pyramid is a useful heuristic, not an inviolable law: the appropriate distribution depends on architecture and risk.

Pipeline metrics should be interpreted together. Delivery frequency, lead time for changes, change failure rate, and time to restore service are useful indicators, but optimising one metric in isolation can encourage harmful behaviour. For example, increasing deployment frequency by reducing testing may increase failure and recovery costs.

6. Progressive Delivery

Progressive delivery reduces blast radius by exposing a change gradually and evaluating evidence before wider release. It requires automation, observability, a clear success hypothesis, and a rollback or mitigation procedure.

Blue-green deployment maintains two production-like environments. Traffic is switched from the blue version to the green version after validation. It enables rapid reversal when both environments are compatible, but it can be expensive and does not by itself solve database migration incompatibility.

Rolling deployment replaces instances incrementally. It uses resources efficiently but can temporarily run multiple versions, requiring backward-compatible APIs, schemas, and messages.

Canary deployment sends a small proportion of traffic to the new version, often selected by user, region, or request percentage. Metrics such as error rate, latency, saturation, and business conversion are compared with a baseline. Statistical significance is not automatically achieved by a small traffic percentage; decision thresholds and observation duration should be defined before release.

Feature flags decouple deployment from feature activation. They support experimentation, tenant-specific release, and rapid disablement, but create configuration complexity and can expose inconsistent behaviour if related flags are not coordinated.

Database evolution should follow an expand-and-contract pattern where possible: add a compatible schema element, deploy code that can use both old and new forms, migrate data, switch reads and writes, and remove obsolete elements only after all consumers have migrated. This is safer than coupling an irreversible schema change to one application release.

7. Testing Distributed Systems

Distributed testing must examine not only correct messages and outputs but also timing, retries, duplication, partition, resource exhaustion, and recovery. A system that works in a failure-free test environment may fail in production because its assumptions about timing and independence are unrealistic.

7.1 Simulation and virtual time

Simulation models components, network behaviour, workload, and failures in a controlled environment. It can explore rare or expensive conditions, such as large-scale retries or long outages. A simulator must state its abstractions: simplified network latency, failure probabilities, scheduling, and workload distributions. Simulation results are conditional evidence, not proof that a real system will behave identically. Validation against production or laboratory measurements is essential.

Virtual time and deterministic scheduling can make tests repeatable. A test may control clock advancement, message delivery order, and random seeds. This is particularly useful for timeout logic, leases, retry policies, and event-time processing. Determinism must not hide concurrency bugs; complementary tests with realistic scheduling are necessary.

7.2 Fault injection

Fault injection deliberately introduces failures such as dropped messages, delayed responses, duplicate events, unavailable dependencies, expired credentials, corrupted payloads, disk exhaustion, or process termination. It should begin in isolated environments with explicit safety limits and gradually expand in scope.

A failure test should define:

• the hypothesis, such as “the checkout service remains available when the recommendation service is unavailable”;
• the fault and its duration;
• the expected control, such as timeout, fallback, retry, or circuit breaker;
• measurable success criteria;
• abort conditions;
• evidence collection and post-test analysis.

Retry design requires particular care. Unbounded or synchronised retries can create a retry storm. Exponential backoff with jitter reduces synchronisation, but retrying non-idempotent operations without protection can duplicate business actions. Timeouts should reflect a bounded latency budget and should be propagated across call chains where possible.

7.3 Chaos engineering

Chaos engineering is the disciplined experimentation on a system to learn about its behaviour under realistic adverse conditions. It is not random destruction. It involves a steady-state hypothesis, controlled introduction of a failure, observation of system and business indicators, and learning that results in improved controls.

Experiments require authorisation, communication, blast-radius limits, monitoring, emergency termination, and protection of users and data. A chaos programme should address organisational readiness as well as technical resilience. If teams cannot explain ownership, recovery procedures, or customer impact, production experiments are premature.

7.4 Reproducible environments

A reproducible environment can be recreated from versioned definitions, pinned dependencies, controlled images, test data, and documented external assumptions. Containers improve packaging consistency but do not reproduce kernel behaviour, network conditions, cloud-provider semantics, or operational policy by themselves.

Environment parity should be risk-based. Development need not duplicate production scale, but it should preserve important interfaces, authentication flows, schema versions, and failure semantics. Ephemeral environments created per change can reduce interference between teams. They also require automated cleanup, cost controls, and appropriate handling of test data.

8. Observability and Operational Feedback

Observability concerns the ability to infer internal system state from outputs. Logs provide event records, metrics provide numerical time series, and traces show causal paths across services. High-quality telemetry includes correlation identifiers, deployment version, tenant or request classification where appropriate, and consistent severity and timing information.

Observability must protect privacy. Sensitive values should be minimised, masked, or tokenised. Retention should reflect operational and legal requirements. Sampling can reduce cost, but important failures and security events should not be discarded indiscriminately.

Service-level indicators (SLIs) measure user-relevant behaviour, such as successful request proportion or latency. Service-level objectives (SLOs) state target performance over a defined period. An error budget is the permitted unreliability implied by the SLO; it supports a governance conversation between delivery speed and reliability. These concepts are useful only when measurements represent user experience rather than merely internal activity.

9. Illustrative Case: Progressive Delivery for an Online Pharmacy

Background

This illustrative case concerns MedDirect, an online pharmacy with bounded contexts for identity, catalogue, prescription validation, payment, fulfilment, and notification. The organisation originally used a tightly coupled application and deployed monthly. A new prescription-validation algorithm was expected to reduce manual review, but a defect could delay medicine delivery or incorrectly reject prescriptions.

Problem

The organisation needed independent deployment and rapid feedback without compromising patient safety. The algorithm depended on catalogue and prescription data, while its output affected fulfilment. A simple global release would create excessive blast radius. In addition, the old system wrote fulfilment state and notification requests separately, causing occasional missing notifications.

Analysis

Domain workshops identified prescription validation as a bounded context with its own rules and a clear interface to fulfilment. The authoritative prescription status remained in the validation context. Fulfilment consumed a versioned `PrescriptionValidated` or `PrescriptionRejected` event. Immediate safety invariants—such as requiring a valid prescription before dispatch—were enforced in the fulfilment workflow, not assumed from eventual arrival of a recommendation event.

The team identified risks: incompatible event schemas, duplicate messages, stale catalogue data, algorithm bias against unusual but valid prescriptions, and operational overload during rollout. It defined an SLO for validation response time and a separate business safety indicator for manual-review escalation.

Approach and solution

The team implemented an outbox for validation events and idempotent consumers keyed by event identifier and prescription identifier. Contract tests ran against a versioned event schema. The pipeline built a signed artifact, scanned dependencies, deployed to an ephemeral environment, and executed integration tests using synthetic prescriptions.

Delivery used a canary release. Initially, only internally selected test cases and a small controlled traffic group used the new algorithm. The system compared latency, rejection distribution, manual-review rate, and downstream fulfilment failures with the baseline. A feature flag allowed immediate deactivation without redeployment. Database changes followed expand-and-contract compatibility rules.

Fault-injection tests delayed the catalogue dependency, duplicated validation events, and terminated validation workers. Expected behaviour was bounded retry, deduplication, preservation of the prescription record, and safe routing to manual review. Chaos experiments were conducted first in a staging environment and later during a low-risk production window with an abort mechanism.

Outcome

The organisation discovered that the algorithm’s average latency was acceptable but its tail latency increased when catalogue calls slowed. The team introduced caching with an explicit freshness policy and changed the workflow to prefer safe manual review over indefinite waiting. Duplicate events no longer created duplicate fulfilment actions. The release proceeded gradually, with rollback available through the feature flag.

Learning points

The case demonstrates that domain boundaries, event reliability, progressive delivery, and failure testing must be designed together. A canary is ineffective without meaningful indicators. An event-driven architecture is not automatically reliable; publication, idempotency, schema evolution, and business compensation remain necessary. Finally, safety-critical outcomes require domain-specific controls rather than reliance on technical availability metrics alone.

10. Governance, Ethics, and Professional Practice

Automation can amplify both good and bad decisions. Infrastructure pipelines with excessive privileges can alter production or expose data. Automated deployment can make an unsafe change reach users rapidly. Governance should therefore include separation of duties where justified, auditable approvals, policy as code, least privilege, change traceability, and tested recovery procedures.

Distributed systems frequently process personal, financial, health, or behavioural data. Engineers should apply data minimisation, purpose limitation, retention controls, access logging, encryption, and appropriate anonymisation or pseudonymisation. Regulatory obligations vary by jurisdiction and sector; legal and compliance specialists should be involved rather than treating a technical control as proof of compliance.

AI-assisted coding and operations can accelerate development but may introduce insecure dependencies, licence concerns, fabricated APIs, data leakage, or unreviewed logic. Generated code requires the same review, testing, provenance, and security analysis as human-written code. AI-based operational decisions should have human oversight where errors could materially harm users.

11. Contemporary Developments and Best Practices

Current practice increasingly combines platform engineering, internal developer platforms, policy-as-code, software supply-chain attestations, ephemeral environments, and automated reliability analysis. Service meshes can provide traffic management and telemetry, but they add operational complexity and should be adopted only for clear requirements. WebAssembly, serverless execution, and managed event services may reduce infrastructure administration while increasing dependence on provider semantics and cost models.

Effective practice is characterised by principles rather than products:

• model the domain before selecting service boundaries;
• prefer explicit contracts and owned data;
• design for retries, duplication, timeout, and partial failure;
• build one immutable, traceable artifact and promote it;
• keep infrastructure and policies versioned and reviewable;
• make deployments reversible or safely forward-compatible;
• measure user and business outcomes, not only technical activity;
• test recovery, not merely normal operation;
• limit blast radius and automate safe termination;
• remove obsolete flags, versions, modules, and infrastructure;
• treat security, privacy, accessibility, and compliance as design concerns.

Chapter Summary

Distributed application engineering joins domain modelling with operational discipline. Domain-driven decomposition uses bounded contexts, aggregates, ownership, and business capabilities to establish meaningful boundaries; it does not require every module to become a networked microservice. APIs and events are contracts whose compatibility, idempotency, ordering, versioning, and failure semantics must be explicit. The transactional outbox, idempotent consumers, and saga patterns address common problems in asynchronous workflows.

Configuration should be externalised, validated, versioned, and separated from secrets. Infrastructure as code improves repeatability and reviewability but requires protected state, drift management, policy checks, and careful handling of destructive operations. CI/CD pipelines provide automated feedback and controls across build, test, security, deployment, and verification stages. Progressive delivery limits blast radius through blue-green, rolling, canary, and feature-flag techniques.

Distributed systems require testing beyond functional correctness. Simulation, deterministic scheduling, fault injection, chaos engineering, and reproducible environments reveal timing and failure behaviour. Observability supplies the evidence needed for release decisions and incident response. Finally, governance, privacy, supply-chain security, ethical review, and human accountability are essential because automation can scale both reliability and harm.

Key Terms

Bounded context: A domain boundary within which terms and models have consistent meaning.

Aggregate: A consistency boundary containing related domain objects and an aggregate root.

Idempotency: The property that repeating an operation produces the same intended effect as performing it once.

Domain event: A record that a meaningful business fact has occurred.

Command: A request asking a component to perform an action.

Transactional outbox: A pattern that stores a business change and its outgoing event in one local transaction.

Saga: A sequence of local transactions coordinated to complete a long-running business process.

Configuration: Environment- or deployment-specific behaviour and values external to application code.

Secret: Confidential information such as a password, private key, or access token.

Infrastructure as code: Versioned definitions used to provision and manage infrastructure.

Drift: A difference between declared infrastructure and the actual deployed state.

Continuous integration: Frequent integration of changes with automated verification.

Continuous delivery: Maintaining software in a releasable state for deployment on demand.

Progressive delivery: Gradually exposing a change while evaluating evidence.

Canary release: A release initially exposed to a small, controlled portion of traffic.

Fault injection: Deliberate introduction of failures to evaluate system behaviour.

Chaos engineering: Controlled experimentation to learn and improve resilience under adverse conditions.

Reproducible environment: An environment that can be recreated from controlled, versioned inputs.

Service-level objective: A target for a user-relevant service indicator over a specified period.

Error budget: The unreliability permitted by a service-level objective.

Review Questions

Short Answer Questions

1. What is a bounded context, and why can the meaning of a term differ between contexts?

2. Distinguish a command from an event.

3. Why do at-least-once delivery systems require idempotent consumers?

4. State two differences between continuous delivery and continuous deployment.

5. What is the purpose of a canary release?

Long Answer Questions

1. Explain how domain-driven design can guide service decomposition. Discuss cohesion, coupling, data ownership, consistency, and operational cost.

2. Describe the transactional outbox pattern, including the failure it addresses and the responsibilities that remain with consumers.

3. Compare blue-green, rolling, and canary deployment strategies, including their assumptions and limitations.

4. Develop a testing strategy for a distributed payment workflow using contract testing, integration testing, fault injection, and chaos engineering.

5. Explain how IaC, CI/CD, observability, and governance work together to create a controlled software delivery system.

Analytical / Application Questions

1. An order service commits an order to its database but crashes before publishing `OrderPlaced`. Analyse the consequences and propose a reliable design.

2. A canary version shows lower average latency but a higher 99th-percentile latency and increased manual-review rates. Decide whether to expand the release and justify the indicators and additional evidence required.

3. Design a failure experiment for a notification service that becomes unavailable after payment succeeds. State the hypothesis, injected fault, expected system behaviour, safety controls, and success criteria.

Multiple Choice Questions with Answers

Question 1

Which concept defines a boundary within which a domain model and terminology have consistent meaning?

A. Bounded context

B. Deployment unit

C. Retry policy

D. Infrastructure state

Correct Answer: A. Bounded context

Question 2

What problem does the transactional outbox pattern primarily address?

A. Slow database queries

B. The dual-write inconsistency between a database change and event publication

C. Global network bandwidth limits

D. User-interface versioning

Correct Answer: B. The dual-write inconsistency between a database change and event publication

Question 3

Which statement best distinguishes an event from a command?

A. An event requests an action, while a command reports history

B. An event reports that something happened, while a command requests an action

C. Events are always synchronous, while commands are always asynchronous

D. Events cannot contain identifiers

Correct Answer: B. An event reports that something happened, while a command requests an action

Question 4

Why should consumers in an at-least-once messaging system be idempotent?

A. Messages are guaranteed to arrive in global order

B. The same message may be delivered more than once

C. Consumers never need to acknowledge messages

D. It eliminates all network latency

Correct Answer: B. The same message may be delivered more than once

Question 5

Which deployment strategy initially sends a small, controlled proportion of traffic to a new version?

A. Blue-green deployment

B. Canary deployment

C. Big-bang deployment

D. Schema rollback

Correct Answer: B. Canary deployment

Question 6

What is a primary security requirement for application secrets?

A. Commit them to source control for traceability

B. Store them in a dedicated, access-controlled secret-management system

C. Print them in logs during startup

D. Embed them permanently in container images

Correct Answer: B. Store them in a dedicated, access-controlled secret-management system

Question 7

Which practice most directly improves infrastructure reproducibility?

A. Manual changes made only by senior engineers

B. Version-controlled infrastructure definitions with controlled inputs

C. Unrecorded emergency edits

D. Using different build steps in every environment

Correct Answer: B. Version-controlled infrastructure definitions with controlled inputs

Question 8

What is the defining feature of a chaos-engineering experiment?

A. Randomly deleting production data

B. A controlled test based on a hypothesis about system behaviour under failure

C. Replacing all automated tests with manual testing

D. Measuring only source-code size

Correct Answer: B. A controlled test based on a hypothesis about system behaviour under failure

Question 9

Which pattern is most appropriate for a schema change when old and new application versions may run simultaneously?

A. Expand and contract

B. Immediate destructive replacement

C. Unversioned mutation

D. Disabling database backups

Correct Answer: A. Expand and contract

Question 10

What does an error budget represent?

A. The total infrastructure purchase price

B. The permitted unreliability implied by a service-level objective

C. The number of defects allowed in source code

D. The maximum size of a log file

Correct Answer: B. The permitted unreliability implied by a service-level objective