Architecture Evaluation and Capstone Design

Introduction and Context

Cloud architecture is not complete when components have been selected or when infrastructure has been deployed. A professional architecture must be evaluated against business outcomes, risk tolerance, technical constraints and operational evidence. Architecture evaluation is therefore a structured process of testing whether a proposed design is fit for purpose. Capstone design extends this process by requiring the architect to integrate requirements, architectural principles, prototypes, decision records, quantitative analysis and stakeholder communication into one defensible proposal.

A cloud solution can be technically elegant yet unsuitable because it is too expensive, legally non-compliant, difficult to operate, or unable to meet peak demand. Conversely, a modest design may be preferable if it provides sufficient reliability at substantially lower complexity and cost. The central professional skill is not maximising every quality attribute independently, but making explicit and defensible trade-offs.

This chapter presents a practical method for producing and defending an end-to-end cloud architecture. It uses an illustrative case involving a digital health platform, but the method applies to financial services, education, retail, government and industrial systems.

Learning Objectives

By the end of this chapter, the learner should be able to:

1. Translate business goals into measurable architecture requirements and quality attributes.
2. Compare architecture alternatives using transparent evaluation criteria.
3. Use prototypes, tests and architecture decision records as evidence.
4. Analyse security, reliability, performance, cost, compliance and sustainability trade-offs.
5. Calculate basic availability, capacity, latency and total-cost measures correctly.
6. Produce an end-to-end cloud architecture and defend it to technical and managerial stakeholders.
7. Identify limitations, residual risks and conditions requiring architectural revision.

1. What Architecture Evaluation Means

Architecture evaluation is the systematic examination of a proposed system structure, its behaviour under relevant conditions and its consequences for stakeholders. It differs from implementation testing. Testing asks whether a particular implementation behaves as expected; architecture evaluation asks whether the chosen structure is likely to satisfy important requirements, including requirements that may not yet be observable in code.

A useful evaluation has five properties:

• Traceability: every major design decision can be linked to a requirement, risk or constraint.
• Evidence: claims are supported by tests, models, vendor documentation, prototypes or credible assumptions.
• Comparability: alternatives are evaluated using common criteria.
• Explicit trade-offs: benefits and disadvantages are recorded rather than hidden.
• Reversibility awareness: decisions that are expensive to undo receive greater scrutiny.

Stakeholders usually judge architecture differently. A chief financial officer may focus on cost predictability, a security officer on threat reduction, an operations team on recoverability, and a product owner on speed of change. The architect must make these perspectives comparable without pretending that all values can be reduced to one number.

2. From Business Goals to Evaluation Criteria

The first task is requirements engineering. Begin with business capabilities and outcomes rather than cloud products. For example, “patients can book appointments” is a capability; “95% of bookings are completed in under three seconds during peak demand” is an evaluable requirement.

Requirements should be classified as follows:

A quality attribute scenario makes a vague requirement testable. It specifies a stimulus, environment, affected component, response and response measure. For example: “During a seasonal demand spike in the production environment, the appointment API shall maintain a 95th-percentile latency below two seconds while demand is below 800 requests per second.”

2.1 Constraints, assumptions and risks

Constraints are conditions the design cannot freely change, such as data residency law, an existing identity provider, a fixed launch date or a legacy database. Assumptions are propositions used in planning but not yet verified, such as expected peak traffic or user growth. Each important assumption should have a validation method and an owner.

A risk is commonly described by probability and impact. A simple qualitative matrix is appropriate during early design, while quantitative analysis becomes useful when historical data exists. Risk treatment may involve avoiding, reducing, transferring, accepting or monitoring the risk. Architecture evaluation should record residual risk after controls have been applied.

3. A Structured Evaluation Framework

A practical evaluation sequence is:

1. Establish business outcomes, constraints and quality-attribute scenarios.
2. Define the system boundary and trust boundaries.
3. Create at least two plausible architecture alternatives.
4. Identify risks and architectural drivers.
5. Build a thin vertical prototype for uncertain or high-risk aspects.
6. Evaluate alternatives using tests, models and expert review.
7. Record decisions and rejected alternatives.
8. Present the recommendation with assumptions, costs, risks and next steps.
9. Re-evaluate after significant changes or operational evidence.

3.1 Weighted decision analysis

A weighted decision matrix can support comparison, but it is not a substitute for judgement. Let s_i be the score of alternative i for criterion j, and let w_j be the criterion weight. If scores are normalised to a common scale, the weighted score is:

The score is meaningful only when criteria, scales and weights are defined consistently. It should not combine incompatible measures without normalisation. Sensitivity analysis is essential: if a small change in weights changes the preferred alternative, the decision is sensitive and should be discussed openly.

The small difference between containers and serverless services indicates that the decision should not be presented as mathematically definitive. Operational skill, workload variability and portability requirements may justify selecting containers despite the slightly lower score.

4. End-to-End Architecture Representation

An architecture description should show more than a collection of services. It should communicate users, external systems, data flows, trust boundaries, deployment units, operational controls and failure domains. A useful set of views includes:

• Context view: users, business systems, regulators and external providers.
• Container or service view: major applications, data stores, queues and integration points.
• Deployment view: regions, zones, networks, clusters and managed services.
• Security view: identities, trust boundaries, secrets, encryption and monitoring.
• Operational view: deployment, observability, backup, recovery and incident response.
• Data-lifecycle view: collection, processing, retention, archival and deletion.

The following simplified view illustrates a defensible cloud pattern.

Figure: A layered cloud architecture showing request flow, data protection and cross-cutting operational controls.

The diagram does not imply that every request must pass through every component. For example, asynchronous notifications may move through a queue, while read-heavy public content may be served by a CDN. Such distinctions belong in accompanying interface and sequence descriptions.

5. Evidence: Prototypes, Tests and Decision Records

5.1 Thin vertical prototypes

A prototype should answer a specific uncertainty, not merely demonstrate that a service can be created. High-value prototypes include:

• a load test to determine throughput and latency;
• a failure injection exercise to examine zone or dependency loss;
• an identity prototype to validate federation and authorisation;
• a restore test to verify backup usability;
• a data-residency test to confirm storage and replication locations;
• a cost experiment using representative traffic and retention.

A prototype is not production evidence unless its workload, configuration and environment are sufficiently representative. Document its scope, limitations and what decision it informs.

5.2 Architecture Decision Records

An Architecture Decision Record (ADR) should contain:

1. Title and status.
2. Context and forces, including constraints.
3. Decision and rationale.
4. Alternatives considered.
5. Consequences, including disadvantages.
6. Evidence and assumptions.
7. Review date or trigger.

Example: “Use a managed relational database for appointment transactions.” The rationale may be strong consistency, mature backup and familiar operational skills. Consequences include licensing or scaling limits and dependence on a managed service. A document that records only benefits is advocacy, not an ADR.

6. Quantitative Evaluation

6.1 Availability and reliability

If independent components are arranged sequentially, service availability is approximately the product of component availabilities:

Here A_k is the availability of component k, expressed as a fraction, and m is the number of sequential components. The approximation assumes statistically independent failures and a clearly defined service path. Correlated failures, shared networks and common control planes can make the estimate optimistic.

For redundant components operating in parallel, if failures are independent, the availability that at least one of n identical components remains available is:

For example, two components each with availability 0.99 provide 1 − (1 − 0.99)² = 0.9999, or 99.99%, under the stated independence assumption. Real systems may achieve less because both replicas can share a zone, credential, deployment pipeline or software defect.

Availability can also be expressed through mean time between failures and mean time to repair:

MTBF is mean time between failures and MTTR is mean time to repair; both must use the same time unit. This model is a long-run approximation and does not represent all planned maintenance, cascading failure or changing workloads.

6.2 Recovery objectives

The Recovery Point Objective (RPO) is the maximum acceptable amount of data loss measured in time. The Recovery Time Objective (RTO) is the target time to restore an agreed service level. A design with an RPO of 15 minutes may require frequent replication or transaction logging; a design with an RTO of 10 minutes requires tested automation, available capacity and clear operational authority. Backups alone do not prove that either objective can be met.

6.3 Capacity and performance

A first-order estimate of average request rate is:

where N is the number of requests in interval T, and λ is requests per unit time. If 36,000 requests occur in 10 minutes, the average rate is 60 requests per second. Peak rate, burst duration, request cost and concurrency must be measured separately.

Little’s Law relates average number of items in a stable system to throughput and average time in the system:

L is average concurrency, λ is throughput and W is average response time; units must be consistent. If a service sustains 100 requests per second with an average response time of 0.2 seconds, average in-flight requests are 20. This does not predict tail latency or prove that the system is stable; it is a relationship observed under stable operating conditions.

Performance evaluation should report percentiles, especially p95 and p99, rather than only averages. A low average can hide severe experiences for a minority of users. Tests should vary payload size, cache state, database contention, dependency latency and failure conditions.

6.4 Cost and sustainability

A transparent monthly cost model separates fixed, variable and one-time costs:

The terms represent baseline resources, usage-based services, storage and data protection, operational tooling and network transfer. The model must state currency, billing period, assumptions, discounts, taxes and excluded human costs. Unit cost can be calculated as C_unit = C_total ÷ U, where U is a defined business unit such as completed bookings.

Sustainability is evaluated through workload efficiency, resource utilisation, data retention, right-sizing, caching, scheduling of non-production environments and selection of lower-carbon regions where legally and operationally appropriate. Cost reduction and sustainability often align, but not always: additional replicas can improve resilience while increasing energy and cost. Sustainability claims should use measurable indicators rather than vague assertions.

7. The Illustrative Case: CareConnect Digital Health

Background

CareConnect is an illustrative medium-sized healthcare provider operating clinics in two cities. It wants a patient-facing platform for appointment booking, reminders, clinician schedules and reporting. Personally identifiable and health-related data must be protected, access must be auditable, and the platform must remain usable during seasonal demand spikes.

Problem and requirements

The organisation expects 1,200 requests per second at the busiest period, with a normal rate of approximately 150 requests per second. Booking transactions require strong consistency. Notifications may be delayed by several minutes without harming the booking transaction. The target is 99.95% monthly availability, an RTO of 60 minutes and an RPO of 15 minutes. Patient data must remain within approved national jurisdictions. The initial budget is constrained, and the internal team has limited experience operating Kubernetes.

Alternatives

Three alternatives were considered:

• A: Virtual machines with a self-managed database. This offers control and possible portability but creates a large patching, backup and scaling burden.
• B: Managed container platform with a managed relational database. This provides control over application packaging while reducing cluster and database administration.
• C: Serverless functions and serverless database services. This can scale rapidly and reduce idle cost, but introduces platform-specific constraints, cold-start considerations, distributed transaction complexity and weaker portability.

Analysis and decision

The evaluation selected Alternative B. The application is packaged as containers and deployed across at least two availability zones. A managed relational database provides transactional integrity, automated backups and a read replica for reporting workloads. A queue separates appointment confirmation from notifications. An API gateway and web application firewall provide edge control. Identity federation, role-based access control, encryption, central logging and security alerting are mandatory controls.

Alternative C was not rejected because it is technically incapable; it was rejected for this stage because the transaction model and team skills made its operational and migration risks less attractive. Alternative A was rejected because the organisation would assume too much responsibility for patching, failover and backup validation.

Prototype plan

Before approval, the team should build a vertical slice containing login, appointment search, booking, database transaction, queue publication and notification simulation. The prototype should use synthetic data and test:

• 1,200 requests per second with realistic read/write proportions;
• p95 booking latency under the agreed threshold;
• duplicate booking prevention under concurrent requests;
• database failover and connection recovery;
• restoration from a backup;
• loss of one availability zone;
• audit-log completeness and access restrictions.

The results must include test configuration, workload generator, observed limits and deviations from production. A successful prototype reduces uncertainty but does not eliminate operational risk.

Security and compliance design

Security begins with threat modelling. Important threats include credential theft, broken object-level authorisation, injection, malicious automation, insider misuse, exposed storage and compromised dependencies. Controls include phishing-resistant authentication where feasible, short-lived credentials, least-privilege service identities, network segmentation, input validation, encryption in transit and at rest, immutable audit logs, vulnerability scanning and tested incident response.

Compliance is not achieved by purchasing a “compliant” cloud service. The provider and customer share responsibilities. CareConnect remains responsible for lawful processing, access governance, retention, incident management, configuration and evidence. Data classification should distinguish patient records, operational metadata, logs and public content. Retention and deletion rules must be technically enforceable and legally reviewed.

Reliability, cost and sustainability trade-offs

Multi-zone deployment reduces the impact of a zone failure but increases compute, database and data-transfer cost. A cross-region standby could improve regional recovery but may create residency, replication, cost and operational complexity. The recommended first release uses multi-zone resilience and tested backups, with cross-region recovery as a roadmap item subject to a formal RTO/RPO exercise.

Autoscaling reduces idle capacity but can be slow during abrupt bursts and may overload a database if application instances scale faster than the data tier. Connection pooling, queue-based smoothing, caching of non-sensitive reference data and admission control are therefore required. Cost budgets and alerts should be combined with engineering review because alerts alone do not prevent waste.

Outcome and learning points

The proposed architecture satisfies the initial business scope while keeping operational responsibility compatible with team capability. Its principal residual risks are database scaling, dependency failure, incorrect authorisation and untested recovery. The case demonstrates that an architecture recommendation is credible only when it states what has been proven, what remains uncertain and what will trigger reconsideration.

8. Presenting and Defending the Capstone

A capstone presentation should be adapted to its audience. Technical stakeholders need interfaces, deployment topology, test evidence, failure modes and operational procedures. Managers need business outcomes, cost ranges, risks, delivery milestones and decision points. Regulators or auditors need control ownership and evidence.

A defensible presentation normally contains:

1. Business problem and success measures.
2. Scope, assumptions and constraints.
3. Context and deployment diagrams.
4. Alternatives and decision criteria.
5. Prototype results and limitations.
6. Security, resilience, performance, cost and sustainability analysis.
7. Implementation roadmap and operating model.
8. Residual risks, mitigations and approval requests.

Use ranges when precision is not justified. Distinguish facts, estimates and recommendations. Do not claim “secure,” “highly available” or “compliant” without defining the boundary and evidence. A useful defence technique is to state the condition under which the decision would change—for example, a sustained growth rate, a new residency requirement or a measured database bottleneck.

9. Challenges and Limitations

Architecture evaluation is affected by incomplete requirements, unreliable forecasts, biased scoring, non-representative prototypes and changing vendor services. Cloud pricing can be difficult to model because usage, transfer, support and discount structures interact. Availability calculations often overestimate real resilience by assuming independent failures. Security reviews may focus on infrastructure while neglecting application authorisation and human processes.

Architectural diagrams can also create false confidence. A diagram shows intended structure, not necessarily deployed configuration or operational behaviour. Infrastructure as code, policy as code, continuous monitoring and periodic recovery exercises are needed to connect design intent with reality.

AI-assisted design tools can help generate alternatives, inspect configuration and summarise logs, but they must not be trusted as autonomous architects. Sensitive data should not be submitted to an unapproved model. Generated recommendations require human review, provenance, security testing and validation against organisational policy. AI systems themselves introduce concerns about data leakage, explainability, model drift and accountability.

Best Practices Checklist

• Start with measurable outcomes and quality-attribute scenarios.
• Identify trust boundaries, failure domains and shared dependencies.
• Compare genuine alternatives, including the option to defer or simplify.
• Prototype the riskiest assumption first.
• Record decisions, rejected options and consequences in ADRs.
• Use percentiles and realistic workload conditions for performance.
• Test restoration, not merely backup creation.
• Separate provider responsibilities from customer responsibilities.
• Model cost using explicit units and sensitivity analysis.
• Treat sustainability as an engineering requirement with measurable indicators.
• Design observability, incident response and governance before production.
• Re-evaluate architecture after major business, workload, regulatory or technology changes.

Chapter Summary

Architecture evaluation determines whether a cloud design is fit for business, technical, operational, legal and environmental requirements. A professional process begins with measurable quality-attribute scenarios, constraints and assumptions. It compares alternatives using common criteria while recognising that weighted scores support, rather than replace, judgement.

End-to-end architecture descriptions should include context, services, deployment, security, data lifecycle and operations. Prototypes provide evidence about uncertain decisions, while Architecture Decision Records preserve rationale and consequences. Quantitative techniques—including availability models, Little’s Law, capacity estimates and cost decomposition—are valuable when their assumptions and limitations are stated.

The CareConnect case illustrates a managed-container architecture selected through analysis of security, reliability, performance, cost, compliance and team capability. The final professional obligation is transparent communication: explain what is known, what is assumed, what remains risky and what evidence will trigger change.

Key Terms

Architecture evaluation: Systematic assessment of whether an architecture satisfies stakeholder requirements.

Architectural driver: A requirement, constraint or risk that significantly influences design.

Quality attribute: A measurable non-functional property such as availability, security or performance.

Quality-attribute scenario: A structured requirement specifying stimulus, environment, response and measure.

Architecture Decision Record: A persistent record of context, decision, alternatives and consequences.

Prototype: A limited implementation created to investigate uncertainty.

Trust boundary: A boundary across which data or authority requires explicit protection.

Failure domain: A set of components exposed to a common failure.

Availability: The proportion of time a service is usable according to an agreed definition.

MTBF: Mean time between failures.

MTTR: Mean time to repair or restore service.

RPO: Maximum acceptable data loss measured in time.

RTO: Target time to restore an agreed service level.

Tail latency: High-percentile response time, commonly p95 or p99.

Little’s Law: The relationship L = λW between concurrency, throughput and response time in a stable system.

Residual risk: Risk remaining after controls and mitigations.

Shared responsibility: Allocation of security and operational duties between provider and customer.

Sensitivity analysis: Examination of how results change when assumptions or weights change.

Infrastructure as code: Declarative, version-controlled management of infrastructure.

Sustainability: Management of environmental impacts and resource efficiency across the system lifecycle.

Review Questions

Short Answer Questions

1. What distinguishes architecture evaluation from implementation testing?
2. Define RPO and RTO, and explain how they differ.
3. What information should an Architecture Decision Record contain?
4. Why are p95 and p99 latency often more useful than average latency?
5. State two limitations of availability calculations based on independent failures.

Long Answer Questions

1. Explain how business outcomes can be transformed into quality-attribute scenarios and evaluation metrics.
2. Describe a complete prototype and evidence plan for a proposed cloud architecture.
3. Compare multi-zone and multi-region resilience, including cost, compliance and operational trade-offs.
4. Explain how security, compliance and shared responsibility should influence capstone architecture design.
5. Discuss how a weighted decision matrix can support, and potentially distort, architectural judgement.

Analytical / Application Questions

1. A service has three sequential components with availabilities of 0.999, 0.995 and 0.9995. Estimate the system availability under the independence assumption, then identify two reasons the real result may differ.
2. Design an evaluation plan for an online examination platform expecting sudden demand spikes. Specify requirements, prototype tests, observability evidence and recovery tests.
3. Prepare a recommendation between managed containers and serverless services for a regulated transaction system. State assumptions, decision criteria, residual risks and the condition that would cause you to revisit the decision.

Multiple Choice Questions with Answers

Question 1

Which item most directly converts a vague quality requirement into a testable statement?

A. A quality-attribute scenario

B. A product catalogue

C. A source-code comment

D. A billing invoice

Correct Answer: A. A quality-attribute scenario

Question 2

Under the stated independence assumption, what is the availability of two parallel components each with availability 0.99?

A. 0.9801

B. 0.9900

C. 0.9999

D. 1.9900

Correct Answer: C. 0.9999

Question 3

Which evidence best demonstrates that a recovery point objective is achievable?

A. A diagram showing a backup service

B. A successful restore and data-loss measurement

C. A higher CPU limit

D. A list of unused regions

Correct Answer: B. A successful restore and data-loss measurement

Question 4

What does Little’s Law express in a stable system?

A. Cost equals usage multiplied by tax

B. Availability equals failures divided by repairs

C. Concurrency equals throughput multiplied by response time

D. Latency equals storage divided by bandwidth

Correct Answer: C. Concurrency equals throughput multiplied by response time

Question 5

What is a principal purpose of an Architecture Decision Record?

A. To hide rejected alternatives

B. To preserve decision context and consequences

C. To replace all security testing

D. To guarantee vendor portability

Correct Answer: B. To preserve decision context and consequences

Question 6

Which statement about a weighted decision matrix is most accurate?

A. It eliminates architectural judgement

B. It is valid without defining scoring scales

C. It supports comparison but requires sensitivity analysis

D. It proves that the highest score is objectively correct

Correct Answer: C. It supports comparison but requires sensitivity analysis

Question 7

Which control most directly addresses broken object-level authorisation?

A. Increasing disk capacity

B. Checking whether the requester may access the specific object

C. Adding a content delivery network

D. Reducing log retention

Correct Answer: B. Checking whether the requester may access the specific object

Question 8

A system receives 36,000 requests in 10 minutes. What is its average request rate?

A. 6 requests per second

B. 60 requests per second

C. 600 requests per second

D. 3,600 requests per second

Correct Answer: B. 60 requests per second

Question 9

Which statement best describes a prototype in architecture evaluation?

A. It proves every production property

B. It investigates a defined uncertainty using representative evidence

C. It is always disposable and undocumented

D. It replaces operational monitoring

Correct Answer: B. It investigates a defined uncertainty using representative evidence

Question 10

Why can adding replicas fail to deliver the expected availability improvement?

A. Replicas always reduce throughput

B. Replicas cannot store data

C. Replicas may share correlated failure dependencies

D. Replicas prevent encryption

Correct Answer: C. Replicas may share correlated failure dependencies