CPS Engineering Lifecycle, Verification and Validation

Introduction and Context

A cyber-physical system (CPS) integrates computational elements, communication networks, sensors, actuators, physical processes, human operators, and organizational constraints. Examples include autonomous vehicles, collaborative robots, smart manufacturing cells, medical devices, and energy-management systems. In such systems, a software defect can cause a physical hazard, while an unmodeled physical effect can invalidate an otherwise correct algorithm.

CPS engineering therefore requires more than conventional software development or traditional mechanical design. Engineers must establish traceable requirements, construct models of interacting domains, simulate behavior, integrate heterogeneous components, verify implementation against specifications, and validate the complete system in its intended operational context. The engineering lifecycle must also control evolving hardware, firmware, software, models, data, and test environments.

Verification and validation are related but distinct. Verification asks, “Did we build the system right?” Validation asks, “Did we build the right system for its intended use?” Both questions must be addressed continuously rather than postponed until final acceptance testing.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Describe a lifecycle for engineering cyber-physical systems.
2. Derive testable requirements from stakeholder needs and operational scenarios.
3. Explain model-based systems engineering, simulation, co-simulation, and hardware-in-the-loop testing.
4. Distinguish verification, validation, testing, inspection, analysis, and demonstration.
5. Apply traceability, configuration management, formal verification, and DevOps practices.
6. Interpret quantitative verification results and identify their limitations.
7. Evaluate safety, cybersecurity, ethical, regulatory, and organizational implications.

1. The CPS Engineering Lifecycle

A CPS lifecycle is a controlled progression from need identification to retirement. A practical lifecycle is iterative rather than strictly linear because test results, operational evidence, and design changes frequently expose new requirements or assumptions.

The principal activities are:

1. Mission and stakeholder analysis: Identify users, operators, maintainers, affected communities, regulators, and environmental constraints.
2. Operational concept development: Describe how the system will be used, including normal, degraded, emergency, maintenance, and misuse situations.
3. Requirements engineering: Convert needs into measurable, feasible, consistent, and traceable system requirements.
4. Architecture and allocation: Partition functions among mechanical, electrical, embedded, network, human, and organizational elements.
5. Modeling and design: Represent structure, behavior, interfaces, states, timing, data, hazards, and physical relationships.
6. Implementation and integration: Build components and integrate them incrementally.
7. Verification: Demonstrate that each product satisfies its specified requirements.
8. Validation: Establish that the integrated system satisfies its intended mission in a representative context.
9. Deployment and operation: Monitor performance, incidents, configuration, maintenance, and changing assumptions.
10. Evolution or retirement: Manage upgrades, obsolescence, safe decommissioning, and disposal of data and hardware.

Figure: An iterative CPS lifecycle in which evidence and controlled feedback connect all stages.

Lifecycle models may be waterfall, V-model, incremental, spiral, or agile-hybrid. The V-model is particularly useful for explaining correspondence between decomposition and integration: stakeholder needs are refined into system, subsystem, and component requirements; implementation occurs near the bottom of the V; integration and acceptance tests proceed upward against the corresponding specifications. In practice, CPS projects often use agile software iterations within a systems-engineering framework because physical prototypes, certification evidence, and procurement constraints require longer planning horizons.

2. Requirements Engineering for CPS

A requirement is a necessary capability, condition, or constraint that a system must satisfy. Good requirements are singular, unambiguous, feasible, necessary, consistent, verifiable, and traceable. A useful structure identifies the subject, required behavior, conditions, measurable threshold, and verification method.

For example, “The mobile robot shall stop within 0.40 m of detecting an obstacle at speeds up to 1.0 m/s on a dry, level surface” is more useful than “The robot shall stop quickly.” The former defines a measurable performance requirement and its operating conditions.

Requirements should be classified as functional, performance, interface, environmental, safety, cybersecurity, human factors, regulatory, maintainability, or lifecycle requirements. Quality attributes such as latency, availability, reliability, explainability, and energy consumption often interact. Improving one can degrade another, so trade-offs must be documented rather than hidden.

A requirements baseline is an approved version of requirements under configuration control. Each requirement should have a unique identifier, rationale, source, priority, allocated element, acceptance criterion, verification method, and status. Bidirectional traceability links stakeholder needs to system requirements, architecture, implementation items, hazards, tests, results, and approved deviations.

Requirements quality and acceptance criteria

A requirement is not fully testable merely because it contains a number. The measurement method, test conditions, tolerances, instrumentation accuracy, sampling rate, and pass/fail rule must be defined. If a temperature sensor must operate from −20 °C to 60 °C, the requirement should also identify whether the range refers to ambient temperature, sensor body temperature, or measured process temperature.

Safety and security requirements should be derived from structured analyses. Failure Mode and Effects Analysis (FMEA) examines component failure modes and their effects. Fault Tree Analysis (FTA) starts with a hazardous top event and decomposes it into combinations of failures. Hazard analysis should consider accidental faults, systematic design errors, foreseeable misuse, adversarial actions, and common-cause failures.

3. Model-Based Systems Engineering

Model-Based Systems Engineering (MBSE) uses an integrated system model as a principal engineering artifact. The model represents stakeholders, requirements, structure, behavior, interfaces, states, constraints, verification cases, and dependencies. It complements documents; it does not eliminate the need for readable specifications and records.

A CPS model commonly contains:

• Structural views: components, ports, connectors, networks, and allocation relationships.
• Behavioral views: activities, state machines, events, modes, and interactions.
• Physical models: differential equations, kinematics, energy flows, and actuator or sensor characteristics.
• Information models: data types, units, timing, ownership, and communication protocols.
• Verification views: requirements, test cases, expected results, evidence, and unresolved risks.

Models support early analysis. A state-machine model can reveal unreachable or deadlock states. A timing model can expose a missed deadline. An interface model can reveal incompatible units or message assumptions. However, a model is an abstraction. Its conclusions apply only within its assumptions, parameter ranges, numerical accuracy, and fidelity.

Modeling assumptions and fidelity

A model should state what it includes and excludes. For a motor-driven axis, a low-order model may omit bearing friction, backlash, quantization, thermal effects, and structural resonance. Such a model may be suitable for controller tuning but unsuitable for predicting mechanical fatigue or acoustic behavior.

Model credibility requires evidence. Parameters should be sourced or measured, units should be explicit, numerical solvers should be appropriate to system stiffness and time scales, and results should be compared with independent observations. Calibration adjusts parameters using data; validation evaluates whether the calibrated model predicts separate data adequately. Calibration alone is not validation.

4. Simulation and Co-Simulation

Simulation executes a model to predict system behavior under specified inputs and assumptions. It can explore unsafe, expensive, rare, or long-duration scenarios before hardware is available. Common forms include:

• Model-in-the-loop (MIL): The controller and plant are both models.
• Software-in-the-loop (SIL): Production or near-production software runs against a simulated plant.
• Processor-in-the-loop (PIL): Code runs on the target processor or a representative processor while the plant remains simulated.
• Hardware-in-the-loop (HIL): Real hardware, such as a controller or actuator, interacts with a real-time plant simulation.

Co-simulation couples models from different tools or domains, such as a vehicle dynamics model, an embedded controller model, and a communication-network simulator. The coupling algorithm must address time synchronization, data exchange, solver step sizes, algebraic loops, numerical stability, and delays. A simulation can produce precise-looking but misleading results if one subsystem uses milliseconds while another interprets the same value as seconds, or if communication latency is omitted.

For a discrete-time controller with sampling period Tₛ, a simple sampled update can be represented as:

Here, x[k] is the state at sample k, u[k] is the control input, d[k] is a disturbance, and f is the modeled state-transition function. The equation assumes that the chosen sampling representation is adequate and that unmodeled continuous behavior between samples does not invalidate the conclusion. If the physical plant has significant dynamics faster than the sampling period, this abstraction may be unsafe.

Simulation results should include scenario definitions, initial conditions, parameter values, solver settings, random seeds where applicable, model versions, and pass/fail criteria. Reproducibility is an engineering requirement, not merely a research preference.

5. Verification and Validation

Verification activities evaluate whether a work product satisfies its specified requirements. Validation evaluates fitness for intended operational use. The same test may contribute to both, but the claims and evidence differ.

Verification must be planned at multiple levels: component, subsystem, integrated system, and acceptance. Verification methods should be selected according to risk. A low-risk display label may be inspected; an emergency braking function may require analysis, simulation, HIL, physical testing, fault injection, and operational validation.

A useful requirement coverage measure is:

where Nᵥ is the number of applicable requirements with approved verification evidence and Nᵣ is the total number of applicable requirements. The measure is meaningful only if requirements are uniquely identified, applicability is justified, and “evidence” meets the defined quality threshold. Coverage does not prove that requirements are correct, nor that tests are sufficient.

Test results should report measured values with units, uncertainty or tolerance where relevant, environmental conditions, equipment calibration status, software and hardware versions, deviations, and disposition of failures. A passing test with uncontrolled configuration is not reliable evidence.

6. Formal Verification

Formal verification uses mathematical models and logical reasoning to establish that specified properties hold. It is particularly valuable for finite-state controllers, communication protocols, scheduling logic, and safety interlocks.

Typical properties include:

• Safety: Something undesirable never occurs, such as two actuators being energized simultaneously.
• Liveness: Something desirable eventually occurs, such as an acknowledged command eventually receiving a response.
• Reachability: A state can or cannot be reached from an initial state.
• Invariants: A condition remains true throughout execution.

Model checking exhaustively explores states of a finite or suitably abstract model. The state-explosion problem can make direct analysis infeasible, so engineers use abstraction, compositional reasoning, symmetry reduction, bounded checking, or assume-guarantee contracts. The proof applies to the model and its assumptions; it does not automatically prove that the physical implementation, sensors, wiring, compiler, or operators satisfy them.

A practical formal specification might state: if the guard is open, the motor-enable command shall be false. To make this meaningful, the model must define how “guard open” is sensed, the timing of the signal, behavior under sensor failure, and whether a transient race is possible. Formal methods therefore strengthen, but do not replace, empirical testing and hazard analysis.

7. Hardware-in-the-Loop and Physical Testing

HIL testing places selected real components in a closed loop with a real-time simulation. For example, an actual motor controller may receive simulated position feedback while its commands drive a simulated motor. HIL enables repeatable fault injection, boundary testing, sensor deception tests, and high-speed execution without risking the full plant.

A valid HIL setup requires adequate real-time performance, calibrated I/O, correct scaling, representative delays, deterministic execution when required, and documented model fidelity. If the simulated plant runs slower than real time, the controller may experience unrealistic timing. If the I/O interface filters a signal differently from the actual sensor, a false conclusion may result.

Physical tests remain necessary because friction, vibration, electromagnetic interference, manufacturing variation, thermal behavior, aging, human interaction, and unforeseen coupling are difficult to model completely. Testing should progress from benign laboratory conditions to increasingly representative environments, with safety controls and a clear rollback or emergency-stop strategy.

8. DevOps, Continuous Integration, and Configuration Management

DevOps in CPS extends collaborative automation across software, firmware, models, test equipment, and sometimes mechanical and electrical design. A continuous integration pipeline can compile software, run static analysis, execute unit and model tests, check interfaces, generate artifacts, and publish evidence. Continuous delivery must be adapted to safety and regulatory constraints: automatic deployment is not appropriate when every release requires independent approval, but automated evidence generation remains valuable.

Configuration management establishes the identity, status, change history, and relationships of controlled items. Controlled items may include requirements, CAD files, schematics, source code, firmware binaries, parameter sets, simulation models, test scripts, datasets, tool versions, calibration records, and field configurations.

The central practices are:

1. Identification: Assign unique names and versions.
2. Baseline control: Freeze approved sets for defined purposes.
3. Change control: Evaluate impact, risk, verification needs, and authorization.
4. Status accounting: Record what version is installed, tested, released, or retired.
5. Configuration audits: Confirm that the product matches its approved description.
6. Reproducible builds and tests: Preserve toolchains, dependencies, seeds, and environments.

A change to a sensor filter may alter control stability, timing, safety behavior, cybersecurity exposure, and certification evidence. Impact analysis must therefore cross technical boundaries. Version control alone is not configuration management unless the organization also controls baselines, approvals, build dependencies, and released hardware.

9. Worked Example: Verification Coverage and Test Evidence

An automated packaging cell has 40 applicable system requirements. During a design review, 32 have approved verification evidence. Six have tests completed but the reports are incomplete, and two have no evidence. If only approved evidence counts, the requirement coverage is:

The six incomplete reports cannot automatically be counted as verified. Their tests may have been technically successful, but missing configuration identifiers, environmental conditions, or calibrated-instrument records weaken the evidence. After the reports are corrected and accepted, coverage could rise to:

The remaining two requirements may concern emergency stopping and cybersecurity. A 95% numerical coverage is not sufficient for release if the uncovered requirements are high risk. Coverage should therefore be read together with hazard severity, test quality, defect status, and residual uncertainty.

10. Illustrative Case: Collaborative Palletizing Robot

Background. A manufacturing company introduces a collaborative robot that picks cartons from a conveyor and places them on pallets. The system includes a vision sensor, robot controller, variable-speed conveyor, safety scanner, networked supervisory software, and a human operator who replenishes cartons.

Problem. Initial demonstrations succeed in normal operation, but the engineering team cannot show that the robot remains safe during network delay, sensor occlusion, restart after an emergency stop, or a changing carton geometry. Requirements are ambiguous, and the test laboratory has used a different firmware version from the production cell.

Analysis. The team conducts stakeholder and operational analyses. Hazards include collision with a person, dropped load, unexpected restart, and unsafe motion caused by stale vision data. Requirements are rewritten with measurable conditions. For example, the robot shall inhibit motion when the safety scanner reports a protective-zone intrusion, and restart shall require a deliberate operator action after the zone is clear. Timing budgets identify sensor acquisition, network transmission, controller computation, and actuator response as separate contributors.

Approach. The team creates an MBSE model linking requirements, robot modes, interfaces, hazards, and verification cases. A state-machine model is formally checked for forbidden transitions from “protective stop” directly to “automatic motion.” The vision and conveyor models are used in MIL and SIL testing. HIL testing injects delayed, missing, duplicated, and inconsistent messages while the real safety controller is connected to a real-time simulated robot. Physical tests then examine scanner occlusion, reflective materials, variable loads, emergency-stop recovery, and operator interaction. Configuration management binds the requirements baseline, controller firmware, safety parameters, model version, test scripts, and calibration records into a release candidate.

Outcome. The team discovers that a stale vision message can remain valid for longer than the assumed maximum age. The requirement is refined to include message age, and the controller rejects data beyond that limit. A second issue shows that the supervisory network can request motion after a safety stop, although the safety controller correctly inhibits the actuator. The interface is changed so that supervisory requests are ignored until a local reset sequence is completed. Validation with operators identifies an ambiguous status indicator, which is redesigned and retested.

Learning points. The case demonstrates that verification is not a final demonstration; it is a chain of evidence. Formal analysis can expose impossible or unsafe state transitions, HIL can reveal timing and communication faults, physical testing exposes environmental effects, and validation can uncover human-factors problems. Most importantly, safety depends on requirements, architecture, implementation, configuration, and operational context together.

11. Challenges, Limitations, and Governance

CPS verification faces several persistent challenges. The physical world is continuous and uncertain, while digital models are discrete and finite. Rare events may be difficult to reproduce. Components may be supplied by different organizations with incomplete evidence. Machine-learning components may not provide a simple deterministic explanation. Cybersecurity threats evolve after release. A successful simulation can create false confidence if model assumptions are not challenged.

AI-enabled perception and control require additional attention to dataset provenance, distribution shift, performance across relevant populations and environments, adversarial inputs, monitoring, and safe fallback behavior. Statistical performance estimates should state the population, sampling method, confidence assumptions, and decision threshold; a high test accuracy on a nonrepresentative dataset is not operational validation.

Governance requires defined authority for release, independent review for high-risk functions, secure handling of logs and personal data, protection of intellectual property, and transparent incident reporting. Safety standards, sector regulations, quality systems, and cybersecurity frameworks may impose specific lifecycle and evidence requirements. Engineers must consult the standards applicable to the domain rather than treating a generic checklist as certification.

Ethically, validation should consider people who are not direct customers but may be affected by the system. Accessibility, worker autonomy, surveillance, environmental impact, and the consequences of failure should be included in stakeholder analysis. “Technically compliant” does not necessarily mean socially acceptable.

12. Best-Practice Checklist

• Begin with operational scenarios, hazards, users, and measurable outcomes.
• Make every high-risk requirement uniquely identifiable and verifiable.
• Maintain bidirectional traceability from need to evidence.
• State model assumptions, units, parameter sources, solver settings, and limitations.
• Use MIL, SIL, PIL, HIL, and physical tests as complementary evidence.
• Select verification methods according to risk, not convenience.
• Include abnormal, degraded, fault, misuse, and cybersecurity scenarios.
• Control requirements, models, code, hardware, data, tools, and calibration records together.
• Automate repeatable checks while preserving human approval where required.
• Treat coverage metrics as indicators, not proof of system adequacy.
• Validate with representative users, environments, and operational workflows.
• Record uncertainty, deviations, residual risk, and the rationale for release decisions.

Chapter Summary

CPS engineering is an iterative lifecycle that connects mission needs, requirements, architecture, models, implementation, integration, verification, validation, operation, and retirement. Requirements must be measurable, traceable, and linked to hazards and acceptance evidence. MBSE provides a structured representation of system structure, behavior, interfaces, physical relationships, and verification relationships, while simulation and co-simulation enable early exploration of interactions and faults.

Verification establishes conformance to specified requirements through inspection, analysis, demonstration, testing, and formal methods. Validation establishes suitability in the intended operational context. MIL, SIL, PIL, HIL, and physical testing provide different kinds of evidence and must be selected according to risk and model limitations. Formal verification can establish properties of defined models, but its conclusions depend on assumptions and do not replace physical evidence.

DevOps and configuration management make CPS development reproducible and controlled across software, firmware, models, hardware, data, tools, and test environments. Traceability, change impact analysis, independent review, cybersecurity, human factors, ethics, and regulatory governance are essential because CPS failures can cross technical and organizational boundaries.

Key Terms

Requirement: A necessary capability, condition, or constraint that a system must satisfy.

Verification: Evaluation of whether a work product satisfies specified requirements.

Validation: Evaluation of whether a system is fit for its intended operational use.

Traceability: Ability to follow relationships among needs, requirements, designs, implementations, tests, and evidence.

Baseline: An approved and controlled version of a set of configuration items.

MBSE: Model-Based Systems Engineering using an integrated system model as a principal engineering artifact.

Model-in-the-loop: Simulation in which controller and plant are both models.

Software-in-the-loop: Execution of software against a simulated plant.

Processor-in-the-loop: Execution on a target or representative processor with a simulated plant.

Hardware-in-the-loop: Real hardware interacting with a real-time simulated environment.

Co-simulation: Coupled execution of models from different tools or domains.

Formal verification: Mathematical analysis establishing properties of a formal model.

Safety: Freedom from unacceptable risk of physical harm or damage.

FMEA: Analysis of component failure modes and their effects.

FTA: Top-down analysis of combinations of causes leading to a hazardous event.

Configuration management: Control of identity, versions, changes, status, and audits of system items.

Continuous integration: Automated and frequent integration and checking of engineering changes.

Requirement coverage: Proportion of applicable requirements supported by approved verification evidence.

Model fidelity: Degree to which a model represents relevant behavior of the real system.

Residual risk: Risk remaining after protective measures are applied.

Review Questions

Short Answer Questions

1. Distinguish verification from validation in one or two sentences.
2. What information should accompany a high-quality CPS requirement?
3. Explain the purpose of hardware-in-the-loop testing.
4. Why is configuration management needed for simulation models as well as source code?
5. Name two limitations of formal verification.

Long Answer Questions

1. Explain an iterative CPS engineering lifecycle and the role of feedback between lifecycle stages.
2. Compare MIL, SIL, PIL, HIL, and physical testing, including the evidence each can provide.
3. Describe how MBSE supports requirements traceability and multidisciplinary communication.
4. Explain how FMEA, FTA, and verification planning can be combined in a safety-critical CPS project.
5. Discuss why DevOps practices must be adapted when developing regulated or safety-critical CPS.

Analytical / Application Questions

1. A robot has 50 applicable requirements. Thirty-eight have approved evidence, seven have incomplete reports, and five have not been tested. Calculate approved requirement coverage and explain why the result alone cannot justify release.
2. A networked actuator occasionally receives commands delayed by 200 ms, although the controller design assumed a maximum delay of 50 ms. Identify the affected lifecycle artifacts and propose a verification and validation strategy.
3. Design a traceability and test strategy for a collaborative robot emergency-stop function. Include requirements, formal or model-based analysis, HIL tests, physical tests, configuration controls, and operator validation.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes verification from validation?

A. Verification evaluates intended use, while validation checks source code syntax.

B. Verification checks conformance to requirements, while validation checks fitness for intended operation.

C. Verification is performed only after deployment, while validation is performed only before design.

D. Verification applies only to hardware, while validation applies only to software.

Correct Answer: B. Verification checks conformance to requirements, while validation checks fitness for intended operation.

Question 2

Which activity most directly establishes bidirectional traceability?

A. Linking stakeholder needs to requirements, design elements, tests, and results

B. Increasing the simulation solver step size

C. Replacing all physical tests with demonstrations

D. Removing identifiers from test reports

Correct Answer: A. Linking stakeholder needs to requirements, design elements, tests, and results

Question 3

In the expression C_R = N_v / N_r × 100%, what does Nᵥ represent?

A. The number of unresolved hazards

B. The number of applicable requirements

C. The number of applicable requirements with approved verification evidence

D. The number of simulation variables

Correct Answer: C. The number of applicable requirements with approved verification evidence

Question 4

What is a principal purpose of hardware-in-the-loop testing?

A. To eliminate the need for requirements engineering

B. To connect real selected hardware to a real-time simulated environment

C. To prove that a physical model contains no assumptions

D. To replace all field validation

Correct Answer: B. To connect real selected hardware to a real-time simulated environment

Question 5

Which property states that an undesirable event never occurs?

A. Liveness

B. Reachability

C. Safety

D. Calibration

Correct Answer: C. Safety

Question 6

Why is model calibration not equivalent to model validation?

A. Calibration adjusts parameters, whereas validation evaluates prediction against separate evidence.

B. Calibration applies only to software, whereas validation applies only to hardware.

C. Calibration removes the need for independent data.

D. Validation never uses measurements.

Correct Answer: A. Calibration adjusts parameters, whereas validation evaluates prediction against separate evidence.

Question 7

A project has 40 applicable requirements and 32 with approved verification evidence. What is the approved requirement coverage?

A. 20%

B. 32%

C. 80%

D. 125%

Correct Answer: C. 80%

Question 8

Which item is most appropriately controlled by configuration management?

A. Only the final source-code file

B. Requirements, firmware, models, test scripts, tool versions, and calibration records

C. Only documents that contain equations

D. Only hardware purchased from external suppliers

Correct Answer: B. Requirements, firmware, models, test scripts, tool versions, and calibration records

Question 9

What is a major limitation of formal verification?

A. It cannot analyze finite-state controllers.

B. Its conclusions apply only to the model and assumptions that were verified.

C. It produces no evidence of any kind.

D. It is identical to informal visual inspection.

Correct Answer: B. Its conclusions apply only to the model and assumptions that were verified.

Question 10

Why should CPS testing include degraded and fault conditions?

A. Normal-operation tests always contain enough evidence for hazards.

B. Faults and abnormal interactions can produce behavior absent from nominal scenarios.

C. Degraded testing makes requirements unnecessary.

D. Fault testing is relevant only to mechanical systems.

Correct Answer: B. Faults and abnormal interactions can produce behavior absent from nominal scenarios.