Safety, Cybersecurity, Ethics and Governance

Introduction

Mechatronic systems increasingly combine sensors, actuators, embedded control, industrial networks, machine learning, cloud services and human operators. This integration creates substantial benefits—greater productivity, adaptive control, predictive maintenance and safer collaboration—but it also changes the nature of engineering risk. A robot may be mechanically safe yet vulnerable to a compromised network. An AI vision system may be accurate on test images but fail when lighting, object orientation or adversarial markings change. A technically lawful data-collection practice may still violate worker expectations of privacy.

Safety and governance must therefore be treated as system properties, not as features added after implementation. Engineers must identify hazards, estimate and reduce risk, protect information and control systems, justify automated decisions, preserve human authority and maintain evidence that decisions were responsible.

This chapter presents an integrated approach to functional safety, AI safety, cybersecurity, privacy, explainability, accountability and human oversight in mechatronic applications.

Learning objectives

After studying this chapter, a student should be able to:

1. Distinguish functional safety, cybersecurity, AI safety, privacy and broader ethical responsibility.
2. Apply a risk-based lifecycle to an AI-enabled machine or robot.
3. Explain the roles of major machinery, robotics, functional-safety, cybersecurity and AI-management standards.
4. Analyze adversarial, data, model, network and human-factor threats.
5. Design practical controls for explainability, accountability, logging and human oversight.
6. Evaluate a mechatronic deployment using a structured governance case.

1. The integrated risk problem

A hazard is a potential source of harm. Risk combines the severity of possible harm with the likelihood that the harm will occur. In a qualitative engineering assessment, risk may be represented as:

where R is risk, S is severity of harm and P is probability or likelihood of occurrence. This is a conceptual relationship, not a universal numerical law. Actual machinery standards commonly use structured categories or risk graphs rather than multiplying arbitrary scores.

A useful distinction is:

These domains overlap. For example, a network attack that changes a robot's speed is both a cybersecurity incident and a functional-safety hazard. A camera used for collision avoidance may also capture identifiable workers, creating privacy obligations.

2. Functional safety of mechatronic systems

Functional safety concerns the ability of a system to achieve or maintain a safe condition in response to inputs, faults or specified operating conditions. It is not identical to general machine safety. Guards, safe distances and mechanical design are important, while functional safety addresses safety-related control systems such as emergency stops, interlocks, safe limited speed and safe torque off.

2.1 Safety lifecycle

A defensible lifecycle includes:

1. Define the machine, intended use, reasonably foreseeable misuse and operating environment.
2. Identify hazards across installation, production, maintenance, cleaning and decommissioning.
3. Estimate and evaluate risk.
4. Select inherently safe design measures before safeguards and administrative measures.
5. Specify safety functions and their required performance.
6. Implement, verify and validate the safety-related control system.
7. Commission under controlled conditions.
8. Monitor, maintain, modify and retire the system.

ISO 12100 provides a central machinery risk-assessment and risk-reduction framework. IEC 61508 provides a generic functional-safety foundation based on safety integrity levels (SIL). ISO 13849-1 addresses safety-related parts of control systems using performance levels (PL). ISO 10218 addresses industrial robot safety, while ISO/TS 15066 provides guidance for collaborative robot applications. The applicable edition and national adoption must always be checked; standards are not substitutes for legislation or competent engineering judgment.

2.2 Safety functions and independence

A safety function should have a clear trigger, safe response, response-time requirement, reset behaviour and diagnostic strategy. For example:

If a protective door opens while hazardous motion is possible, the system shall remove the relevant drive energy within the specified stopping time and shall prevent automatic restart until the hazard is controlled and a deliberate reset occurs.

AI should not normally be the sole decision-maker for a critical protective function unless its safety case, validation evidence and applicable regulations support that use. A conventional safety-rated circuit may supervise a machine-learning perception system, or a model may provide advisory information while an independent safety layer limits speed and force.

Verification asks whether the design satisfies specified requirements. Validation asks whether the complete machine, in its intended context, actually provides adequate risk reduction. Both must include foreseeable faults, sensor occlusion, communication loss, restart conditions and maintenance modes.

3. AI safety and assurance

AI safety extends beyond component failure. Machine-learning systems can fail because of incomplete data, distribution shift, ambiguous labels, unsafe optimization objectives, overconfident predictions or interactions with people and other machines.

Important safety properties include:

• Validity: outputs correspond adequately to the intended task.
• Robustness: performance remains acceptable under noise, variation and foreseeable disturbances.
• Uncertainty awareness: the system can identify low-confidence or out-of-distribution cases.
• Fail-safe behaviour: uncertainty or malfunction leads to a controlled state rather than an unsafe command.
• Traceability: data, model versions, configuration and decisions can be reconstructed.
• Human controllability: authorized people can supervise, override, pause or shut down the system.

A classifier's accuracy alone is insufficient. In a collision-avoidance system, a missed detection may be much more serious than a false alarm. Engineers should therefore report class-specific measures such as precision and recall, test performance by operating condition and define an abstention or escalation policy.

For binary classification:

Here TP is true positives, FP false positives and FN false negatives. These ratios are meaningful only when the classes, decision threshold, test population and costs of errors are stated. A safety case should not claim that a model is safe merely because its average accuracy is high.

3.1 Safety cases

A safety case is a structured argument supported by evidence. A practical claim–argument–evidence structure is:

• Claim: The robot cannot exceed the permitted collaborative speed when a person enters the monitored zone.
• Argument: Independent protective sensing detects entry, a safety-rated controller limits motion, and model uncertainty causes a safe stop.
• Evidence: Requirements, hazard analysis, test results, fault-injection tests, validation records, source and configuration control, and maintenance procedures.

AI components require additional evidence: dataset provenance, labeling quality, coverage of operating conditions, robustness tests, model drift monitoring and change-impact analysis.

4. Cybersecurity for connected mechatronics

Industrial systems have distinctive cybersecurity constraints. Availability and physical safety may be more important than confidentiality; a software update that is routine in an office may stop production or create hazardous motion in a factory.

Threats include:

• unauthorized access through remote services or exposed controllers;
• manipulation of sensor data or commands;
• ransomware and denial of service;
• theft of models, recipes, production data or personal information;
• supply-chain compromise in firmware, libraries or vendor tools;
• replay, spoofing and man-in-the-middle attacks;
• adversarial examples that cause perception errors;
• insecure maintenance laptops, removable media and default passwords.

A practical defense-in-depth architecture separates enterprise IT, industrial DMZ, supervisory systems, control networks and safety systems. Segmentation reduces propagation but does not eliminate risk. Controls include asset inventory, threat modeling, least privilege, multifactor authentication for remote access, secure configuration, network monitoring, cryptographic protection, signed updates, vulnerability management, backups, incident response and recovery testing.

IEC 62443 is particularly relevant to industrial automation and control-system security. ISO/IEC 27001 provides an information-security management-system framework. IEC 61508 and machinery standards address safety rather than replacing cybersecurity controls. Security mechanisms must be designed so that authentication failure, loss of communication or security monitoring does not produce an uncontrolled physical condition.

Figure: A defense-in-depth architecture in which AI assists control while an independent safety layer constrains hazardous motion.

5. Privacy and data governance

Sensors in factories can collect faces, body posture, voice, location, productivity patterns and biometric information. Privacy engineering begins with purpose specification and data minimization, not with storing everything “just in case.”

Key principles include:

• lawful, fair and transparent processing;
• purpose limitation;
• collection of only necessary data;
• accuracy and correction mechanisms;
• retention limits;
• confidentiality and access control;
• accountability and documented impact assessment.

Possible design measures include edge processing, anonymization or pseudonymization, low-resolution sensing, deletion schedules, role-based access and clear notices. Anonymization is difficult: combining time, location and production records may re-identify individuals. Consent may be problematic in an employment relationship because workers may not feel free to refuse. Engineers must work with legal, occupational-safety and labor representatives rather than assuming that technical capability creates permission.

6. Explainability, accountability and human oversight

Explainability means making system behaviour understandable for its audience and purpose. A maintenance engineer may need a feature contribution or confidence trend; a worker affected by an allocation decision may need a plain-language reason and a route for review. A technically impressive explanation that does not reflect the model's actual causal operation can mislead.

Useful forms include:

• Global explanations: what factors generally influence the model.
• Local explanations: why a particular output was produced.
• Example-based explanations: similar cases or counterfactuals.
• Process explanations: data source, model version, threshold and human approvals.

Explainability does not prove correctness. A model can provide a plausible explanation for an unsafe result. It should therefore complement testing, monitoring and independent controls.

Accountability requires named responsibility across the lifecycle: owner, designer, integrator, operator, maintainer, cybersecurity lead and incident investigator. A responsibility matrix should define who may approve deployment, change thresholds, accept residual risk, stop operation and notify affected parties.

Human oversight must be meaningful, not ceremonial. The human must have sufficient competence, time, information and authority to intervene. “Human in the loop” is inadequate if the operator receives an alarm too late, cannot understand its significance or is penalized for stopping a machine. High-risk systems should support a graduated response: warn, slow, pause, transfer control or shut down.

7. Ethical and regulatory governance

Ethical analysis asks whether a deployment respects safety, dignity, fairness, autonomy, sustainability and legitimate expectations. Relevant questions include:

• Who benefits and who bears the risk?
• Could a proxy variable discriminate against a group?
• Is surveillance proportionate to the safety purpose?
• Can an affected person challenge an automated decision?
• Does automation deskill workers or create unrealistic performance pressure?
• Are energy use, electronic waste and supply-chain impacts acceptable?

Risk governance should classify systems by intended use and potential harm. The European Union AI Act, where applicable, uses a risk-based legal framework and imposes obligations that vary by system category. Other jurisdictions use sectoral safety, privacy, product-liability and cybersecurity laws. Standards such as ISO/IEC 42001 support an AI management system, while ISO/IEC 23894 provides guidance on AI risk management. These instruments do not remove the need to comply with machinery, occupational-safety, data-protection and product regulations.

A governance board or deployment authority should require a documented impact assessment before operational use. The assessment should cover intended purpose, prohibited uses, affected people, hazards, cybersecurity threats, privacy, performance limits, residual risk, monitoring, incident response and retirement.

8. Risk-based governance workflow

A robust workflow is:

1. Define context: system boundaries, stakeholders, intended use and foreseeable misuse.
2. Classify consequences: safety, environmental, financial, privacy and social impacts.
3. Analyze hazards and threats: use hazard analysis, threat modeling and data-protection assessment together.
4. Specify controls: prioritize inherent risk reduction, independent safeguards and secure architecture.
5. Set assurance gates: require evidence before prototype testing, pilot operation and production release.
6. Operate with monitoring: track safety events, near misses, drift, cybersecurity alerts, complaints and overrides.
7. Manage change: reassess risk after new data, software, tooling, layout, suppliers or operating conditions.
8. Learn and retire: investigate incidents, preserve evidence, communicate lessons and decommission securely.

A simple residual-risk concept is useful:

This is a conceptual expression only. Risk reduction is not generally additive or measured reliably by subtracting arbitrary scores. It emphasizes that controls must be evaluated and that residual risk requires explicit acceptance by an authorized role.

9. Worked example: AI vision in a collaborative assembly cell

Consider an illustrative case. A manufacturer introduces a camera-based model to identify component orientation and guide a collaborative robot. The model also estimates whether a person's hand is entering the work zone.

Background and problem

The system improves cycle time, but pilot tests show occasional missed detections when gloves have reflective surfaces. The camera records workers continuously, and the robot controller is connected to a vendor remote-support service.

Analysis

The hazard is contact between robot motion and a person. Contributing causes include perception failure, network compromise, incorrect model configuration, delayed stopping and unsafe restart. Privacy risks arise from images and worker movement records. Cybersecurity risks include stolen remote credentials and manipulation of camera streams.

Suppose a validation test contains 1,000 relevant hand-entry events: 980 are correctly detected, 10 are false alarms and 20 are missed. The recall is:

Thus recall is 98% for this test set. This result does not establish acceptability because the test may not represent all lighting, gloves, speeds or occlusions, and a 2% miss rate may be unacceptable for the hazard.

Approach and solution

The engineering team takes the following actions:

• uses the AI for task guidance but not as the sole protective function;
• installs an independent safety-rated presence-sensing and speed-limiting function;
• causes low confidence, camera loss or abnormal latency to produce a controlled pause;
• tests reflective gloves, occlusion, lighting changes, network loss and adversarial markings;
• segments the robot network and disables persistent vendor access;
• requires time-limited, approved remote sessions with multifactor authentication;
• processes images at the edge, avoids retaining identifiable video and publishes a worker notice;
• records model version, threshold, operator intervention and safety events;
• establishes a change board and periodic revalidation trigger;
• trains operators to stop the cell and report near misses without penalty.

Outcome

The deployment is approved only for the defined work cell and operating envelope. The safety function is validated independently, while the AI is monitored for drift and confidence changes. A later software update triggers a partial reassessment rather than being released automatically.

Learning points

The case demonstrates that accuracy is evidence, not a safety argument by itself. Independence of protective controls, privacy-by-design, secure remote access, human authority and change management are all necessary. Governance converts technical assumptions into explicit operational responsibilities.

10. Verification, monitoring and incident response

Before release, teams should maintain a requirements traceability matrix linking hazards to controls, tests and evidence. Testing should include nominal operation, boundary conditions, faults, sensor failure, model drift, data poisoning, network interruption, unauthorized commands and recovery after power loss.

Operational indicators may include near misses, emergency stops, unsafe overrides, false-negative rates, confidence distributions, system availability, patch age, failed logins and unresolved vulnerabilities. Metrics must not create perverse incentives. For example, reducing recorded emergency stops by discouraging operators from using them is not improved safety.

An incident plan should define detection, immediate stabilization, communication, evidence preservation, root-cause analysis, corrective action and return-to-service approval. Safety incidents and cyber incidents should be coordinated because containment actions—such as disconnecting a controller—can themselves affect physical safety.

11. Challenges and limitations

Risk assessments are limited by incomplete information and changing environments. Standards may lag emerging foundation models and autonomous behaviours. Certification of a component does not certify the whole integrated system. Explainability methods can be unstable or misleading. Privacy protection may conflict with the data needed for debugging. Human oversight can fail through alarm fatigue, automation bias or inadequate staffing. Cybersecurity controls can increase complexity and maintenance burden.

These limitations favor conservative system boundaries, explicit uncertainty, independent safeguards, staged deployment and continuous reassessment. No governance framework can guarantee zero risk; responsible engineering makes assumptions visible and keeps residual risk controlled and accountable.

12. Best-practice checklist

• Define intended use, excluded use and operating envelope.
• Apply ISO 12100-style hazard identification and risk reduction.
• Specify safety functions independently of convenience features.
• Use appropriate ISO 13849, IEC 61508, ISO 10218 or related guidance.
• Threat-model controllers, sensors, models, suppliers and remote access.
• Apply IEC 62443 principles and an information-security management process.
• Minimize personal data and define retention and access rules.
• Test AI by condition, class, threshold and consequence—not average accuracy alone.
• Log data, model, configuration, decisions, interventions and changes.
• Provide competent, empowered human oversight and a physical stop mechanism.
• Establish named accountability, approval gates and incident procedures.
• Reassess after changes, near misses, attacks or evidence of drift.

Chapter Summary

Safe AI-enabled mechatronics requires integration of functional safety, AI assurance, cybersecurity, privacy, ethics and governance. Functional safety specifies and validates protective functions; AI safety addresses uncertainty, robustness, drift and unsafe generalization. Cybersecurity protects physical systems from malicious compromise and must be coordinated with safety. Privacy requires purpose limitation, minimization and controlled retention. Explainability supports understanding but cannot replace testing. Accountability assigns lifecycle responsibilities, while meaningful human oversight preserves intervention authority. Standards including ISO 12100, ISO 13849, IEC 61508, ISO 10218, ISO/TS 15066, IEC 62443, ISO/IEC 27001, ISO/IEC 23894 and ISO/IEC 42001 provide useful structures when applied with applicable law and competent judgment. Risk-based governance uses documented context, hazard and threat analysis, assurance gates, monitoring, change control and incident learning.

Key Terms

Hazard: A potential source of harm.

Risk: The combination of possible harm severity and likelihood.

Functional safety: Safety achieved through the correct operation of safety-related control systems.

Safety function: A control function intended to reduce a specified hazard.

Performance level: A reliability-related classification used for safety-related control systems under ISO 13849.

Safety integrity level: A measure of the ability of a safety-related system to perform its function under IEC 61508.

Safety case: A structured claim, argument and evidence demonstrating justified safety.

AI safety: Assurance that AI behaves acceptably, robustly and controllably within its intended context.

Distribution shift: A change between training conditions and operational conditions.

Adversarial example: An input deliberately or unintentionally designed to cause an AI system to make an incorrect output.

Defense in depth: Multiple independent or partially independent layers of protection.

Threat modeling: Systematic identification of assets, threats, vulnerabilities and mitigations.

Privacy by design: Incorporating privacy protections into system architecture from the beginning.

Explainability: The degree to which system behaviour or outputs can be understood by an appropriate audience.

Accountability: The assignment and acceptance of responsibility for decisions and outcomes.

Human oversight: Competent human supervision with information, time and authority to intervene.

Residual risk: Risk remaining after controls have been implemented.

Model drift: Deterioration or change in model behaviour as data or operating conditions change.

Governance: Structures and processes for directing, controlling and evidencing responsible system use.

Review Questions

Short Answer Questions

1. Distinguish functional safety from cybersecurity in an AI-enabled robot.
2. Why is accuracy alone insufficient for assessing a safety-critical AI model?
3. State two purposes of an independent safety layer.
4. What is meant by meaningful human oversight?
5. Give two privacy-by-design measures for a factory vision system.

Long Answer Questions

1. Explain how ISO 12100, ISO 13849 and IEC 61508 contribute to the safety lifecycle of a mechatronic system.
2. Discuss defense in depth for a connected collaborative robot, including technical and organizational controls.
3. Explain how a safety case can be constructed for an AI perception function.
4. Analyze the relationship among explainability, accountability and human oversight.
5. Develop a risk-based governance workflow for deploying an AI-enabled production cell.

Analytical / Application Questions

1. A vision model has 950 true positives, 30 false positives and 50 false negatives. Calculate its precision and recall, state the assumptions, and explain why these results do not by themselves establish safety.
2. A vendor requests permanent remote access to a robot controller for maintenance. Analyze the cybersecurity, safety, privacy and governance risks and propose an alternative access design.
3. An AI system used for worker posture monitoring shows lower performance for one group of workers and stores identifiable video for twelve months. Recommend an engineering and governance response, including testing, privacy controls and worker oversight.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes functional safety from cybersecurity?

A. Functional safety addresses only confidentiality, while cybersecurity addresses only stopping motion.

B. Functional safety concerns safe responses to faults and hazards, while cybersecurity concerns malicious compromise and unauthorized access.

C. Functional safety applies only to software, while cybersecurity applies only to mechanical guards.

D. They are identical terms with different names.

Correct Answer: B. Functional safety concerns safe responses to faults and hazards, while cybersecurity concerns malicious compromise and unauthorized access.

Question 2

In the chapter’s worked example, what is the recall for 980 true positives and 20 false negatives?

A. 0.50

B. 0.90

C. 0.98

D. 0.99

Correct Answer: C. 0.98

Question 3

Which standard is specifically associated with industrial automation and control-system cybersecurity?

A. IEC 62443

B. ISO 12100

C. ISO 10218

D. ISO/TS 15066

Correct Answer: A. IEC 62443

Question 4

Why should an AI vision model not normally be the sole protective function for hazardous robot motion?

A. AI models cannot process images in real time.

B. A model may fail under uncertainty, distribution shift or adversarial conditions.

C. Safety functions must always be mechanical.

D. Vision systems cannot be tested.

Correct Answer: B. A model may fail under uncertainty, distribution shift or adversarial conditions.

Question 5

Which measure most directly supports privacy by design in a factory camera system?

A. Retaining all video indefinitely

B. Publishing the model architecture only

C. Processing necessary information at the edge and minimizing retention

D. Allowing all supervisors unrestricted access

Correct Answer: C. Processing necessary information at the edge and minimizing retention

Question 6

What is the main purpose of a safety case?

A. To advertise the robot’s productivity

B. To provide a structured claim, argument and evidence for justified safety

C. To replace all physical safeguards

D. To eliminate the need for validation

Correct Answer: B. To provide a structured claim, argument and evidence for justified safety

Question 7

Which condition is necessary for meaningful human oversight?

A. The operator has no authority to stop the system.

B. The operator receives only a final decision after the event.

C. The operator has competence, time, information and authority to intervene.

D. The operator accepts every automated recommendation.

Correct Answer: C. The operator has competence, time, information and authority to intervene.

Question 8

What does defense in depth mean in industrial cybersecurity?

A. Using one highly complex security tool

B. Using multiple layers of protection so one failure does not expose the whole system

C. Removing network segmentation

D. Prioritizing confidentiality over physical safety in every case

Correct Answer: B. Using multiple layers of protection so one failure does not expose the whole system

Question 9

Which event should normally trigger reassessment of an AI-enabled machine?

A. A change to software, operating conditions or model data

B. Routine cleaning performed as specified

C. Printing an operating manual

D. A normal shift change with no system modification

Correct Answer: A. A change to software, operating conditions or model data

Question 10

What is the principal limitation of an explanation generated by an AI model?

A. It always proves that the output is correct.

B. It can be plausible without demonstrating correctness or causal validity.

C. It cannot be communicated to operators.

D. It eliminates the need for performance testing.

Correct Answer: B. It can be plausible without demonstrating correctness or causal validity.