13. Testability, Reliability, Safety and Hardware Security

Introduction and Context

Digital systems are no longer evaluated only by whether they implement their intended logic. A contemporary system-on-chip, industrial controller, automotive electronic control unit, medical instrument, or secure processor must also be testable, dependable over time, safe under abnormal conditions, and resistant to deliberate attack. These properties are related but not interchangeable. Testability concerns the ease with which manufacturing and field faults can be detected and diagnosed. Reliability concerns the probability that the system will perform its required function for a specified time under stated conditions. Safety concerns preventing unreasonable risk of injury, equipment damage, or environmental harm. Hardware security concerns protection against unauthorized observation, modification, imitation, or misuse.

A design may be highly testable but insecure if its scan chain exposes confidential data. It may be secure but unsafe if a fault causes an actuator to remain energized. Similarly, redundancy can improve reliability while increasing design complexity and creating additional attack surfaces. Engineering decisions must therefore consider the complete life cycle: specification, design, verification, production test, deployment, maintenance, retirement, and forensic analysis.

This chapter develops a unified framework for fault modeling, design-for-testability (DFT), built-in self-test (BIST), boundary scan, reliability assessment, functional safety, and hardware security. Particular attention is given to trade-offs among area, performance, power, coverage, diagnostic resolution, cost, safety integrity, and resistance to attack.

Learning Objectives

After studying this chapter, the learner should be able to:

1. Distinguish logical, physical, transient, intermittent, and malicious hardware faults.

2. Explain fault models and calculate basic fault coverage and test metrics.

3. Describe scan-based testing, BIST, boundary scan, and diagnosis strategies.

4. Evaluate ageing mechanisms, soft errors, redundancy, and reliability models.

5. Relate fault tolerance to functional safety and explain the role of safety standards.

6. Analyse secure boot, hardware roots of trust, side channels, and hardware Trojans.

7. Select suitable test, reliability, safety, and security controls for an engineered system.

One System, Several Notions of Correctness

A digital circuit has at least four relevant notions of correctness. Functional correctness means that the circuit produces specified outputs for valid inputs. Manufacturing quality means that fabricated devices do not contain unacceptable defects. Operational dependability means that the device continues to function despite environmental stress, ageing, or occasional disturbances. Trustworthiness means that the device behaves only as authorized and does not reveal or manipulate protected information.

These notions use different evidence. Functional verification relies on simulation, formal methods, and emulation. Production testing applies input patterns and observes responses. Reliability engineering uses failure-rate data, stress models, and accelerated testing. Safety engineering uses hazard analysis, diagnostic coverage, fault injection, and safety cases. Security evaluation considers adversarial capabilities, attack paths, information leakage, and supply-chain provenance.

The distinction between a fault, an error, and a failure is fundamental. A fault is a physical or logical cause of incorrect behavior, such as a shorted transistor or an incorrect control bit. An error is an incorrect internal state caused by a fault. A failure occurs when the delivered service violates its specification. A fault may remain dormant, an error may be masked by logic, and not every error produces an externally visible failure.

Fault Models and Test Metrics

A fault model is an abstraction that represents a class of physical defects in a form suitable for test generation and analysis. The single stuck-at model assumes that a signal line is permanently fixed at logic 0 or logic 1. Although simple, it detects many defects and remains a basic model for combinational and sequential circuits. A transition-delay fault represents a signal that cannot make a required low-to-high or high-to-low transition within the specified time. Path-delay and small-delay-defect models are more appropriate when timing defects are important.

Other models include bridging faults between conductors, open faults, transistor-level defects, memory-specific faults, and cell-aware faults that exploit the actual layout and electrical behavior of a standard cell. In memories, common models include stuck-at, transition, coupling, address-decoder, and retention faults. A soft error is generally caused by a transient disturbance, such as an energetic particle, rather than permanent physical damage. A malicious fault may be intentionally induced through voltage, clock, electromagnetic, laser, or software-assisted manipulation.

A test pattern detects a modeled fault when the fault changes an observable response. For a stuck-at fault, a test must first activate the fault by applying the opposite logic value and then propagate its effect to an observable output or scan element. The classical D-algorithm formalizes fault activation, propagation, and justification. Automatic test-pattern generation (ATPG) tools use these principles, often combined with implication, backtracking, and symbolic reasoning.

The principal metric is fault coverage:

Fault coverage = detected faults / total modeled faults × 100%.

Coverage must be interpreted carefully. A reported 99% coverage may exclude unmodeled defects, untestable faults, power-related failures, timing defects, or security-relevant modifications. Defect coverage estimates the fraction of physical defects detected, while fault coverage refers to an abstraction. Diagnostic coverage measures the proportion of relevant faults that a diagnostic mechanism detects, often with sufficient localization to support a safe response. Test escape is a defective part that passes production test; test yield loss is a good part rejected by test.

Test quality also depends on test application time, automatic test equipment (ATE) memory, pattern power, observability, controllability, and diagnosis resolution. A high-coverage test that causes excessive switching may create IR drop and falsely reject good devices or fail to detect defects under unrealistic operating conditions.

Design for Testability: From Controllability to Diagnosis

A circuit is controllable when internal nodes can be placed in required states through available inputs, and observable when internal behavior can be inferred at accessible outputs. Deep sequential logic often has poor controllability and observability because many clock cycles are needed to establish an internal state. DFT modifies the architecture to improve both properties.

The dominant technique is scan design. Ordinary storage elements are replaced or augmented with scan flip-flops. In functional mode they behave normally; in shift mode they form one or more serial shift registers. A typical test sequence is:

1. Shift a test state into the scan chain.

2. Apply one or more functional capture clocks.

3. Capture the circuit response.

4. Shift the response out for comparison.

Scan compression uses on-chip decompression logic, multiple internal chains, and response compaction to reduce external test data and test time. A multiple-input signature register (MISR) compacts many response bits into a signature. Compaction introduces aliasing: different responses may produce the same signature. The aliasing probability depends on the polynomial and the response structure, and must be assessed rather than assumed negligible.

Scan design creates trade-offs. Scan cells increase area and routing; shift activity can cause peak power; test modes require clock and reset control; and scan access may expose internal states. Locking scan access, encrypting test data, partitioning sensitive chains, and authenticating test modes are therefore important in security-sensitive products. Test points can improve coverage by adding controllability or observability, but they add overhead and may affect timing.

At-speed testing combines scan loading with functional-speed launch and capture. Launch-on-capture and launch-on-shift methods are used to detect transition and delay faults. Clock-domain crossings, asynchronous logic, memories, analog interfaces, and high-speed I/O frequently require specialized procedures beyond conventional scan.

Built-In Self-Test and Boundary Scan

In logic BIST (LBIST), circuitry on the chip generates test patterns, applies them to logic, compacts responses, and compares the final signature with a reference. A linear-feedback shift register (LFSR) is commonly used as a pseudorandom pattern generator, while a MISR performs response compaction. LBIST reduces dependence on external ATE and supports periodic in-system testing, but random patterns may poorly test random-pattern-resistant faults. Deterministic top-up patterns, weighted pattern generation, or test points may be needed.

Memory BIST (MBIST) applies an algorithm designed for memory fault behavior. March algorithms repeatedly write and read specified values while traversing addresses in increasing and decreasing order. A suitable March sequence can detect stuck-at, transition, coupling, and address faults more efficiently than general logic patterns. MBIST controllers must consider destructive testing, repair information, redundancy analysis, and the need to preserve or restore mission data.

Boundary scan, standardized principally through the IEEE 1149.1 family, places test cells at device pins and connects them into a test access port. Boundary scan can isolate board interconnects from core logic, drive known values onto nets, sample board signals, and identify device information through instruction registers. It is particularly valuable for dense boards where physical probing is difficult. Related standards extend the concept for advanced test access, embedded instruments, and system-level description.

Boundary scan is not a complete substitute for internal test. It may detect an open or short on an interconnect but cannot necessarily diagnose an internal timing defect. Its access mechanism must also be secured because an unlocked test port may permit unauthorized signal control, debug access, or extraction of sensitive state.

Reliability Across the Life Cycle

Reliability is often expressed as the probability that an item performs its required function for time t under specified conditions. For a constant failure rate λ, the exponential model gives:

R(t) = e^(-λt) and MTBF = 1/λ.

The constant-rate assumption is useful for random failures but does not describe all life stages. The bathtub curve commonly represents early failures, a useful-life region with approximately stable failure rate, and wear-out failures whose rate rises with age. Screening and process improvement reduce early failures; thermal management and derating reduce useful-life failures; replacement or refurbishment addresses wear-out.

Digital hardware is affected by several ageing mechanisms. Electromigration moves metal atoms under high current density and can create opens or shorts. Bias temperature instability shifts transistor threshold voltage. Hot-carrier effects degrade device characteristics. Time-dependent dielectric breakdown damages gate dielectrics. Negative-bias temperature instability and positive-bias temperature instability are especially relevant in advanced CMOS. Interconnect stress, solder fatigue, package delamination, and connector wear can dominate system-level reliability.

Temperature strongly influences failure rates. The Arrhenius relationship is often used for thermally activated mechanisms:

AF = exp[Ea/k (1/Tuse − 1/Tstress)],

where Ea is activation energy, k is Boltzmann’s constant, and temperatures are absolute. Accelerated testing is useful only when the stress mechanism at the test condition is representative of field behavior. Excessive voltage or temperature can activate a different failure mechanism and produce misleading conclusions.

Soft errors arise from transient charge disturbances caused by radiation, electrical noise, crosstalk, or power transients. They are not necessarily associated with permanent damage. Mitigations include error-detecting and correcting codes, parity, temporal redundancy, hardened latches, guard rings, spatial separation, watchdogs, and system-level retry. A single-event upset in configuration memory or control logic may have more serious consequences than one in a data register because it can persist or alter system behavior.

Redundancy may be spatial, temporal, informational, or functional. Dual modular redundancy detects disagreement but cannot determine which replica is correct. Triple modular redundancy (TMR) masks one faulty replica if the voter and remaining replicas operate correctly. Redundancy introduces common-cause vulnerabilities: shared power, clock, reset, software, physical proximity, or a faulty voter can defeat nominal independence. Reliability claims must therefore account for diagnostic coverage, repair, common-cause failure, and maintenance.

Functional Safety and Risk-Based Engineering

Safety begins with a hazard: a condition that can cause harm. Engineers identify hazardous events, estimate severity and probability, and define safety goals. A safety mechanism then detects or prevents dangerous faults, while a safe state limits consequences. Examples include de-energizing an actuator, limiting motor torque, activating a brake, or placing a medical device in a controlled fallback mode.

A useful distinction is between a safe fault and a dangerous fault. A safe fault either causes no hazardous behavior or is detected before it can do so. A dangerous fault can prevent a safety function from operating or can directly cause a hazard. Diagnostic coverage, fault reaction time, independence, and architectural constraints determine the residual risk.

Functional-safety standards are domain-specific. IEC 61508 provides a general framework for electrical, electronic, and programmable electronic safety-related systems and defines Safety Integrity Levels (SIL). ISO 26262 addresses road-vehicle functional safety and uses Automotive Safety Integrity Levels (ASIL). IEC 62304 is relevant to medical-device software, while IEC 60601 addresses important safety aspects of medical electrical equipment. These standards do not guarantee safety merely by applying a checklist; they require a defensible life-cycle process, evidence, traceability, verification, and management of systematic and random failures.

Safety and security increasingly overlap. A cyberattack that disables a protective controller is a safety event, and a safety diagnostic interface may become a security entry point. Security controls must not prevent emergency operation, while safety mechanisms must not be exploitable to create denial of service or unsafe actuation. Hazard analysis and threat modeling should therefore exchange assumptions and failure scenarios.

Hardware Security: Establishing Trust in Silicon

A hardware security architecture begins with assets, trust boundaries, adversaries, and attack surfaces. Assets may include keys, firmware, sensor data, design intellectual property, and control authority. Adversaries may have remote software access, physical possession, laboratory equipment, or insider privileges. Attack surfaces include debug ports, scan chains, boot ROM, firmware-update paths, memory buses, clock and power interfaces, and third-party intellectual-property blocks.

Secure boot establishes a chain of trust. An immutable root of trust verifies the signature or cryptographic hash of the next boot stage; that stage verifies the next component, and so forth. The device should reject unauthorized or rolled-back firmware and record an auditable measurement where required. Secure boot provides authenticity and integrity at startup, but it does not automatically protect runtime secrets or guarantee that authorized software is safe.

A hardware root of trust may contain a secure element, physically unclonable function (PUF), immutable key material, cryptographic accelerator, monotonic counter, and tamper-response logic. Keys should be generated, stored, used, rotated, and destroyed under a defined key-management policy. PUF outputs require error correction and helper-data protection; otherwise environmental variation can make key reconstruction unreliable or leak information.

Side-channel attacks exploit physical leakage rather than logical bugs. Power analysis, electromagnetic analysis, timing observation, and fault injection can reveal secret-dependent computation. Countermeasures include constant-time control flow, balanced or masked logic, randomized execution, noise generation, filtering, shielding, voltage and clock monitors, and careful physical layout. No countermeasure is universally effective: masking can be weakened by glitches and coupling, while balancing may be defeated by process variation.

A hardware Trojan is an intentionally inserted malicious modification that may remain dormant until a rare trigger and then alter functionality or leak information. Detection approaches include trusted design flows, split manufacturing, logic testing, formal property checking, side-channel comparison, destructive reverse engineering, and runtime monitoring. Each has limitations, especially when the Trojan is small, conditionally activated, or hidden in third-party intellectual property. Supply-chain assurance therefore requires provenance, signed design artifacts, access control, independent review, and reproducible evidence.

An Illustrative Case: Safety-Critical Motor Controller

Background

Consider an illustrative motor controller for an automated industrial vehicle. It contains a multicore processor, PWM power stages, SRAM, external sensors, a JTAG port, and a network interface. A failure that commands unintended torque can injure personnel or damage equipment.

Situation and Analysis

Initial functional simulation is successful, but production testing reveals intermittent failures at elevated temperature. Threat analysis also identifies an unlocked debug port and unsigned firmware-update capability. Reliability analysis predicts SRAM soft errors during high-radiation operation, while safety analysis identifies a single processor reset line as a common point of failure.

The engineering team separates concerns but coordinates their controls. Scan insertion and MBIST improve manufacturing coverage. At-speed patterns are added for clock and interconnect timing. ECC is added to SRAM, with a fault-tolerant error handler and periodic memory scrubbing. Independent watchdogs monitor execution, and a redundant reset architecture prevents one stuck reset signal from defeating both channels.

A secure boot ROM verifies signed firmware and checks a monotonic version counter. The JTAG port is disabled in deployed units unless an authenticated service procedure enables restricted access. Power and clock monitors detect fault-injection attempts and force a safe state. The safety architecture uses independent shutdown paths to remove gate-drive power, while the controller reports diagnostic information without allowing the diagnostic channel to override the shutdown.

Outcome

The resulting design does not claim that every fault is impossible. Instead, it demonstrates how faults are detected, how dangerous behavior is prevented, how test access is controlled, and how evidence supports the safety case. Manufacturing test, field diagnostics, security monitoring, and maintenance procedures are documented together so that a late change to one domain does not invalidate another.

Learning Points

The case illustrates four principles. First, coverage metrics are meaningful only when the fault universe and assumptions are stated. Second, redundancy must avoid common causes. Third, security controls can support safety but may also create availability risks. Fourth, production test, reliability qualification, safety analysis, and security assessment should be treated as connected engineering activities rather than isolated compliance tasks.

Engineering Trade-offs, Limitations and Common Mistakes

The most common mistake is treating a high structural test coverage number as proof of product quality. Coverage can be inflated by excluding hard faults or by ignoring power, delay, analog, and physical defects. A second mistake is applying an unrealistic fault model to a new process or architecture. Cell-aware, memory, timing, and transient faults may be necessary additions.

Another mistake is adding redundancy without analysing common-cause failures. Replicated logic sharing a clock tree, power supply, compiler defect, or voter is not fully independent. Similarly, a safety monitor that shares the same vulnerable sensor as the primary controller may provide only apparent protection.

Security weaknesses often arise from development convenience: permanent debug enablement, test keys shared across products, undocumented backdoors, predictable PUF helper data, unprotected scan compression, or firmware updates that authenticate identity but not version freshness. Security mechanisms must be evaluated against realistic attackers and maintained throughout the product life cycle.

Test insertion also affects power, timing, area, and yield. Shift patterns can produce thermal stress; BIST circuitry can consume valuable silicon; boundary scan increases pin-cell complexity; and aggressive compression can complicate diagnosis. Design reviews should use a multi-objective decision process rather than optimizing one metric.

Governance, Standards and Professional Practice

Trustworthy engineering requires traceability from requirement to implementation, test, and evidence. Safety requirements should be allocated to hardware, software, mechanical, and operational controls. Security requirements should identify assets, attack assumptions, and response obligations. Changes to scan logic, boot ROM, test keys, or diagnostic firmware should pass configuration control and regression analysis.

Standards provide terminology and process expectations, but certification is not equivalent to absolute security or safety. Independent assessment, supplier evaluation, vulnerability response, incident reporting, and secure end-of-life procedures remain necessary. Engineers have a professional obligation to report residual risk honestly, avoid unsupported coverage claims, protect confidential test data, and ensure that maintenance personnel cannot accidentally bypass safety or security controls.

Current Developments and Best Practices

Modern systems increasingly combine scan compression, embedded instruments, adaptive test, telemetry, and machine-learning-assisted diagnosis. Data collected from production and field returns can reveal spatial, lot, voltage, and temperature correlations. Such analytics must distinguish genuine degradation from changes in measurement conditions and should not replace physical failure analysis.

Chiplet integration and advanced packaging create new test boundaries and new security concerns. Die-to-die links require interconnect test, power integrity analysis, and provenance controls. Post-quantum migration is also influencing long-lived secure-boot architectures, particularly where firmware authenticity must remain trustworthy for decades.

Recommended practice is to:

• define fault, threat, and hazard assumptions before selecting mechanisms;
• combine structural, functional, timing, memory, and physical tests;
• report coverage together with exclusions, untestable items, and diagnostic quality;
• design scan, debug, boot, and update interfaces as security-sensitive assets;
• analyse common-cause failures and independence before claiming redundancy;
• use derating, thermal control, ECC, monitoring, and maintenance plans for ageing;
• validate accelerated tests against actual field mechanisms;
• connect hazard analysis, threat modeling, FMEA, FTA, and fault injection;
• preserve traceability and evidence throughout the product life cycle.

Chapter Summary

Testability, reliability, safety, and hardware security address different failure sources but must be engineered together. Fault models simplify physical defects into analyzable forms, while ATPG, scan, BIST, MBIST, and boundary scan improve controllability, observability, and production quality. Coverage metrics require explicit assumptions and should be complemented by diagnosis, timing, power, and physical validation.

Reliability analysis considers failure rates, ageing, temperature, soft errors, redundancy, and common-cause mechanisms. Functional safety uses hazard analysis, safety goals, diagnostics, safe states, and evidence-based life-cycle processes. Hardware security establishes trust through secure boot, roots of trust, protected debug, key management, side-channel resistance, and supply-chain assurance. The central engineering lesson is that no single mechanism is sufficient: dependable digital systems result from layered controls, explicit trade-offs, and continuous evidence across design, production, operation, and retirement.

Key Terms

Fault: A physical or logical cause of incorrect system behavior.

Error: An incorrect internal state produced by a fault.

Failure: A delivered service that violates its specification.

Fault model: An abstraction used to represent defects for analysis and test generation.

Stuck-at fault: A signal fault permanently fixed at logic 0 or logic 1.

Fault coverage: The proportion of modeled faults detected by a test set.

Controllability: The ease of forcing an internal node to a required value.

Observability: The ease of determining an internal state from accessible responses.

Scan design: A DFT method that converts storage elements into controllable shift chains.

ATPG: Automatic test-pattern generation for detecting specified faults.

BIST: Built-in self-test performed by circuitry embedded in the system.

Boundary scan: A test technique using cells at device pins and a serial test-access port.

Soft error: A transient error caused by a disturbance without permanent physical damage.

ECC: Error-correcting code used to detect and correct selected data errors.

Redundancy: Use of additional resources to detect, tolerate, or recover from faults.

Diagnostic coverage: The proportion of relevant faults detected by diagnostic mechanisms.

Functional safety: The ability of a system to avoid unreasonable risk through correct and fault-tolerant operation.

Secure boot: A boot process that verifies authenticity and integrity through a chain of trust.

Side channel: Physical leakage that reveals information about internal computation.

Hardware Trojan: A malicious hardware modification activated under specified conditions.

Review Questions

Short Answer Questions

1. Distinguish a fault, an error, and a failure.

2. What are controllability and observability in a digital circuit?

3. State the basic expression for fault coverage.

4. Why can triple modular redundancy fail despite having three replicas?

5. What is the purpose of a secure-boot chain of trust?

Long Answer Questions

1. Explain scan-based DFT, including its operating sequence, benefits, costs, and security implications.

2. Compare LBIST, MBIST, and boundary scan with respect to purpose, architecture, and limitations.

3. Discuss major CMOS ageing mechanisms and methods for improving long-term reliability.

4. Explain how hazard analysis, diagnostic coverage, redundancy, and safe-state design contribute to functional safety.

5. Describe a hardware-security architecture involving secure boot, key management, protected debug, and side-channel countermeasures.

Analytical / Application Questions

1. A production team reports 99% stuck-at fault coverage but experiences field failures caused by timing defects and power droop. Analyse why the metric was insufficient and propose an improved test strategy.

2. Design a fault-tolerant architecture for a motor shutdown function in which a single controller fault must not cause hazardous actuation. Identify possible common-cause failures.

3. A secure processor has an unlocked JTAG port, unsigned firmware updates, and ECC-protected memory. Evaluate its security and reliability posture and recommend a prioritized remediation plan.

22. Multiple Choice Questions with Answers

Question 1

A test pattern detects a stuck-at-0 fault on a node when it primarily performs which two actions?

A. It forces the node to 0 and masks all outputs

B. It activates the fault with a 1 and propagates the effect to an observable point

C. It disables the clock and resets the circuit

D. It measures only the circuit’s power consumption

Correct Answer: B. It activates the fault with a 1 and propagates the effect to an observable point

Question 2

Which limitation is most directly associated with response compaction using a MISR?

A. It eliminates the need for scan chains

B. It prevents all timing faults from occurring

C. Different faulty responses may produce the same final signature

D. It can operate only on analog circuits

Correct Answer: C. Different faulty responses may produce the same final signature

Question 3

Which technique is specifically designed to test memory-specific coupling and transition faults?

A. March-based MBIST

B. Boundary scan alone

C. Secure boot

D. Triple modular redundancy

Correct Answer: A. March-based MBIST

Question 4

A reported 99% fault coverage should be treated cautiously because it may not include

A. the number of package pins

B. unmodeled physical, timing, power, or security defects

C. the clock frequency used in functional operation

D. the silicon area of the processor

Correct Answer: B. unmodeled physical, timing, power, or security defects

Question 5

What is the main reliability concern when three redundant modules share one voter and one power supply?

A. They cannot perform logical operations

B. Their scan chains become inaccessible

C. A common-cause failure can defeat all nominal redundancy

D. Their failure rate is necessarily zero

Correct Answer: C. A common-cause failure can defeat all nominal redundancy

Question 6

Which statement best describes a soft error?

A. A permanent open circuit caused by electromigration

B. A transient disturbance that changes stored information without necessarily causing physical damage

C. A malicious firmware update signed by an attacker

D. A manufacturing defect that always remains observable

Correct Answer: B. A transient disturbance that changes stored information without necessarily causing physical damage

Question 7

In secure boot, the purpose of a chain of trust is to

A. increase the switching activity of the processor

B. verify each authorized software stage before execution

C. replace all hardware fault tests

D. make debug ports permanently accessible

Correct Answer: B. verify each authorized software stage before execution

Question 8

Which attack exploits power consumption or electromagnetic radiation to infer secret-dependent computation?

A. Side-channel attack

B. March test

C. Stuck-at fault

D. Boundary interconnect test

Correct Answer: A. Side-channel attack

Question 9

Why is an accelerated temperature test not automatically valid evidence of field reliability?

A. Temperature has no effect on digital hardware

B. The accelerated condition may activate a different failure mechanism

C. Failure rates cannot be measured experimentally

D. Only software can be tested at elevated temperature

Correct Answer: B. The accelerated condition may activate a different failure mechanism

Question 10

For a safety-critical controller with an unlocked JTAG port and unsigned firmware updates, the highest-priority concern is that

A. the circuit will necessarily have low scan coverage

B. an attacker may modify control behavior through maintenance or boot interfaces

C. ECC will correct too many memory errors

D. boundary scan will increase transistor threshold voltage

Correct Answer: B. an attacker may modify control behavior through maintenance or boot interfaces