Debugging, Testing, and Software Quality

Introduction

Software used in electrical engineering often controls instruments, processes measurements, communicates with embedded devices, or analyzes safety-critical data. A program can compile successfully and still produce incorrect results, fail under unusual inputs, or become impossible to maintain. Debugging and testing are therefore not optional final activities; they are engineering practices used throughout the software life cycle.

Debugging seeks to locate and correct the cause of an observed failure. Testing evaluates whether software behaves as specified and reveals defects. Software quality is broader: it includes correctness, reliability, usability, maintainability, efficiency, portability, security, and suitability for purpose. These activities are related but not identical. Testing may expose a defect, debugging explains it, and quality practices reduce the probability that defects are introduced or remain undetected.

Learning Objectives

After studying this chapter, you should be able to:

• distinguish errors, failures, faults, and defects;
• apply a systematic debugging process;
• use assertions appropriately;
• design unit, integration, system, and regression tests;
• apply equivalence partitioning and boundary-value analysis;
• automate tests and interpret test results;
• conduct effective code reviews and use static-analysis tools;
• use profiling to investigate performance problems;
• apply coding standards and write useful technical documentation;
• evaluate software quality in an engineering context.

1. A Model of Software Defects

Several terms are important. A mistake is a human action that introduces an incorrect requirement, design decision, or program statement. A fault (or bug) is an incorrect element in software. A failure is an observable deviation from required behavior during execution. A defect is often used as a general term for an imperfection in a work product.

The causal chain can be represented as:

mistake → fault → activation by an input or state → failure.

A fault may remain dormant for a long time. For example, an incorrect conversion from an unsigned sensor value to a signed integer may fail only when the measured value exceeds a threshold. A test that never exercises that range will not reveal the problem.

Quality has both functional and non-functional dimensions. Functional correctness asks whether a system produces the required outputs. Non-functional quality considers properties such as response time, memory use, robustness, readability, security, and ease of modification. In engineering, a numerically correct algorithm that misses a real-time deadline may still be unacceptable.

2. Systematic Debugging

Debugging should be evidence-driven rather than a sequence of guesses. A disciplined process is:

1. Reproduce the failure. Record the input, software version, hardware configuration, operating conditions, and exact observed behavior.
2. Reduce the case. Find the smallest input or sequence that still produces the failure.
3. Form hypotheses. Identify possible causes from the evidence.
4. Instrument and observe. Use logs, a debugger, assertions, traces, or temporary measurements.
5. Locate the fault. Distinguish the symptom from the statement or assumption that caused it.
6. Correct the cause. Prefer a principled correction over a local workaround.
7. Retest. Run the reproducing test and relevant existing tests.
8. Prevent recurrence. Add a regression test and document the cause when appropriate.

A useful debugging question is: “What observation would distinguish these two hypotheses?” For example, if a motor-control program sometimes commands excessive current, two hypotheses might be sensor scaling and stale control state. Logging the raw sensor value, converted value, and state-update time can discriminate between them.

Debugging tools

A debugger permits execution control through breakpoints, stepping, watch expressions, and call-stack inspection. Logging is valuable for intermittent or deployed failures, but logs should include timestamps, identifiers, units, and relevant state while avoiding sensitive information. Tracing records a sequence of events and is especially useful in concurrent or distributed systems.

The binary-search principle is often effective: divide the suspected execution path or data range into sections and determine which section contains the first incorrect state. Always seek the first point at which program state becomes invalid, not merely the final location where the failure becomes visible.

A small example

Suppose a voltage conversion is written as:

voltage = raw / 1023 * reference_voltage

In a language where raw and 1023 are integers, integer division may occur first, producing zero for most readings. The failure appears as an unexpectedly small voltage. A correction should make the intended numeric type explicit:

voltage = (real(raw) / 1023.0) * reference_voltage

The associated test should include zero, a mid-scale value, and the maximum representable reading. The underlying fault is not merely “a wrong result”; it is an unintended type and evaluation rule.

3. Assertions and Defensive Programming

An assertion is an executable statement of an assumption that should hold at a particular point in a program. For example:

assert sample_period > 0
assert buffer_length <= MAX_BUFFER_LENGTH

Assertions are appropriate for programmer assumptions, class invariants, and conditions that indicate an internal defect. They are not substitutes for handling invalid external input. User input, network data, and sensor measurements should be validated through ordinary error-handling logic because such values can legitimately be invalid or noisy.

Assertions should be:

• specific and close to the assumption they protect;
• inexpensive enough for the intended execution environment;
• independent of side effects;
• documented when the condition is non-obvious.

Do not write an assertion whose evaluation changes program state, such as assert(read_sensor()), because disabling assertions could change behavior. In deployed safety-related systems, assertion failure policy must be defined: halt, enter a safe state, reset, report, or continue with degraded functionality.

Defensive programming also includes range checks, explicit error codes or exceptions, safe resource release, timeouts, and validation of units and dimensions. These mechanisms improve fault containment but do not eliminate the need for testing.

4. Testing as an Engineering Activity

Testing compares actual behavior with expected behavior under selected conditions. A test has at least four elements: setup, stimulus, observation, and evaluation. A good test is repeatable, independent where possible, fast enough to run regularly, and diagnostic when it fails.

Test levels

Unit tests should focus on one behavior and minimize dependencies. Integration tests are necessary because individually correct modules can disagree about units, data formats, timing, ownership, or error conventions. Hardware-in-the-loop testing may be needed when physical timing, ADC behavior, actuators, or communication buses cannot be represented adequately in a software-only test.

Test oracles

A test oracle determines whether an outcome is correct. It may be an exact expected value, a reference implementation, a conservation law, a permitted range, or a relation between outputs. For a filter, an oracle might check that the output remains within a physical range and that a constant input produces the expected steady state. Oracle design is difficult when outputs are approximate or nondeterministic.

Test doubles

A stub provides controlled responses for a called component. A mock verifies expected interactions. A fake is a simplified working implementation, such as an in-memory data store. A simulator models relevant behavior of a physical system. Test doubles reduce cost and improve repeatability, but excessive mocking can test implementation details rather than useful behavior.

5. Test Design: Partitions and Boundaries

Exhaustive testing is usually impossible. Test selection should be systematic.

Equivalence partitioning divides inputs into classes expected to behave similarly. At least one representative is selected from each valid and invalid class. If a temperature input is valid from −40 °C through 125 °C, useful partitions include below range, valid range, and above range. More partitions may be necessary if the algorithm changes behavior inside the valid range.

Boundary-value analysis focuses on values at and near transitions, because faults frequently occur in comparisons, array indexing, rounding, and capacity limits. For an inclusive integer range [10, 20], relevant values include 9, 10, 11, 19, 20, and 21.

Other useful techniques include:

• decision-table testing for combinations of conditions;
• state-transition testing for protocols and controllers;
• property-based testing for general invariants over many generated inputs;
• fault-based testing in which likely faults are deliberately introduced;
• randomized testing, provided results are reproducible through recorded seeds.

A test case should state its purpose, preconditions, input, expected result, and cleanup. Tests should also specify units and tolerances. For a measured quantity, an exact floating-point comparison may be inappropriate. If the requirement permits absolute error no greater than ε, the test can verify that |actual − expected| ≤ ε, with ε defined by the measurement and numerical assumptions. The tolerance must not be chosen merely to make a failing test pass.

6. Automation and Continuous Testing

Automated tests can be executed consistently whenever code changes. A typical continuous-integration pipeline performs formatting checks, compilation, unit tests, integration tests, static analysis, and packaging. A failed stage should prevent unverified software from being promoted.

A useful test suite is fast, deterministic, isolated, and informative. Flaky tests—tests that pass and fail without a relevant code change—reduce trust. Their causes may include timing assumptions, shared state, uncontrolled randomness, environmental dependence, or inadequate cleanup. Flaky tests should be repaired or quarantined temporarily with explicit ownership; silently ignoring them is dangerous.

Coverage metrics can help identify untested code, but coverage is not proof of correctness. Statement coverage measures executed statements; branch coverage measures executed decision outcomes; condition coverage examines Boolean subexpressions. High coverage can still miss an incorrect expected result, an invalid requirement, or an important combination of states. Coverage should therefore guide test improvement rather than serve as the sole quality target.

7. Code Review and Static Analysis

Code review is a structured examination by one or more people other than the author. Reviewers should consider correctness, requirements, interfaces, error handling, readability, security, performance, test adequacy, and maintainability. Small changes are easier to review than large undifferentiated changes.

Effective review practice includes a clear change description, automated checks before human review, focused comments about the code rather than the person, and explicit resolution of defects. Reviewers should ask:

• Are units, ranges, and assumptions explicit?
• Are failure paths tested?
• Can integer overflow, division by zero, aliasing, or race conditions occur?
• Are resources released on every path?
• Does the implementation match the requirement rather than an unstated interpretation?

Static analysis examines source code or compiled representations without executing the program. Linters detect style and suspicious constructs. Type checkers identify incompatible operations. Data-flow analysis can reveal uninitialized values or unreachable code. Security analyzers search for vulnerable patterns. These tools are valuable early because defects are cheaper to correct before integration, but tools produce false positives and false negatives. Results require engineering judgment.

8. Profiling and Performance Diagnosis

Performance problems should be measured, not inferred from intuition. A profiler records where execution time, memory, I/O, or energy is being consumed. Sampling profilers periodically record the active location; instrumentation profilers record selected events but may add overhead.

A simple performance model is:

Here, T_total is total elapsed time, while the other terms represent approximate contributions from computation, memory activity, input/output, and synchronization. The model is descriptive rather than universally additive: overlap and measurement overhead may cause interactions. It is useful only when the terms and measurement method are defined consistently.

Profiling should use representative workloads. Optimize the measured bottleneck, preserve correctness, and rerun functional and performance tests after modification. In embedded systems, measure not only average execution time but also worst-case or high-percentile behavior when deadlines matter. An optimization that improves average time but creates occasional deadline misses may reduce system quality.

9. Coding Standards and Documentation

Coding standards reduce variation and make defects easier to see. They should define naming, formatting, module organization, comments, error handling, resource ownership, allowed language features, and review requirements. Standards should be enforceable automatically where practical.

Documentation should explain intent, interfaces, assumptions, units, valid ranges, failure modes, timing constraints, and examples. Comments should explain why a non-obvious decision exists, not restate the syntax. Outdated documentation is a quality defect; documentation must be maintained as part of the change.

For engineering software, documentation should also identify configuration, calibration constants, hardware dependencies, version information, and verification evidence. Traceability links requirements to design elements, implementation, and tests. This is particularly important where safety, compliance, or auditability is required.

10. Illustrative Case: Sensor-Based Overcurrent Protection

Background. An illustrative laboratory controller measures motor current and disconnects a load when current exceeds a specified limit. The controller occasionally trips during startup and, in one test, failed to trip under a sustained overload.

Problem analysis. The team first reproduced both behaviors and recorded raw ADC values, converted current, timestamps, motor state, and trip decisions. Boundary analysis showed that the limit comparison used > although the requirement specified trip at or above the limit. A separate review found that startup readings were interpreted before the sensor offset calibration had completed. Finally, an integration test revealed that the protection task ran every 20 ms, while the requirement allowed a maximum detection delay of 10 ms.

Approach. The team added unit tests for conversion at zero, nominal, maximum, and negative-offset values; boundary tests exactly at the trip limit; and state-transition tests for calibration, armed, tripped, and reset states. The comparison was corrected to >=. The controller was prevented from arming until calibration completed, and the protection task scheduling was redesigned and measured. Assertions checked that calibration constants were within configured ranges. A hardware-in-the-loop test injected controlled current waveforms.

Outcome. The startup false trips were eliminated, the exact-limit requirement was satisfied, and measured detection time met the stated constraint under the tested workload. The team also added a regression test for every discovered defect and documented the sensor units, calibration procedure, and timing assumptions.

Learning points. The case demonstrates that unit tests alone would not have exposed the scheduling and sensor-interface problems. Requirements must be precise about comparison operators, timing, units, and state. Observability and reproducible test conditions are essential for debugging physical systems.

11. Limitations, Ethics, and Contemporary Practice

No test suite proves that software is defect-free. Requirements may be incomplete, the environment may differ from the test environment, and untested interactions may exist. Simulation can omit physical effects; hardware testing can be expensive; formal methods can be powerful but require precise models and may not cover the entire system.

Engineers have an ethical responsibility to report known limitations, avoid suppressing inconvenient test results, protect test data, and not claim safety or reliability beyond the evidence. Quality records should be retained when required by organizational policy or regulation. Automated tools, including AI-assisted code generation and review, can accelerate development but may introduce insecure, inaccurate, or license-incompatible code. AI output must be reviewed, tested, traced to requirements, and evaluated for confidential-data exposure. Human accountability remains essential.

Contemporary practice increasingly combines pre-commit checks, continuous integration, reproducible builds, dependency scanning, software bills of materials, mutation testing, observability, and deployment monitoring. These practices complement—not replace—sound requirements, engineering judgment, and tests that reflect real operating conditions.

Chapter Summary

Debugging is a systematic process of reproducing, reducing, hypothesizing, observing, correcting, retesting, and preventing recurrence. Assertions expose violated internal assumptions, while defensive validation handles unreliable external inputs. Testing occurs at unit, integration, system, acceptance, and regression levels, and requires explicit oracles. Equivalence partitions and boundary analysis provide disciplined test selection when exhaustive testing is impossible. Automation supports repeatability and rapid feedback, but coverage is not proof of correctness. Code reviews and static analysis detect defects early; profiling identifies measured performance bottlenecks. Coding standards, documentation, traceability, and ethical reporting support long-term software quality. Engineering-quality software results from the coordinated use of all these practices.

Key Terms

Assertion: An executable check of an internal assumption.

Boundary-value analysis: Testing values at and near input or state boundaries.

Code review: Human examination of a change for correctness and quality.

Defect: An imperfection in a software or engineering work product.

Equivalence partitioning: Dividing inputs into classes expected to behave similarly.

Failure: Observable deviation from required behavior.

Fault: An incorrect element in software that may cause a failure.

Flaky test: A test whose result changes without a relevant code change.

Integration testing: Testing interactions among components.

Mock: A test double that verifies expected interactions.

Oracle: A mechanism for determining whether a result is correct.

Profiling: Measuring resource use and execution behavior.

Regression testing: Rechecking behavior after a change.

Static analysis: Examination of code without executing it.

Stub: A test double that supplies controlled responses.

System testing: Testing the complete integrated system.

Test coverage: A measure of which code elements or paths tests execute.

Test double: A substitute component used during testing.

Unit testing: Testing an individual function, class, or module.

Verification: Evaluation of whether implementation satisfies specified requirements.

Review Questions

Short Answer Questions

1. Distinguish a fault from a failure and give an engineering example.
2. Why should assertions not normally be used to validate user input?
3. What is the purpose of a test oracle?
4. Explain equivalence partitioning and boundary-value analysis.
5. Why does high code coverage not prove software correctness?

Long Answer Questions

1. Describe a complete, evidence-driven debugging process and explain the purpose of each stage.
2. Compare unit, integration, system, acceptance, and regression testing in an embedded engineering project.
3. Explain how code review, static analysis, coding standards, and documentation reinforce one another.
4. Discuss the value and limitations of test doubles, simulation, and hardware-in-the-loop testing.
5. Explain how profiling should be used to improve performance without compromising correctness or real-time behavior.

Analytical / Application Questions

1. A function accepts an integer speed command from 0 through 100 inclusive. Design an equivalence-partition and boundary-value test set, and justify each selected value.
2. A protection routine passes all unit tests but misses an overload when the sensor is recalibrated during operation. Identify likely integration faults and propose tests to isolate them.
3. A continuous-integration pipeline reports 95% branch coverage, yet field failures continue. Analyze at least four reasons this can occur and recommend a quality-improvement plan.

Multiple Choice Questions with Answers

Question 1

Which sequence best represents disciplined debugging?

A. Guess, patch, deploy, and document

B. Reproduce, reduce, hypothesize, observe, correct, and retest

C. Review, compile, format, and archive

D. Profile, optimize, delete tests, and release

Correct Answer: B. Reproduce, reduce, hypothesize, observe, correct, and retest

Question 2

Which condition is most appropriate for an assertion?

A. A user entering an invalid password

B. A network packet containing an unsupported request

C. An internal buffer invariant being violated

D. A sensor occasionally producing a value outside its physical range

Correct Answer: C. An internal buffer invariant being violated

Question 3

For an inclusive valid integer range from 10 through 20, which value is a boundary test just below the lower boundary?

A. 9

B. 10

C. 11

D. 15

Correct Answer: A. 9

Question 4

What is the primary purpose of integration testing?

A. To verify formatting rules only

B. To test interactions among components

C. To replace all unit tests

D. To measure source-code size

Correct Answer: B. To test interactions among components

Question 5

Which statement about test coverage is correct?

A. 100% coverage proves the requirements are correct

B. Coverage measures whether expected results are correct

C. Coverage can reveal untested areas but does not prove correctness

D. Branch coverage is identical to code review

Correct Answer: C. Coverage can reveal untested areas but does not prove correctness

Question 6

A test double that verifies whether a component received the expected call is usually called a:

A. Mock

B. Compiler

C. Profiler

D. Linker

Correct Answer: A. Mock

Question 7

What is the main purpose of profiling?

A. To prove that an algorithm is logically correct

B. To measure where resources are being consumed

C. To replace requirements analysis

D. To generate user documentation automatically

Correct Answer: B. To measure where resources are being consumed

Question 8

Which practice most directly reduces the recurrence of a discovered defect?

A. Removing the failing test

B. Adding a regression test for the defect

C. Disabling all assertions

D. Increasing code indentation

Correct Answer: B. Adding a regression test for the defect

Question 9

Why can integer division cause an incorrect sensor conversion?

A. It always creates a floating-point result

B. It may discard the fractional part before later multiplication

C. It changes the sensor's physical voltage

D. It prevents compilation in every language

Correct Answer: B. It may discard the fractional part before later multiplication

Question 10

Which activity is a form of static analysis?

A. Executing a motor under a simulated load

B. Stepping through a running program with a debugger

C. Scanning source code for unreachable code and suspicious constructs

D. Measuring current with an oscilloscope

Correct Answer: C. Scanning source code for unreachable code and suspicious constructs