Testing, Debugging, Build Systems, and Version Control

Introduction

Mechatronic software connects algorithms to sensors, actuators, communication networks, and physical mechanisms. A defect in a desktop application may produce an incorrect screen; a defect in motor-control software may damage equipment or injure a person. Reliable C++ development therefore requires more than writing code that appears to work. Engineers must construct evidence that software behaves correctly, diagnose failures systematically, reproduce builds, and preserve a traceable history of design decisions.

This chapter presents an integrated engineering workflow for C++ projects. It covers testing at several levels, test doubles and hardware-in-the-loop (HIL) testing, debugging and profiling, static analysis and sanitizers, CMake and package management, Git and code review, continuous integration (CI), and technical documentation. The emphasis is on practices suitable for embedded and robotic systems, where timing, resource limits, concurrency, and hardware dependence complicate conventional software testing.

Learning Objectives

By the end of this chapter, students should be able to:

• distinguish unit, integration, system, HIL, and regression testing;
• design useful tests using equivalence classes, boundary values, and observable behavior;
• use mocks, fakes, stubs, sanitizers, static analyzers, debuggers, and profilers appropriately;
• organize a portable C++ build with CMake and manage dependencies responsibly;
• use Git branches, commits, pull requests, tags, and code review effectively;
• design a CI pipeline that builds, tests, analyzes, and reports software quality;
• document interfaces, assumptions, limitations, and verification evidence;
• evaluate the safety, ethical, and operational implications of engineering decisions.

1. Quality as an Engineering Property

Software quality is multidimensional. Correctness means that specified behavior is produced for valid inputs and that invalid inputs are handled safely. Reliability concerns the probability of continued correct operation over time. Maintainability concerns the effort required to understand, modify, and verify the system. In mechatronics, real-time behavior, numerical stability, electromagnetic interference, sensor uncertainty, and actuator failure also matter.

Verification asks, “Did we build the product correctly according to its specification?” Validation asks, “Did we build the product that users and the physical system actually need?” Tests provide evidence for both questions, but no finite test suite proves that a program is universally correct. Testing should therefore be risk-based: functions affecting safety, motion limits, energy delivery, and fault handling deserve stronger evidence than experimental logging code.

A useful quality workflow is:

1. specify observable behavior and safety constraints;
2. implement a small change;
3. build reproducibly;
4. run fast automated tests;
5. inspect failures and diagnose causes;
6. perform broader integration and hardware tests;
7. review the change and record evidence;
8. preserve a versioned artifact.

2. Testing Levels and Test Design

2.1 Unit testing

A unit test checks a small unit in isolation, usually a function, class, or cohesive module. For example, a test may verify that a proportional controller computes the expected command for a known error, or that a ring buffer handles wraparound correctly. Unit tests should be fast, deterministic, repeatable, and independent of network services and physical hardware.

A well-designed unit test follows the Arrange–Act–Assert pattern:

• Arrange: construct inputs, dependencies, and initial state;
• Act: call the behavior under test;
• Assert: compare observable results with the requirement.

Tests should normally examine public behavior rather than private implementation details. If a test breaks whenever an internal data structure changes, it may be specifying the implementation rather than the contract.

2.2 Integration and system testing

Integration testing checks interactions between components: a controller with a sensor abstraction, a parser with a communication driver, or a planner with a collision map. These tests expose mismatched units, incorrect data formats, timing assumptions, and ownership errors that isolated unit tests cannot detect.

System testing evaluates the complete application against operational requirements. It may include startup, normal operation, communication loss, emergency stop, recovery, and shutdown. System tests are slower and more environment-dependent, so they should complement—not replace—fast unit tests.

2.3 Hardware-in-the-loop testing

In HIL testing, real control software interacts with a simulated plant or selected real hardware. A motor controller may receive simulated position feedback while its output is connected to a load emulator. HIL testing enables repeatable fault injection without exposing personnel or expensive machinery to unnecessary risk.

A HIL test plan should define:

• the simulated and real elements;
• sensor and actuator interfaces;
• sampling periods and acceptable latency;
• initial conditions and reset behavior;
• injected faults, such as stuck sensors or communication loss;
• pass/fail limits and data-recording requirements;
• independent safety interlocks.

HIL is not a substitute for final physical validation. A model can omit friction, backlash, thermal effects, quantization, or electromagnetic interference. Model assumptions must therefore be documented and, where possible, calibrated against measured data.

2.4 Regression testing

A regression test protects behavior that previously worked or prevents recurrence of a reported defect. Each significant defect should lead to a test that fails before the fix and passes afterward. Regression suites should be layered: smoke tests run on every change, broader tests run on merges, and expensive HIL or endurance tests run on scheduled or release pipelines.

2.5 Test selection

Three practical techniques are especially useful:

• Equivalence partitioning: divide inputs into classes expected to behave similarly, such as valid, invalid, empty, and malformed messages.
• Boundary-value analysis: test values at, just below, and just above limits. For a permitted speed interval [0, 2] m/s, test −0.01, 0, 0.01, 1.99, 2, and 2.01 m/s.
• Property-based thinking: state an invariant that should hold for many generated inputs, such as decoding an encoded message returning the original value, subject to the encoding contract.

A test oracle defines how correctness is judged. It may be an exact expected value, a tolerance interval, an invariant, a reference implementation, or a safety rule. Floating-point tests should use tolerances justified by sensor resolution and numerical error rather than arbitrary decimal equality.

3. Test Doubles and Testability

Hardware and external services are difficult to use in every unit test. A stub supplies predetermined responses; a mock verifies expected interactions; a fake provides a lightweight working implementation; and a spy records calls for later inspection. These terms are sometimes used differently by testing frameworks, so the project should define its terminology.

Dependency inversion improves testability. Instead of constructing a concrete serial driver inside a controller, the controller can depend on an interface such as ISensor. Production code supplies a real driver, while tests supply a fake sensor. This reduces coupling and permits deterministic fault scenarios.

Mocks should be used carefully. Verifying every call in exact sequence can make tests fragile and encourage implementation-specific design. Prefer checking meaningful outcomes: a safety relay is commanded open after an over-temperature event, or a retry stops after the specified limit. Time should also be abstracted when possible. A clock interface allows tests to advance time deterministically instead of waiting in real time.

4. Debugging, Static Analysis, and Runtime Tools

Debugging is a disciplined process, not random modification. A practical sequence is:

1. reproduce the failure reliably;
2. minimize the input or sequence that triggers it;
3. classify the symptom—wrong value, crash, hang, race, timing failure, or resource exhaustion;
4. form a falsifiable hypothesis;
5. collect evidence using logs, a debugger, or instrumentation;
6. apply the smallest justified correction;
7. add a regression test;
8. verify that the correction did not create a new failure.

A debugger permits breakpoints, stepping, stack inspection, watch expressions, and conditional stops. For concurrent programs, a failure may disappear when stepping because timing changes; logging and deterministic replay can then be more useful. Core dumps and symbol files are important for diagnosing failures outside the development workstation.

Static analysis examines source or an intermediate representation without executing the program. Compilers with high warning levels can detect narrowing conversions, unreachable code, suspicious signedness, and misuse of language constructs. Tools such as clang-tidy can identify modernization opportunities and common defects. Static analysis is not infallible: warnings require engineering judgment, and suppressions should be narrow, justified, and documented.

Sanitizers detect classes of runtime defects during instrumented execution:

• AddressSanitizer detects many out-of-bounds accesses, use-after-free errors, and double frees.
• UndefinedBehaviorSanitizer detects selected forms of undefined behavior, such as invalid shifts and signed overflow in applicable configurations.
• ThreadSanitizer detects many data races in supported environments.
• MemorySanitizer can detect some uses of uninitialized memory, although it has stricter platform requirements.

Sanitizers add overhead and may not support every embedded target. They should be used in host-based tests and compatible target builds, not treated as proof that unsanitized production execution is defect-free.

Profiling answers performance questions. Sampling profilers identify where execution spends time; instrumentation profilers measure selected events but may perturb timing. Measure before optimizing, define a representative workload, and distinguish CPU time, wall-clock time, memory allocation, latency, and deadline misses. In a real-time controller, average execution time may be less important than worst-case or high-percentile execution time.

5. C++ Build Systems and Dependency Management

A build system describes how source files become libraries, executables, and tests. It should encode targets, include directories, compiler features, link dependencies, generated files, and installation or packaging rules. CMake is widely used because it generates native build files for multiple platforms and supports target-based dependency modeling.

A minimal structure might be:

project/
  CMakeLists.txt
  include/robot/Controller.hpp
  src/Controller.cpp
  tests/controller_test.cpp
  app/main.cpp

A modern CMake file should prefer targets over global compiler flags:

cmake_minimum_required(VERSION 3. twenty)
project(robot_control LANGUAGES CXX)

add_library(robot_core src/Controller.cpp)
target_include_directories(robot_core PUBLIC include)
target_compile_features(robot_core PUBLIC cxx_std_20)

add_executable(robot_app app/main.cpp)
target_link_libraries(robot_app PRIVATE robot_core)

enable_testing()
add_executable(controller_test tests/controller_test.cpp)
target_link_libraries(controller_test PRIVATE robot_core)
add_test(NAME controller_test COMMAND controller_test)

In actual use, the version should be a valid CMake version such as 3.20; the example illustrates structure, not a universal minimum. Build directories should be separate from source directories, and generated files should not normally be committed.

CMake distinguishes configuration from compilation. A typical workflow is:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build --parallel
ctest --test-dir build --output-on-failure

For multi-configuration generators, configuration may be selected at build time, for example with --config Debug. Reproducibility requires recording compiler versions, language standard, options, dependency versions, target platform, and relevant environment variables.

Dependencies may be supplied by the operating system, CMake package configuration, or package managers such as Conan or vcpkg. Pin compatible versions, review licenses and security advisories, and avoid uncontrolled downloads during safety-critical builds. A lockfile or manifest improves repeatability. A dependency is part of the system’s attack surface and maintenance burden; “available” does not mean “approved.”

6. Git, Code Review, and Continuous Integration

Git records a directed history of snapshots. A commit should represent one coherent, buildable change where practical. Good commit messages state the purpose and, when useful, the reason for the change. Branches support parallel work, but long-lived branches increase merge risk. Small pull requests are easier to review and test.

A professional review examines:

• requirement coverage and behavior at boundaries;
• ownership, lifetime, exception, and concurrency risks;
• error handling and recovery behavior;
• test quality and missing cases;
• portability, performance, and security;
• documentation and backward compatibility.

Review is a technical control, not a popularity vote. Automated checks should provide objective evidence, while reviewers address design and context that tools cannot infer.

A CI pipeline commonly contains the following stages:

1. configure and build in a clean environment;
2. compile with warnings enabled;
3. run unit and integration tests;
4. run sanitizers and static analysis;
5. collect coverage and test artifacts where meaningful;
6. package a versioned artifact;
7. perform target or HIL tests when infrastructure is available.

Coverage is a diagnostic measure, not a quality score. High line coverage can coexist with weak assertions, while safety-critical branches may require explicit evidence even when they are difficult to execute. CI should fail for reproducible defects, publish logs, identify toolchain versions, and make failures accessible to developers.

7. Documentation and Evidence

Technical documentation should enable another engineer to build, use, test, and maintain the system. Important documents include:

• a README with prerequisites and quick-start instructions;
• architecture and interface descriptions;
• configuration and calibration records;
• test plans, procedures, results, and known limitations;
• safety assumptions and hazard controls;
• release notes and compatibility information;
• API documentation generated from source comments where appropriate.

Requirements should be traceable to implementation and verification evidence. For example, a requirement that a drive disables torque within a specified interval should identify the responsible software component, test method, measurement equipment, result, and software version. Records must distinguish simulated results from physical measurements.

8. Illustrative Case: A Robotic Conveyor Cell

Background

Consider an illustrative university-industry project: a C++ controller detects packages on a conveyor, estimates their position from an encoder, and commands a pneumatic diverter. The system includes an encoder driver, a vision-message parser, a scheduling algorithm, a fieldbus interface, and a safety relay.

Problem

During demonstrations, some packages were diverted one position late. The failure was intermittent and could not be reproduced reliably on the laboratory machine. A direct hardware-only debugging approach risked damaging packages and obscured the software cause.

Analysis

The team first recorded timestamps for encoder samples, message reception, scheduling, and actuator commands. They reproduced the issue with a deterministic fake clock and recorded message sequence. Unit tests showed that the scheduler was correct for individual messages. Integration testing revealed that a delayed vision message was accepted after its validity interval. The controller used message arrival time rather than the measurement timestamp.

Approach and solution

The team introduced a timestamped message interface and an explicit validity policy. A fake clock and fake fieldbus enabled unit tests for fresh, stale, duplicated, and out-of-order messages. A regression test reproduced the original sequence. Static analysis found an implicit narrowing conversion in the time-to-position calculation, and sanitizers found an out-of-bounds access in a malformed-message test path.

HIL testing then connected the production controller to a simulated conveyor model. The team injected communication delays, encoder dropouts, and invalid measurements. Finally, controlled physical tests verified calibration, timing, emergency stop behavior, and recovery. The build was defined in CMake, dependencies were pinned, and CI ran host tests on every pull request. HIL tests ran nightly because they required dedicated equipment.

Outcome

The corrected controller rejected stale measurements, and the regression suite prevented recurrence. The project also gained a documented timing budget, a reproducible build, review records, and a fault-injection test matrix. The result was not merely a corrected line of code; it was stronger evidence that the complete system could handle realistic failure modes.

Learning points

The case demonstrates that intermittent faults often require controlled time, recorded inputs, and tests at more than one level. Hardware testing is essential, but abstraction and simulation make many failures safer and faster to investigate. Documentation and CI convert individual debugging knowledge into organizational capability.

9. Limitations, Safety, and Contemporary Practice

No tool detects all defects. Unit tests may omit integration assumptions; mocks may model hardware incorrectly; sanitizers may be unavailable on the target; static analyzers may produce false positives; and CI environments may differ from deployment systems. Independent review, physical validation, and hazard analysis remain necessary.

Safety-related software should use a recognized development process appropriate to its domain and risk classification. Requirements, changes, test evidence, tool qualification considerations, access control, and release approval may be regulated. Credentials, proprietary designs, and personal data must not be exposed through logs or public repositories. Generated code and AI-assisted code require the same review, licensing checks, testing, and security analysis as human-written code. AI tools can suggest tests or explain diagnostics, but they do not establish correctness and may produce insecure or incompatible code.

Best practice is therefore layered assurance: clear requirements, simple interfaces, defensive programming, automated tests, independent analysis, controlled hardware experiments, code review, reproducible builds, and traceable releases. The appropriate level of rigor depends on consequence, complexity, uncertainty, and change frequency.

Chapter Summary

Reliable C++ mechatronic systems require an integrated lifecycle rather than isolated debugging. Unit tests provide fast feedback on local behavior; integration and system tests reveal interaction failures; HIL tests exercise realistic timing and fault scenarios; and regression tests preserve lessons from previous defects. Mocks, fakes, and dependency inversion make external behavior controllable without hiding important integration risks.

Debuggers, logs, static analyzers, sanitizers, and profilers answer different questions and must be used with an understanding of their limitations. CMake expresses portable target relationships, while package management and pinned dependencies improve reproducibility and security. Git provides traceable change history, code review supplies human design judgment, and CI automates repeatable evidence. Documentation links requirements, implementation, tests, risks, and releases. Together these practices support safer, more maintainable, and more employable engineering work.

Key Terms

Unit test: A test of a small isolated software unit.

Integration test: A test of interactions among components.

System test: A test of the complete application against operational requirements.

Hardware-in-the-loop: Testing real software with simulated or partially real physical elements.

Regression test: A test that prevents a previously corrected defect from returning.

Test oracle: The rule or reference used to determine whether behavior is correct.

Stub: A test replacement that supplies predetermined responses.

Mock: A test replacement that verifies expected interactions.

Fake: A lightweight working implementation used in tests.

Static analysis: Examination of code without executing it.

Sanitizer: Instrumented tooling that detects selected runtime defects.

Debugger: A tool for controlled execution and state inspection.

Profiler: A tool for measuring resource use and performance behavior.

CMake: A cross-platform build-system generator.

Package manager: A tool that obtains and manages external software dependencies.

Git: A distributed version-control system.

Continuous integration: Automated building and checking of proposed changes.

Code review: Structured inspection of a change by other engineers.

Traceability: The relationship among requirements, implementation, tests, and evidence.

HIL model fidelity: The degree to which a simulated plant represents relevant real behavior.

Review Questions

Short Answer Questions

1. What distinguishes verification from validation?
2. Why are unit tests normally kept independent of physical hardware?
3. Give one appropriate use for AddressSanitizer.
4. What information should be pinned or recorded for a reproducible build?
5. Why is code coverage not, by itself, a measure of software quality?

Long Answer Questions

1. Compare unit, integration, system, HIL, and regression testing, including the defects each can reveal.
2. Explain how dependency inversion, fakes, mocks, and a controllable clock improve C++ testability.
3. Describe a professional debugging workflow for an intermittent timing failure.
4. Design a CI pipeline for a mechatronic C++ project and justify its stages.
5. Discuss the role of documentation and traceability in safety-related software development.

Analytical / Application Questions

1. A motor controller accepts a speed command from 0 to 2 m/s. Design equivalence classes and boundary-value tests, including invalid inputs.
2. A HIL test passes, but the physical system fails under vibration. Analyze at least four model or process limitations that could explain the difference and propose follow-up tests.
3. A pull request changes a parser, adds no tests, updates a dependency without a lockfile, and passes compilation. Decide whether it should be approved and specify the evidence and changes you would require.

Multiple Choice Questions with Answers

Question 1

Which testing level primarily checks a small function or class in isolation?

A. Unit testing

B. System testing

C. Hardware-in-the-loop testing

D. Acceptance testing

Correct Answer: A. Unit testing

Question 2

What is the principal purpose of a regression test?

A. To measure processor frequency

B. To prevent a previously corrected defect from returning

C. To replace all integration tests

D. To generate production hardware

Correct Answer: B. To prevent a previously corrected defect from returning

Question 3

Which test double is generally intended to verify interactions with a dependency?

A. Stub

B. Mock

C. Compiler

D. Profiler

Correct Answer: B. Mock

Question 4

Which tool category can detect many use-after-free and out-of-bounds memory errors during execution?

A. AddressSanitizer

B. Linker

C. Package manager

D. Source formatter

Correct Answer: A. AddressSanitizer

Question 5

Why should CMake projects prefer target-based include directories and link dependencies?

A. They eliminate the need for source files

B. They make dependency relationships explicit and scoped

C. They prevent all runtime errors

D. They require only one operating system

Correct Answer: B. They make dependency relationships explicit and scoped

Question 6

What is a major purpose of continuous integration?

A. To automate repeatable build and verification checks for changes

B. To remove the need for code review

C. To guarantee mathematical proof of correctness

D. To replace version control

Correct Answer: A. To automate repeatable build and verification checks for changes

Question 7

Which practice best supports reproducible builds?

A. Downloading the newest dependency on every build

B. Recording toolchain settings and pinning dependency versions

C. Committing compiler-generated object files

D. Building only on one developer’s workstation

Correct Answer: B. Recording toolchain settings and pinning dependency versions

Question 8

In debugging, what should normally happen after a defect is corrected?

A. Delete the failing test

B. Add or retain a regression test and rerun relevant checks

C. Disable all warnings

D. Rewrite unrelated modules

Correct Answer: B. Add or retain a regression test and rerun relevant checks

Question 9

Why is HIL testing valuable for mechatronic systems?

A. It guarantees that the physical model is perfect

B. It permits controlled, repeatable testing of software and physical interactions

C. It makes safety interlocks unnecessary

D. It eliminates the need for physical validation

Correct Answer: B. It permits controlled, repeatable testing of software and physical interactions

Question 10

What does code coverage most appropriately provide?

A. A diagnostic indication of which code was exercised

B. A formal proof that requirements are satisfied

C. A replacement for test assertions

D. A measurement of electrical power consumption

Correct Answer: A. A diagnostic indication of which code was exercised