Software Testing and Quality Control

Introduction

Software testing is a planned engineering activity used to evaluate whether a software product satisfies specified requirements, behaves reliably under defined conditions, and is suitable for use. Quality control is the operational part of quality management concerned with examining deliverables, identifying nonconformities, and taking corrective action. Testing is therefore not merely the final activity before release. It is a continuous control mechanism that supplies evidence for project decisions throughout the software life cycle.

Testing cannot prove that software contains no defects. It can demonstrate that selected behaviours have been examined under selected conditions and can reduce the probability that unacceptable failures remain undiscovered. Effective testing combines disciplined planning, appropriate test design, skilled observation, defect management, and communication with project stakeholders.

For a software project manager, testing must be coordinated with scope, schedule, cost, risk, configuration management, and release planning. A test activity that is technically sound but scheduled too late may delay delivery; a rapid release with inadequate testing may transfer hidden costs and risks to users.

Learning Objectives

By the end of this chapter, the learner should be able to:

1. Distinguish test levels, test types, and test techniques.
2. Explain how test planning and test design support quality control.
3. Describe the defect life cycle and classify defects by severity and priority.
4. Explain regression testing and its role in change management.
5. Describe user acceptance testing and its relationship to release decisions.
6. Coordinate testing activities with project milestones and delivery constraints.
7. Interpret basic testing measures without treating them as complete measures of quality.

1. Testing, Quality Assurance, and Quality Control

Quality assurance (QA) is primarily process-oriented. It establishes and improves the processes used to create software, including standards, reviews, training, audits, and process assessments. Quality control (QC) is primarily product-oriented. It examines work products and identifies whether they conform to requirements and agreed standards. Testing is one important QC technique, alongside inspections, walkthroughs, static analysis, demonstrations, and operational monitoring.

A useful distinction is:

• Verification: Are we building the product correctly according to specifications?
• Validation: Are we building the right product for users and business needs?

Testing contributes to both, although requirements reviews and user evaluation are especially important for validation. A product may conform perfectly to an incorrect or incomplete specification; testing against the specification alone will not expose every form of business failure.

Testing has several objectives:

• finding defects;
• providing information for release and risk decisions;
• demonstrating that specified behaviours have been evaluated;
• reducing the likelihood and impact of failures;
• assessing non-functional characteristics such as performance, security, and usability;
• supporting confidence among developers, customers, regulators, and operational teams.

Testing should be independent enough to provide credible challenge, but closely integrated with development and delivery. Independence does not mean isolation. Testers, developers, product owners, business representatives, and operations personnel must share information about risks and expected behaviour.

2. Test Levels

Test levels organize testing according to the scope of the item being examined. They are complementary rather than strictly sequential; modern iterative projects often perform several levels within each iteration.

2.1 Unit or Component Testing

Unit testing examines the smallest independently testable software components, such as functions, classes, services, or modules. Developers usually create and maintain these tests because they understand the internal design and can execute tests frequently.

Typical objectives include checking calculations, decision logic, error handling, input validation, and boundary behaviour. Unit tests should be fast, repeatable, isolated from external systems where practical, and easy to diagnose. Test doubles such as stubs, drivers, mocks, and fakes may represent unavailable or expensive dependencies.

Unit testing does not establish that components integrate correctly, satisfy user workflows, or perform acceptably in production. High unit-test coverage can coexist with serious defects in requirements, interfaces, configuration, and system behaviour.

2.2 Integration Testing

Integration testing examines interactions between components, services, databases, external systems, or infrastructure elements. It focuses on interfaces, data transfer, sequencing, protocols, error propagation, and dependency assumptions.

Integration may be performed incrementally as components are added, or across a larger subsystem. Incremental integration often makes failures easier to localize. Important integration risks include incompatible data formats, incorrect authentication assumptions, timing problems, transaction failures, and differences between development and production environments.

2.3 System Testing

System testing evaluates the complete integrated system against specified functional and non-functional requirements. It is normally performed in an environment that resembles the intended operational environment.

Functional system tests may evaluate end-to-end workflows, business rules, reports, search, notifications, and recovery. Non-functional system tests may assess performance, security, accessibility, compatibility, reliability, maintainability, and regulatory requirements. System testing should use realistic configurations and representative data, subject to privacy and security controls.

2.4 Acceptance Testing

Acceptance testing evaluates whether the system is acceptable to its customer, business owner, or other authorized users. It addresses fitness for intended use, contractual obligations, business processes, and operational readiness.

Acceptance testing may include user acceptance testing, contractual acceptance testing, regulatory acceptance testing, and operational acceptance testing. It is not simply a repetition of system testing. Business representatives judge whether the product supports real work and whether agreed acceptance criteria have been met.

3. Test Types and Quality Characteristics

A test type describes a purpose or quality characteristic being examined. The same test level may contain several test types.

Functional Testing

Functional testing examines what the system does. Test conditions are derived from requirements, use cases, user stories, business rules, interfaces, and error conditions. Both valid and invalid inputs should be considered.

Non-functional Testing

Non-functional testing examines how well the system operates. Important categories include:

• Performance testing: response time, throughput, resource use, and behaviour under workload.
• Load testing: behaviour under an expected workload.
• Stress testing: behaviour beyond normal capacity or under resource constraint.
• Security testing: authentication, authorization, confidentiality, integrity, vulnerability, and abuse resistance.
• Usability testing: effectiveness, efficiency, learnability, and user satisfaction.
• Accessibility testing: support for users with disabilities and conformance to applicable standards.
• Compatibility testing: operation across browsers, devices, operating systems, networks, and integrations.
• Reliability and recovery testing: behaviour after faults, interruptions, backup restoration, and service restart.
• Installation and deployment testing: correctness of installation, upgrade, rollback, configuration, and removal.

Static and Dynamic Testing

Dynamic testing executes software. Static testing examines artefacts without executing the program, including requirements reviews, code inspections, architecture reviews, and static analysis. Static techniques can identify ambiguity, unreachable code, interface inconsistencies, and standards violations early, when correction is generally less expensive.

Manual and Automated Testing

Manual testing is valuable for exploratory work, usability assessment, investigation, and situations where human judgment is central. Automated testing is valuable for repeatable checks, frequent regression suites, data-driven tests, and continuous integration.

Automation is not automatically superior. It requires design, maintenance, suitable tools, stable environments, and meaningful assertions. A large suite of fragile or poorly designed automated tests can create false confidence and maintenance cost.

4. Test Planning and Coordination

A test plan explains the testing approach and its relationship to project delivery. Depending on project scale, it may be a separate document or part of an overall quality plan. It should define:

• scope and objectives;
• items to be tested and excluded;
• test levels and types;
• responsibilities and independence arrangements;
• environments, tools, data, and access requirements;
• entry and exit criteria;
• risks, assumptions, dependencies, and constraints;
• defect recording and reporting procedures;
• deliverables and milestones;
• communication and escalation arrangements.

Entry and Exit Criteria

Entry criteria are conditions that should be satisfied before a test activity begins. Examples include an available build, approved requirements, a prepared environment, suitable test data, and a completed code review. Entry criteria prevent teams from wasting time testing an unstable or undefined item.

Exit criteria specify the evidence needed to conclude a test activity. Examples include completion of planned tests, resolution or formal acceptance of critical defects, achievement of required coverage, and delivery of a test summary report. Exit criteria should not be reduced to “all tests passed,” because the set of tests may be incomplete and some known defects may be accepted with explicit risk ownership.

Coordinating Testing with Delivery

Testing should be mapped to the delivery plan rather than placed in an isolated final phase. A practical sequence is:

1. Review requirements and acceptance criteria before development.
2. Prepare test conditions, data, environments, and automation during construction.
3. Execute unit and integration tests continuously.
4. Conduct system and non-functional testing on integrated increments.
5. Run regression testing after significant changes.
6. Conduct acceptance testing using realistic business scenarios.
7. Make a release decision using evidence, residual risk, and stakeholder approval.
8. Monitor production behaviour and feed lessons into subsequent planning.

A project manager should protect time for defect investigation and retesting. Compressing testing without revising scope or risk tolerance merely moves work and risk into later, more expensive stages.

5. Test Design Techniques

Test design converts requirements and risks into test cases. A test case normally identifies preconditions, inputs, actions, expected results, data, environment, and postconditions. Expected results should be stated before execution wherever possible to reduce confirmation bias.

Equivalence Partitioning

Inputs are divided into groups expected to behave similarly. One or more representative values are selected from each group. For an age field accepting whole numbers from 18 through 65, useful partitions include values below 18, values from 18 to 65, and values above 65. Additional partitions may be required for blank, non-numeric, decimal, or extremely large values.

Boundary Value Analysis

Defects frequently occur at the edges of permitted ranges. For the age example, tests might include 17, 18, 19, 64, 65, and 66. Boundary testing must reflect the actual rule, including whether limits are inclusive or exclusive.

Decision Tables

Decision tables represent combinations of conditions and expected actions. They are particularly useful for policies involving multiple rules, such as discounts, approval limits, or access permissions. Redundant or impossible combinations should be identified rather than blindly converted into tests.

State Transition Testing

State transition testing examines how events move an item from one state to another. An account might move from “pending” to “active,” “suspended,” or “closed.” Tests should include valid transitions, invalid events, repeated events, and recovery paths.

Use-Case and Scenario Testing

Scenario testing follows realistic user goals across an end-to-end workflow. It is useful for discovering integration and usability problems that isolated component tests may miss.

Exploratory Testing

Exploratory testing involves simultaneous learning, test design, and execution. The tester investigates areas suggested by risk, observation, and product knowledge. Exploratory sessions should still have a documented mission, time limit, evidence, and results. It is disciplined investigation, not unstructured clicking.

6. Defects and the Defect Life Cycle

A defect is a flaw or condition in a work product that may cause it to fail to meet a requirement or expectation. A failure is an observable incorrect behaviour when software executes. A fault is the underlying incorrect implementation or condition. These terms should not be treated as identical: a defect may exist without producing a failure under the tested conditions.

A typical defect life cycle is:

New → Triaged → Assigned → In Progress → Fixed → Ready for Retest → Verified and Closed

Alternative outcomes include Rejected when the report is invalid, Duplicate when another report already covers it, Deferred when correction is postponed, and Reopened when the fix does not work or the problem recurs.

A useful defect report includes:

• unique identifier;
• concise and precise summary;
• environment, build, and configuration;
• preconditions and test data;
• reproducible steps;
• expected and actual results;
• evidence such as logs or screenshots, handled securely;
• severity and suggested priority;
• reporter, owner, dates, and links to requirements or test cases.

Defect triage is the structured review of reports to confirm validity, remove duplicates, assign ownership, set priority, estimate effort, and decide whether work should be fixed, deferred, or rejected.

7. Severity and Priority

Severity describes the technical or business impact of a defect if it occurs. Priority describes the urgency with which the organization should address it. Severity and priority are related but not identical.

Priority may be urgent, high, normal, or low, depending on release commitments, user visibility, contractual obligations, and risk. A low-severity spelling error on a legal notice may have high priority. A technically severe defect in an unused experimental feature may be deferred, provided the decision is explicit and controlled.

Severity should be assessed using impact evidence. Priority is a management decision that also considers timing, cost, dependencies, and risk tolerance. Testers may recommend classifications, but authorized product or project owners should approve decisions that affect release risk.

8. Regression Testing and Change Control

Regression testing checks whether changes have introduced unintended effects in previously working functionality. Changes include bug fixes, new features, refactoring, configuration changes, infrastructure updates, database migrations, and third-party library updates.

Regression testing has several forms:

• Retest: checks whether a particular reported defect has been corrected.
• Regression test: checks whether other behaviour remains unaffected by the change.
• Smoke test: a small, broad check that a build is stable enough for deeper testing.
• Sanity test: a focused check of a changed area and closely related functions.

A regression suite should be risk-based. Tests covering critical workflows, shared services, complex interfaces, security controls, and historically defect-prone areas deserve high selection priority. Automation is especially valuable for stable, repeatable regression checks, but exploratory regression may reveal unexpected consequences that scripted tests do not anticipate.

Change control should link each change to affected requirements, code, configuration, tests, and defects. Continuous integration can run fast automated checks on each change, while broader suites may run on a schedule or before release. The aim is rapid feedback, not merely a high number of automated executions.

9. User Acceptance Testing

User acceptance testing (UAT) is a formal evaluation by representative users or business authorities to determine whether the product supports intended business operations and meets agreed acceptance criteria. The development team may support UAT, but the business should retain ownership of the acceptance decision.

Effective UAT requires:

• clear, testable acceptance criteria;
• identified and trained business participants;
• realistic but controlled data;
• a representative environment;
• defined responsibilities for recording and resolving issues;
• time for retesting and decision-making;
• documented approval or rejection.

UAT scenarios should reflect real tasks rather than internal technical structure. For example, a finance user may need to create an invoice, apply an approved adjustment, obtain authorization, issue the invoice, and reconcile the result. Each step may depend on roles, data, integrations, and policies.

UAT is not an opportunity to introduce unlimited new scope. New requirements discovered during UAT should enter change control and be assessed for impact. Acceptance does not imply that the software is defect-free; it indicates that authorized stakeholders accept the residual risk under defined conditions.

10. Measures and Quality Evidence

Metrics can support decisions, but no single metric represents software quality. Useful measures include:

• planned versus executed test cases;
• pass, fail, blocked, and not-run counts;
• defect counts by severity, component, and origin;
• defect age and time to resolution;
• requirements or risk coverage;
• regression failures after changes;
• escaped defects discovered after release;
• environment availability and test execution time.

A simple test execution rate can be calculated as:

The rate is meaningful only when the denominator is clearly defined and the tests are relevant. Executing 95% of a poorly designed test set does not establish high quality. Similarly, a low defect count may indicate quality, insufficient testing, under-reporting, or low product usage.

Coverage measures require careful interpretation. Requirements coverage asks whether requirements have associated tests; code coverage describes which code elements were executed. Neither proves correctness. Coverage should be combined with risk analysis, defect history, exploratory testing, and review evidence.

11. Illustrative Case: Municipal Permit Portal

Background

A municipality commissions an online portal for residents to submit building-permit applications, upload documents, pay fees, and track approval status. The planned release is tied to a public service deadline. The project uses two-week iterations and an external payment service.

Problem

During system testing, the team finds that payments succeed, but the application status is not updated when the payment provider sends a delayed callback. A separate test finds that users can upload a file type prohibited by municipal policy. The project sponsor asks whether the release can proceed because the main submission workflow appears successful.

Analysis

The delayed callback is an integration and reliability defect. It can cause duplicate payment attempts, incorrect status information, and manual reconciliation. Its severity is high because it affects a critical workflow and financial records. The prohibited upload is a security and compliance concern. Even if few users encounter it, its priority is high because it may violate policy and expose the municipality to risk.

The team performs boundary and negative testing on file size and type, state-transition testing on application statuses, and failure-injection testing for delayed and repeated payment callbacks. Review of the acceptance criteria reveals that “payment completed” was defined only as a successful response from the payment screen, not as confirmed synchronization with the portal.

Approach and Solution

The project manager coordinates the following actions:

1. The payment defect is assigned to the integration team and linked to a new acceptance criterion covering delayed, repeated, and failed callbacks.
2. The upload defect is blocked from release until server-side validation is implemented; client-side validation alone is rejected as insufficient.
3. Regression tests are added for payment, status updates, refunds, and reconciliation.
4. UAT participants receive realistic scenarios, including interrupted payment and resubmission.
5. The release plan is revised to include retesting and a controlled pilot.
6. A release decision is made using explicit residual-risk approval rather than informal optimism.

Outcome

The corrected system processes delayed callbacks idempotently, meaning repeated notifications do not create duplicate transactions. Invalid file types are rejected by the server, and the user receives a clear message. UAT confirms that staff can reconcile exceptional cases. The public release is delayed by several days, but the municipality avoids incorrect payment records and policy violations.

Learning Points

The case demonstrates that successful screen-level testing does not prove end-to-end correctness. Integration behaviour, negative conditions, security controls, and business acceptance criteria must be tested explicitly. It also shows why project delivery decisions should be based on risk evidence and authorized acceptance of residual risk, not solely on schedule pressure.

12. Challenges, Governance, and Contemporary Practice

Testing is affected by incomplete requirements, unstable environments, limited data, changing scope, distributed teams, third-party dependencies, and insufficient time. Test environments may not reproduce production conditions, while privacy rules may prevent the use of real customer data. Synthetic or masked data, access controls, audit trails, and secure disposal are therefore important.

Testing activities should respect confidentiality, least privilege, accessibility, and applicable legal or regulatory requirements. Security findings must be handled through controlled reporting rather than broad circulation. Testers should not alter production data without authorization. UAT participants should understand what evidence is collected and how it will be used.

Artificial intelligence can assist with test-case suggestion, code generation, log classification, data generation, and anomaly detection. However, AI-generated tests may reproduce biased assumptions, miss business context, or create insecure data. Human review, traceability to requirements, privacy controls, and validation of generated outputs remain necessary. AI should augment professional judgment rather than replace accountability for release decisions.

Contemporary delivery practices increasingly integrate testing into continuous integration and continuous delivery. Quality gates may stop a pipeline when critical tests fail, vulnerabilities exceed thresholds, or required approvals are absent. This approach is effective only when gates are meaningful, fast enough to support feedback, and maintained as the product changes. DevOps and agile methods do not remove the need for formal evidence; they distribute quality activities across the life cycle.

Best-Practice Principles

1. Start testing and quality analysis when requirements are created, not only after coding.
2. Derive test conditions from both requirements and risks.
3. Test positive, negative, boundary, integration, and recovery behaviour.
4. Keep test environments and data controlled, secure, and representative.
5. Record defects precisely and separate severity from priority.
6. Retest corrections and perform risk-based regression testing.
7. Automate repeatable checks, but retain exploratory and usability testing.
8. Define entry and exit criteria before execution.
9. Give business users genuine ownership of acceptance decisions.
10. Treat metrics as evidence for judgment, not as substitutes for judgment.
11. Make residual risk visible and obtain explicit authorization for accepted risk.
12. Learn from escaped defects and improve both product and process.

Chapter Summary

Software testing is an evidence-based activity that evaluates software against requirements, risks, and intended use. Quality control uses testing and other inspection techniques to detect nonconformities and support corrective action. Test levels range from unit and integration testing to system and acceptance testing, while test types address functional, performance, security, usability, accessibility, compatibility, and recovery concerns.

Effective test design uses techniques such as equivalence partitioning, boundary value analysis, decision tables, state transitions, scenarios, and exploratory testing. Defects should be documented, triaged, assigned, corrected, retested, and closed through a controlled life cycle. Severity describes impact, whereas priority describes urgency. Regression testing protects existing functionality after change, and UAT determines whether authorized business stakeholders accept the product for intended use.

Testing must be coordinated with delivery planning through entry and exit criteria, realistic environments, risk-based coverage, meaningful metrics, and explicit release decisions. No metric proves quality by itself, and acceptance does not mean that all defects have disappeared. Professional quality control makes residual risk visible and ensures that responsible stakeholders decide whether it is acceptable.

Key Terms

Acceptance testing: Testing performed to determine whether stakeholders accept a system for intended use.

Boundary value analysis: A technique that tests values at and around input limits.

Defect: A flaw in a work product that may cause failure or nonconformance.

Defect life cycle: The controlled progression of a defect from reporting through resolution and closure.

Dynamic testing: Testing that executes the software.

Entry criteria: Conditions required before a test activity begins.

Equivalence partitioning: Dividing inputs into groups expected to produce similar behaviour.

Exploratory testing: Simultaneous learning, test design, and execution guided by a test mission.

Integration testing: Testing interactions among components, systems, or external services.

Priority: The urgency assigned to correcting a defect.

Regression testing: Testing performed to detect unintended effects of changes.

Severity: The impact or seriousness of a defect.

Smoke testing: A brief, broad check that a build is suitable for further testing.

Static testing: Examination of work products without executing the software.

System testing: Testing of a complete integrated system.

Test case: Defined conditions, inputs, actions, and expected results used for testing.

Test level: A scope of testing, such as unit, integration, system, or acceptance testing.

User acceptance testing: Business-user evaluation of whether a product meets intended operational needs.

Review Questions

Short Answer Questions

1. What is the difference between quality assurance and quality control?
2. Name the four principal test levels and state the focus of each.
3. Why are severity and priority not interchangeable?
4. What is the purpose of regression testing?
5. Identify three items that should appear in a defect report.

Long Answer Questions

1. Explain how a project manager can coordinate testing with an iterative software delivery plan.
2. Compare functional testing, non-functional testing, static testing, and dynamic testing.
3. Explain equivalence partitioning, boundary value analysis, decision tables, and state transition testing, including an example of when each is useful.
4. Describe the defect life cycle from discovery to closure and discuss the role of triage.
5. Explain the purpose, responsibilities, preparation, and limitations of user acceptance testing.

Analytical / Application Questions

1. A payment application passes all unit tests but sometimes records a successful payment without updating the order. Identify the likely test level and test types required, and propose a regression strategy.
2. A low-severity spelling error appears on a legally required customer notice, while a high-severity defect affects an unused prototype feature. Recommend priorities and justify your reasoning.
3. A release has executed 180 of 200 planned test cases, but all 20 unexecuted cases concern security and data recovery. Calculate the test execution rate and explain why the rate alone is insufficient for a release decision.

Multiple Choice Questions with Answers

Question 1

Which test level primarily examines interactions between components and external services?

A. Unit testing

B. Integration testing

C. Acceptance testing

D. Exploratory testing

Correct Answer: B. Integration testing

Question 2

A defect prevents users from completing a critical business workflow and has no practical workaround. Which severity is most appropriate?

A. Low

B. Medium

C. High

D. Cosmetic

Correct Answer: C. High

Question 3

Which technique focuses on testing values at and around the limits of an input range?

A. Boundary value analysis

B. Decision table testing

C. Statement coverage

D. Load testing

Correct Answer: A. Boundary value analysis

Question 4

What is the main purpose of regression testing?

A. To confirm that a specific defect report has been logged

B. To determine whether changes caused unintended effects elsewhere

C. To replace all unit testing before release

D. To obtain business approval for a product

Correct Answer: B. To determine whether changes caused unintended effects elsewhere

Question 5

Who should normally own the final business acceptance decision in user acceptance testing?

A. The compiler

B. The business or customer representative

C. The database administrator alone

D. The automated test runner

Correct Answer: B. The business or customer representative

Question 6

A spelling error on a legally required notice is technically minor but must be corrected before release. Which classification best explains this situation?

A. Low severity and low priority

B. High severity and low priority

C. Low severity and high priority

D. Critical severity and no priority

Correct Answer: C. Low severity and high priority

Question 7

Which item is an example of an exit criterion?

A. The project has a proposed name

B. A test environment has been requested but not prepared

C. Critical defects are resolved or formally accepted by the authorized owner

D. A developer has started writing code

Correct Answer: C. Critical defects are resolved or formally accepted by the authorized owner

Question 8

A test plan includes 200 planned cases, of which 180 are executed. What is the test execution rate?

A. 10%

B. 20%

C. 80%

D. 90%

Correct Answer: D. 90%

Question 9

Which statement about test automation is most accurate?

A. Automation eliminates the need for exploratory testing

B. Automation is valuable for repeatable checks but requires maintenance

C. Automation proves that requirements are correct

D. Automation should be used only after release

Correct Answer: B. Automation is valuable for repeatable checks but requires maintenance

Question 10

Which statement correctly distinguishes verification from validation?

A. Verification asks whether the right product was built; validation asks whether code compiles

B. Verification concerns building the product correctly; validation concerns fitness for intended use

C. Verification is always dynamic; validation is always static

D. Verification applies only to acceptance testing

Correct Answer: B. Verification concerns building the product correctly; validation concerns fitness for intended use