Software Testing and Quality Assurance

Introduction

Software testing is the systematic evaluation of software to discover defects, provide information about quality, and reduce the risk that the system will fail in operation. Quality assurance (QA) is broader: it includes the planned processes, standards, reviews, measurements, and improvement activities used to build quality into the software development life cycle (SDLC). Testing is therefore an important quality-control activity within a wider quality-management approach.

Testing cannot prove that software contains no defects. Instead, it provides evidence about whether specified requirements have been satisfied and whether unacceptable risks remain. Effective testing begins early, is based on explicit objectives, and combines human judgment with appropriate tools and automation.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish quality assurance, quality control, verification, validation, and testing;
• plan static and dynamic testing at unit, integration, system, and acceptance levels;
• apply functional, regression, performance, usability, and security testing techniques;
• design useful test cases and select suitable test data;
• explain test automation, defect management, and test reporting;
• use basic quality models and testing metrics responsibly;
• evaluate the limitations, ethical issues, and professional implications of testing.

1. Quality in Software Development

Software quality has several dimensions. A system may be functionally correct but difficult to use, insecure, slow, or impossible to maintain. A commonly used quality model, reflected in standards such as ISO/IEC 25010, considers characteristics including functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, and portability.

Quality is not identical to conformance. Conformance asks whether the product meets documented requirements. Fitness for use asks whether it solves users’ real problems. For example, a payroll application may calculate salaries correctly but still be poor quality if employees cannot understand their payslips or if processing takes several hours.

Quality assurance and quality control

Quality assurance is process-oriented. It establishes development standards, reviews, audits, training, measurement, and preventive actions. Quality control is product-oriented. It identifies defects in work products through inspections and testing. Testing is primarily a quality-control activity, although test planning and process improvement contribute to QA.

• Verification: Are we building the product correctly?
• Validation: Are we building the correct product?
• Error: A human mistake, such as misunderstanding a requirement.
• Defect or fault: An imperfection in a work product caused by an error.
• Failure: Observable incorrect behaviour when a defect is executed.

A defect may exist without causing a failure in every execution. For example, an incorrect boundary condition may remain unnoticed until a user enters the maximum permitted value.

2. The Testing Process

A professional testing process normally includes:

1. Test planning: define scope, objectives, risks, resources, schedule, environments, and entry and exit criteria.
2. Test analysis: identify test conditions from requirements, architecture, interfaces, risks, and user workflows.
3. Test design: select techniques, create test cases, prepare data, and define expected results.
4. Implementation: organise test cases into suites, prepare scripts, configure tools, and establish traceability.
5. Execution: run tests, record evidence, compare actual with expected results, and report failures.
6. Defect management: classify, prioritise, assign, correct, retest, and close defects.
7. Reporting and closure: evaluate exit criteria, document residual risk, archive materials, and identify improvements.

Testing should be planned using risk. High-impact and high-likelihood risks receive greater attention. A safety-critical calculation, payment transaction, or authentication function normally deserves more rigorous testing than a low-risk visual preference.

Test levels

The levels are complementary. Passing unit tests does not guarantee that components communicate correctly, and passing system tests does not guarantee that users will accept the product.

3. Static Testing

Static testing examines software work products without executing the program. It can be applied to requirements, designs, source code, test cases, configuration files, and documentation.

Reviews

A review may be informal, a walkthrough, a technical review, or an inspection. An inspection is a structured examination using defined roles, preparation, checklists, recorded findings, and follow-up. Reviews can identify ambiguity, inconsistency, omission, duplication, security weaknesses, and violations of standards before they become expensive to correct.

For example, a requirement stating that a response must be “fast” is ambiguous. A review should request a measurable condition such as “95% of search requests shall return within two seconds under the stated workload.”

Static analysis

Static-analysis tools examine source code or other artefacts automatically. They may identify unreachable code, duplicated code, type errors, insecure API use, excessive complexity, or violations of coding standards. Results require professional interpretation because tools can produce false positives and may miss defects that depend on runtime data.

Static testing is valuable because it starts early, does not require a complete executable system, and often finds defects cheaply. It does not replace dynamic testing: a program may satisfy syntax and style rules while still producing an incorrect business result.

4. Dynamic Testing Techniques

Dynamic testing executes software and observes its behaviour. Test cases normally contain preconditions, inputs, steps, expected results, and postconditions. Good test cases are traceable to requirements and include both expected and unexpected conditions.

Black-box techniques

Black-box testing derives tests from externally visible behaviour without relying on internal code structure.

Equivalence partitioning divides inputs into groups expected to behave similarly. If an age field accepts integers from 18 through 65, useful partitions include below 18, 18–65, above 65, and invalid non-integer values. At least one representative from each relevant partition should be tested.

Boundary-value analysis focuses on values at and around limits, because defects frequently occur at boundaries. For the same field, values such as 17, 18, 19, 64, 65, and 66 are more informative than only testing 30.

Decision-table testing represents combinations of conditions and resulting actions. It is useful for business rules such as discounts, eligibility, or access control.

State-transition testing examines how a system responds to events in different states. An account might move from active to locked after several failed login attempts and return to active after an approved reset.

Use-case and scenario testing follows realistic user goals, including normal, alternative, and exception flows.

White-box techniques

White-box testing uses knowledge of internal structure. Statement coverage measures the proportion of executable statements run by tests:

Branch coverage measures the proportion of decision outcomes exercised:

The measures assume that the set of executable statements or branches has been identified consistently. Coverage is an indicator of test thoroughness, not proof of correctness. A test suite can achieve 100% statement coverage while failing to check whether outputs are correct.

Positive and negative testing

Positive testing checks valid inputs and expected use. Negative testing checks invalid, unexpected, malicious, or abnormal conditions. Both are necessary. A registration form should be tested with valid data, missing fields, excessive lengths, unexpected character sets, duplicate accounts, and requests made without authorisation.

5. Testing Across the SDLC

The traditional “shift-left” principle encourages testing activities early in development. Requirements can be reviewed before coding; designs can be analysed before implementation; unit tests can be written alongside code. In Agile and DevOps environments, testing is continuous and integrated with version control, build pipelines, code review, deployment, and monitoring.

Unit testing

Unit tests should be fast, repeatable, isolated, and deterministic. They often use stubs, mocks, or fakes to replace external dependencies. A unit test should normally test one behaviour and clearly identify the reason for failure. Excessive mocking can make tests reflect implementation details rather than useful behaviour.

Integration testing

Integration testing examines interactions among modules, services, databases, devices, or external systems. Strategies include big-bang integration, top-down, bottom-up, and incremental integration. Incremental approaches usually make fault localisation easier. Contract testing is particularly useful for service interfaces because it checks that a provider and consumer agree on request and response structures.

System testing

System testing evaluates the complete product in an environment resembling production. It includes functional workflows and non-functional characteristics such as performance, reliability, usability, compatibility, and security.

Acceptance testing

Acceptance testing determines whether the product is acceptable to its customer or intended users. User acceptance testing (UAT), operational acceptance testing, contractual acceptance testing, and regulatory acceptance testing may have different objectives. Acceptance criteria should be agreed before execution, not invented after results are known.

6. Regression, Performance, Usability, and Security Testing

Regression testing

Regression testing checks that changes have not introduced new defects or reactivated old ones. It may be triggered by a new feature, defect fix, library update, configuration change, or infrastructure migration. A regression suite should be prioritised rather than expanded without control. Tests covering critical business paths, changed components, and high-risk interfaces normally receive priority.

Performance testing

Performance testing evaluates responsiveness, throughput, resource use, scalability, and stability. Common forms include load testing under expected demand, stress testing beyond expected demand, spike testing for sudden changes, endurance testing over long periods, and volume testing with large data sets.

Important measurements include response time, throughput, error rate, CPU utilisation, memory use, and database performance. Averages alone can hide poor user experiences; percentiles such as the 95th percentile are often more informative. A requirement might state that 95% of checkout requests must complete within two seconds under 500 concurrent users.

Usability testing

Usability testing observes representative users attempting realistic tasks. It can reveal confusing terminology, inefficient workflows, inaccessible controls, and design assumptions. Ethical practice requires informed consent, protection of participant data, and avoidance of blaming users for design weaknesses. Accessibility testing should consider keyboard operation, screen readers, colour contrast, text alternatives, and appropriate standards such as WCAG.

Security testing

Security testing evaluates confidentiality, integrity, availability, authentication, authorisation, input handling, session management, logging, and resilience against common attacks. Activities may include threat modelling, code analysis, vulnerability scanning, penetration testing, and abuse-case testing. Testing must be authorised and controlled; probing a system without permission may be unlawful.

Security defects should be prioritised according to exploitability, affected assets, exposure, and business impact. Test data must not expose real personal or financial information unnecessarily.

7. Test Automation

Test automation uses tools to execute tests, compare results, generate reports, or support other testing activities. Automation is especially valuable for repeatable regression tests, API tests, unit tests, and data-driven checks.

A sustainable automation approach has the following characteristics:

• tests are version-controlled with the product;
• test data and environments are reproducible;
• failures provide clear diagnostic evidence;
• tests are independent where practical;
• unstable tests are investigated rather than ignored;
• automated checks supplement, rather than eliminate, exploratory testing.

The test pyramid suggests many fast unit tests, fewer integration tests, and a smaller number of slower end-to-end tests. A test trophy or other model may be appropriate for some applications, but no model is universal. The correct balance depends on architecture, risk, technology, and team capability.

Automation has costs. Scripts require maintenance when interfaces change, and a high number of brittle end-to-end tests can slow delivery. Automation should therefore be selected by return on investment and risk reduction, not by the number of scripts created.

8. Defect Management

A defect report should contain a concise title, environment, build or version, preconditions, reproduction steps, expected result, actual result, evidence, severity, and priority.

• Severity describes the impact of the defect, such as critical, major, or minor.
• Priority describes how urgently it should be addressed.

A cosmetic defect may have low severity and priority, while a rare but catastrophic data-loss defect may have high severity even if its priority requires investigation of operational conditions. Severity and priority should not be treated as synonyms.

A typical workflow is: new → triaged → assigned → fixed → ready for retest → verified or reopened → closed. A defect may be rejected, duplicated, deferred, or marked as not reproducible. Defect reports should be factual and respectful; they exist to improve the product, not to assign blame.

Useful metrics include defect ageing, reopened defects, escaped defects, and defect distribution by component. Metrics can be misused when teams optimise the number of defects closed instead of actual product quality.

9. Test Metrics and Quality Models

Metrics support decisions when their definitions and limitations are understood. Defect density can be expressed as:

The size unit may be thousands of lines of code, function points, or another defined measure. Comparisons are meaningful only when the measurement method, scope, and maturity are comparable.

Defect removal efficiency (DRE) estimates how effectively defects are found before release:

For example, if 90 defects are found before release and 10 after release, DRE is 90 ÷ (90 + 10) × 100 = 90%. This assumes defects are counted consistently and that the observation period for post-release defects is defined.

Traceability links requirements to design elements, code, test cases, and defects. A traceability matrix can show whether every critical requirement has tests and whether failed tests affect specific business commitments.

Quality models help teams discuss quality systematically, but they do not replace stakeholder judgment. A model should be adapted to the product’s context, risks, users, and legal obligations.

10. Illustrative Case: Online Clinic Booking System

Background

A clinic is developing an online booking system. Patients search for doctors, select available appointments, pay a fee, and receive confirmation. Clinic staff can reschedule or cancel appointments.

Problem

Early demonstrations appear successful, but pilot users report duplicate bookings, slow searches, confusing cancellation rules, and concern that one patient can view another patient’s appointment information.

Analysis

The failures indicate several interacting risks. Unit tests may have covered date calculations but not concurrent booking requests. Integration tests may not have checked transaction behaviour between the booking service and database. Usability testing was absent, and security testing did not verify object-level authorisation.

Approach

The team establishes acceptance criteria for search response time, booking uniqueness, cancellation deadlines, and access control. Static review identifies ambiguous cancellation requirements. Unit tests cover boundary dates and fee calculations. Integration tests submit simultaneous booking requests and verify database constraints. System tests cover the complete patient workflow. Security tests attempt to access another patient’s booking by changing an identifier in a request. Usability sessions involve patients with different levels of digital experience. Automated regression tests run in the build pipeline, while exploratory testing examines unusual journeys.

Defects are prioritised by patient safety, privacy, financial impact, and likelihood. The booking transaction is redesigned to enforce uniqueness at the database level, and authorisation checks are applied on the server rather than relying on interface controls.

Outcome

The revised system prevents duplicate bookings, rejects unauthorised record access, communicates cancellation rules clearly, and meets the agreed performance threshold in a representative load test. The team retains a risk-based regression suite and monitors production errors after release.

Learning points

Quality is multidimensional. Testing must cover interactions, boundaries, security, usability, and operational conditions. A successful demonstration is not evidence that a complete system is ready for production. Requirements, design, tests, and defects must remain traceable throughout change.

11. Challenges, Ethics, and Contemporary Practice

Testing is limited by time, incomplete requirements, unavailable environments, unrealistic data, and the impossibility of examining every input combination. A test environment that differs substantially from production can produce misleading confidence. Teams should document assumptions and residual risks rather than claiming absolute certainty.

Professional testers must protect confidential information, obtain permission for security tests, preserve evidence integrity, and report significant risks honestly. Accessibility and privacy are quality responsibilities, not optional enhancements. Test data should be minimised, anonymised, or synthetically generated when possible.

AI-assisted tools can generate test ideas, summarise failures, create synthetic data, detect code patterns, or prioritise regression tests. They can also generate incorrect tests, reproduce bias, expose confidential code, and create false confidence. Human review, secure tool configuration, traceability, and independent validation remain necessary. AI should assist professional judgment rather than replace accountability.

Contemporary teams increasingly practise continuous testing, infrastructure-as-code, service virtualisation, contract testing, observability-based production feedback, and chaos or resilience testing. These approaches are effective only when aligned with risk and supported by reliable environments and clear ownership.

Best-Practice Checklist

• Begin testing with requirements and risk analysis.
• Use both static and dynamic techniques.
• Test valid, invalid, boundary, and abuse conditions.
• Separate test environments and protect test data.
• Automate stable, repeatable checks with useful diagnostics.
• Keep tests traceable to requirements and risks.
• Distinguish severity from priority.
• Review coverage metrics without treating them as proof of quality.
• Include performance, usability, accessibility, and security early.
• Record residual risk and obtain informed acceptance before release.

Chapter Summary

Software testing supplies evidence about software behaviour, while quality assurance establishes the processes and controls that promote quality throughout the SDLC. Static testing reviews work products without execution; dynamic testing executes software and observes results. Unit, integration, system, and acceptance testing address different risks and must be coordinated.

Black-box techniques such as equivalence partitioning, boundary-value analysis, decision tables, state transitions, and use-case testing help design effective tests. White-box coverage measures provide useful indicators but cannot prove correctness. Regression, performance, usability, accessibility, and security testing address important non-functional risks. Automation is valuable when applied selectively and maintained responsibly. Defect management, traceability, metrics, and quality models support informed decisions, but professional judgment remains essential. Ethical testing requires authorisation, privacy protection, accessibility awareness, and honest communication of residual risk.

Key Terms

Quality assurance: Planned processes and controls used to build and improve quality.

Quality control: Activities that identify defects in products or work products.

Static testing: Examination of artefacts without executing the software.

Dynamic testing: Testing performed by executing software.

Verification: Checking whether the product is built according to specification.

Validation: Checking whether the product meets real user and business needs.

Unit testing: Testing an individual software component.

Integration testing: Testing interactions among components or systems.

System testing: Testing the complete integrated product.

Acceptance testing: Testing used to determine stakeholder acceptance.

Regression testing: Testing that changes have not damaged existing behaviour.

Equivalence partitioning: Dividing inputs into groups expected to behave similarly.

Boundary-value analysis: Testing values at and near input or output limits.

Test automation: Use of tools to execute or support repeatable testing activities.

Defect: An imperfection in a software work product.

Severity: The impact of a defect.

Priority: The urgency assigned to resolving a defect.

Traceability: The ability to relate requirements, implementation, tests, and defects.

Defect removal efficiency: A measure of defects removed before release relative to total observed defects.

Exploratory testing: Simultaneous learning, test design, and execution guided by professional investigation.

Review Questions

Short Answer Questions

1. Distinguish quality assurance from quality control.
2. What is the difference between verification and validation?
3. Why are boundary values important in test design?
4. State two limitations of code coverage metrics.
5. Distinguish defect severity from defect priority.

Long Answer Questions

1. Explain the testing process from planning through test closure.
2. Compare unit, integration, system, and acceptance testing, including their objectives and typical responsibilities.
3. Explain equivalence partitioning, boundary-value analysis, decision-table testing, and state-transition testing.
4. Discuss the benefits and limitations of test automation and explain the test-pyramid concept.
5. Explain how performance, usability, accessibility, and security testing contribute to software quality.

Analytical / Application Questions

1. An age field accepts integer values from 18 to 65 inclusive. Design equivalence classes and a boundary-value test set, and justify your choices.
2. A team reports 120 defects before release and 30 after release. Calculate defect removal efficiency and explain one assumption necessary for interpreting the result.
3. Design a risk-based test strategy for an online payment feature, covering static testing, dynamic testing, automation, defect management, security, and acceptance criteria.

Multiple Choice Questions with Answers

Question 1

Which activity is an example of static testing?

A. Executing a checkout transaction

B. Reviewing a requirements document for ambiguity

C. Measuring response time under load

D. Testing password reset in a browser

Correct Answer: B. Reviewing a requirements document for ambiguity

Question 2

What is the primary purpose of integration testing?

A. To test individual statements in isolation

B. To determine whether customers accept the product

C. To detect defects in interactions among components

D. To measure employee productivity

Correct Answer: C. To detect defects in interactions among components

Question 3

For an input valid from 18 through 65 inclusive, which value is a boundary test?

A. 30

B. 40

C. 50

D. 66

Correct Answer: D. 66

Question 4

Which statement about code coverage is correct?

A. 100% coverage proves that the software is correct

B. Coverage is irrelevant to test planning

C. Coverage indicates exercised code but does not prove correct results

D. Coverage can only be measured during acceptance testing

Correct Answer: C. Coverage indicates exercised code but does not prove correct results

Question 5

Which test type examines sudden increases in workload?

A. Spike testing

B. Usability testing

C. Static analysis

D. Compatibility testing

Correct Answer: A. Spike testing

Question 6

What does regression testing primarily seek to establish?

A. That no new requirements are needed

B. That changes have not damaged existing behaviour

C. That every user interface is attractive

D. That all defects have identical priority

Correct Answer: B. That changes have not damaged existing behaviour

Question 7

A defect’s severity describes its:

A. Urgency relative to a release date

B. Number of developers assigned to it

C. Impact on the system or users

D. Age in the defect database

Correct Answer: C. Impact on the system or users

Question 8

If 90 defects are removed before release and 10 are found after release, what is the defect removal efficiency?

A. 10%

B. 50%

C. 90%

D. 100%

Correct Answer: C. 90%

Question 9

Which practice is most appropriate for a security test?

A. Testing production systems without permission

B. Authorising and controlling the test before execution

C. Using real personal data whenever possible

D. Relying only on visual inspection

Correct Answer: B. Authorising and controlling the test before execution

Question 10

Which characteristic is most desirable in a unit test suite?

A. Slow and dependent on external services

B. Fast, isolated, repeatable, and deterministic

C. Executed only after production deployment

D. Focused exclusively on interface colours

Correct Answer: B. Fast, isolated, repeatable, and deterministic