Testing, Debugging, and Quality Engineering

Introduction

A mobile application is not complete when it builds or appears to work on a developer’s device. It is complete when its important behaviours are verified under realistic inputs, devices, network conditions, permissions, lifecycle transitions, and accessibility requirements. Testing therefore forms part of software design rather than a final inspection activity.

React Native and Flutter add distinctive quality concerns. React Native applications combine JavaScript or TypeScript with native iOS and Android components; Flutter applications combine Dart code with a rendering framework and platform services. A test strategy must therefore cover application logic, rendered interfaces, navigation, asynchronous work, native integrations, and physical devices.

Quality engineering extends testing to the whole delivery system. It includes prevention through static analysis and code review, detection through automated tests and crash reporting, diagnosis through systematic debugging, and improvement through measurement and learning.

Learning Objectives

By the end of this chapter, students should be able to:

1. Distinguish unit, component or widget, integration, end-to-end, accessibility, and compatibility testing.
2. Design testable React Native components and Flutter widgets.
3. Use mocks, fakes, stubs, and fixtures appropriately.
4. Automate tests in a continuous integration pipeline.
5. Diagnose failures systematically and interpret crash reports.
6. Apply static analysis, code review, and risk-based test selection.
7. Evaluate limitations, accessibility, privacy, and ethical implications of quality practices.

1. A Model of Mobile Software Quality

A useful quality model considers both functional and non-functional properties. Functional correctness asks whether the application produces the required result. Non-functional quality asks whether it is usable, accessible, secure, reliable, maintainable, and compatible.

The test pyramid is a practical allocation model:

• Unit tests are numerous and fast. They test isolated functions or classes.
• Component/widget tests verify a screen or reusable UI element with controlled dependencies.
• Integration tests verify cooperation among modules, such as a repository, database, and state manager.
• End-to-end (E2E) tests exercise the compiled application through user-visible behaviour.

The pyramid is not a law. A hardware-heavy application may require more device and E2E tests, while a calculation-heavy application may benefit from extensive unit tests. Test selection should reflect risk, not an arbitrary percentage target.

A simple risk-prioritisation model is:

where R is risk priority, I is impact if a failure occurs, and L is likelihood of occurrence. The scales must be defined by the team—for example, 1–5—and the product is a prioritisation aid, not a probability. A payment defect may deserve more testing than a cosmetic spacing defect even if both have similar implementation effort.

2. Test Levels and Test Design

2.1 Unit testing

A unit test verifies a small unit in isolation. Typical units include validators, reducers, formatters, pricing rules, and authentication-token handlers. Good unit tests are deterministic, fast, and independent of network access, clocks, random generators, and device state.

Example TypeScript function:

export function finalTotal(subtotal: number, fee: number, discount: number): number {
  if (subtotal < 0 || fee < 0 || discount < 0) throw new Error('Invalid amount');
  return Math.max(0, subtotal + fee - discount);
}

The clamping rule is explicit: a discount cannot produce a negative payable total. For S=50, F=5, and D=70, the unclamped expression is 50+5−70=−15, so the final result is max(0,−15)=0. An alternative business rule could reject a discount greater than the subtotal plus fee; the specification must choose one rather than leaving the boundary implicit.

Jest tests might be:

describe('finalTotal', () => {
  test('adds fee and subtracts discount', () => {
    expect(finalTotal(50, 5, 10)).toBe(45);
  });
  test('clamps excessive discount at zero', () => {
    expect(finalTotal(50, 5, 70)).toBe(0);
  });
  test('rejects negative amounts', () => {
    expect(() => finalTotal(-1, 0, 0)).toThrow();
  });
});

Boundary-value analysis tests zero, the smallest valid value, the largest permitted value, and just-outside values. Equivalence partitioning divides inputs into classes expected to behave alike, such as valid email, malformed email, and empty email.

2.2 React Native component testing

A component test renders a component in a test renderer and observes behaviour through its public interface. It should test what a user can see and do, not private implementation details.

function LoginButton({onLogin, loading}: {onLogin: () => void; loading: boolean}) {
  return (
    <Pressable
      accessibilityRole="button"
      accessibilityLabel="Log in"
      disabled={loading}
      onPress={onLogin}>
      <Text>{loading ? 'Logging in…' : 'Log in'}</Text>
    </Pressable>
  );
}

Using React Native Testing Library:

it('calls login when pressed', () => {
  const onLogin = jest.fn();
  const {getByRole} = render(<LoginButton onLogin={onLogin} loading={false} />);
  fireEvent.press(getByRole('button', {name: 'Log in'}));
  expect(onLogin).toHaveBeenCalledTimes(1);
});

The test checks a role, accessible name, and user action. A separate test should verify that a loading button is disabled. Network requests should not occur in this component test; they belong behind a mocked service or in a higher-level test.

2.3 Flutter widget testing

Flutter’s widget test environment renders widgets without requiring a physical device. A Key, semantic label, or visible text can provide a stable test locator.

class SaveButton extends StatelessWidget {
  final VoidCallback onSave;
  final bool saving;
  const SaveButton({required this.onSave, required this.saving, super.key});

  @override
  Widget build(BuildContext context) => ElevatedButton(
    onPressed: saving ? null : onSave,
    child: Text(saving ? 'Saving…' : 'Save'),
  );
}

testWidgets('invokes save once', (tester) async {
  var calls = 0;
  await tester.pumpWidget(MaterialApp(
    home: SaveButton(onSave: () => calls++, saving: false),
  ));
  await tester.tap(find.widgetWithText(ElevatedButton, 'Save'));
  expect(calls, 1);
});

pump() advances the widget tree after a frame; pumpAndSettle() waits for scheduled animations and asynchronous frames to settle. The latter should be used carefully because an intentionally repeating animation may never settle.

2.4 Integration testing

An integration test verifies collaboration across boundaries. An example is submitting a login form, validating input, calling a repository, storing a token, and updating navigation. The test may use a local test server or a controlled fake repository. It should verify contracts such as request shape, error mapping, persistence, and state transitions.

Integration tests reveal defects that isolated tests miss: incorrect serialization, dependency injection mistakes, database migrations, and navigation configuration errors. They are slower and more environment-sensitive, so they should be fewer and purposeful.

2.5 End-to-end testing

E2E testing runs the packaged application through a user-visible workflow on an emulator, simulator, or real device. A representative React Native flow using Detox is conceptually:

it('allows a user to sign in', async () => {
  await element(by.id('email')).typeText('[email protected]');
  await element(by.id('password')).typeText('correct-password');
  await element(by.id('login-submit')).tap();
  await expect(element(by.text('Dashboard'))).toBeVisible();
});

A Flutter project may use integration_test and find.text('Dashboard'). E2E tests should use stable identifiers rather than screen coordinates. They need controlled accounts, predictable test data, cleanup, and a policy for network dependencies. A failed E2E test is evidence of a problem, but not necessarily proof that the product code is wrong; device setup, timing, or test data may be responsible.

3. Test Doubles and Determinism

A stub supplies predefined responses. A mock records interactions and can assert that they occurred. A fake is a lightweight working implementation, such as an in-memory repository. A spy wraps a real or fake object to observe calls. These terms are sometimes used differently across frameworks; the important distinction is purpose and behaviour.

Example repository abstraction:

interface UserRepository {
  signIn(email: string, password: string): Promise<{token: string}>;
}

A unit test can inject a fake:

const repo: UserRepository = {
  signIn: jest.fn().mockResolvedValue({token: 'test-token'})
};

Tests should not over-specify implementation. Assert that the correct user-visible result occurs and that an essential contract is honoured. Excessive interaction assertions make harmless refactoring appear to break behaviour.

Control nondeterminism by injecting a clock, random-number generator, UUID generator, and network client. Freeze or fake time when testing expiration. Reset mocks between tests. Do not use production secrets, real customer records, or uncontrolled third-party services in automated tests.

4. Accessibility and Device Compatibility

Accessibility testing asks whether people with different abilities can perceive, operate, and understand the application. Important practices include:

• provide meaningful labels and roles;
• preserve logical focus order;
• support dynamic text size without clipped content;
• maintain sufficient colour contrast and do not rely on colour alone;
• label form errors and announce important state changes;
• ensure touch targets are usable;
• test with VoiceOver on iOS and TalkBack on Android.

React Native uses accessibility properties such as accessibilityLabel, accessibilityRole, and accessibilityState. Flutter uses Semantics, semantic labels, and appropriate Material or Cupertino controls. Automated accessibility checks are valuable but cannot replace manual screen-reader testing.

Compatibility dimensions include operating-system version, screen size, pixel density, orientation, architecture, locale, time zone, font scale, dark mode, permissions, keyboard behaviour, and low-memory lifecycle events. A device matrix should be risk-based. For example, test the minimum supported OS, the most common device class, a small screen, a large screen, and at least one physical device for hardware-dependent behaviour.

5. Debugging as a Systematic Process

Debugging is hypothesis-driven fault localisation, not random code alteration. A disciplined process is:

1. Reproduce the failure and record exact steps, inputs, device, OS, build, and timing.
2. Reduce it to the smallest reliable reproduction.
3. Classify the failure: logic, state, rendering, platform, network, data, performance, or environment.
4. Form a falsifiable hypothesis.
5. Add focused logs, assertions, breakpoints, or traces.
6. Change one relevant factor at a time.
7. Confirm the fix and add a regression test.
8. Check neighbouring behaviour and document the root cause.

A stack trace identifies where an exception was observed, not always where the defect originated. Read the causal chain, inspect the first application frame, and correlate timestamps with network and navigation events. In asynchronous code, preserve error context and avoid swallowing rejected promises or exceptions.

Useful tools include React Native’s debugger and console, Flipper where supported, Android Studio Logcat, Xcode Instruments and console, Flutter DevTools, Dart analyzer, and platform profilers. Performance debugging should distinguish build time, layout time, JavaScript or Dart execution, GPU rendering, memory, and network latency.

6. Crash Reporting and Observability

Crash reporting systems collect uncaught exceptions, native crashes, breadcrumbs, device context, release versions, and sometimes performance signals. A good report includes a de-obfuscated stack trace, application version, operating-system version, device model, and a privacy-reviewed event trail.

Configure symbolication for native builds and source-map upload for JavaScript bundles where applicable. Tag releases so that a regression can be associated with a deployment. Track crash-free users or sessions, but interpret the denominator carefully: a small number of active users can make a percentage unstable, and non-fatal errors may be more informative than crashes.

Never log passwords, authentication tokens, precise location, health information, or unnecessary personal data. Apply retention limits, access control, consent requirements, and regional privacy obligations. Crash reporting is a diagnostic aid, not a substitute for tests: it observes failures after release, whereas tests seek evidence before release.

7. Static Analysis, Review, and Automation

Static analysis examines code without executing it. TypeScript type checking, ESLint, formatting tools, dependency audits, and React Native platform checks can detect defects early. Flutter teams commonly use dart analyze, dart format, lint rules, and package vulnerability checks. Static analysis cannot prove that a workflow is usable or that a server returns the intended business data.

A constructive code review examines:

• correctness and edge cases;
• state and lifecycle handling;
• accessibility semantics;
• error and loading states;
• security and privacy;
• test quality and maintainability;
• performance and compatibility;
• observability and rollback implications.

A CI pipeline commonly runs formatting verification, static analysis, unit tests, component/widget tests, integration tests, dependency checks, and selected device tests. Builds should fail for new critical defects, not merely produce a large undifferentiated list of warnings. Tests must be isolated and parallelised where safe, with artifacts such as logs, screenshots, and reports retained for diagnosis.

Coverage is useful as a map of untested code, not as a quality score by itself. Statement coverage can be high while important behaviours, failure paths, and accessibility properties remain untested. Mutation testing, in which small code changes are introduced to assess whether tests detect them, can reveal weak assertions.

8. Applied Case: Illustrative Campus Payments App

Background

Consider a fictional mobile application, CampusPay, allowing students to view a balance and pay fees through React Native. The payment screen includes a subtotal, service fee, discount code, and a final total.

Problem

A release candidate displays a negative amount when a large promotional discount is entered. Some users also report that the payment button cannot be reached with a screen reader. A crash report shows failures after the application resumes from the background during payment.

Analysis

The total rule was expressed without a boundary policy. The team specifies that the payable total must never be negative and adopts T=max(0,S+F−D), where S, F, and D are non-negative monetary amounts in the same currency and unit. For S=50, F=5, and D=70, T=0. Currency rounding is defined separately at the smallest supported currency unit; no mixing of cents and major currency units is permitted.

The accessibility defect is traced to a custom touchable lacking a role and label. The resume crash results from using a payment-session object after its native activity has been recreated.

Approach and solution

The team adds unit tests for valid, zero, excessive, and invalid amounts; a component test for the accessible payment button; an integration test for repository and session restoration; and an E2E test using a sandbox payment account. The payment session is represented by an explicit state machine—idle, authorising, completed, failed, and cancelled—and is revalidated after resume rather than assumed to remain valid.

A fake payment gateway is used for deterministic tests. A device test covers backgrounding during authorisation on both Android and iOS. Static analysis rejects nullable-session misuse, while crash reporting records release, state, and non-sensitive transaction identifiers.

Outcome

The negative-total defect is prevented by unit tests, the screen reader can identify and operate the control, and resume handling produces a recoverable error instead of a crash. The team also discovers that a successful sandbox response must not be treated as proof of production payment correctness; production verification remains a separate operational concern.

Learning points

The case demonstrates that quality is cross-layer: a business rule needs unit tests, a control needs accessibility semantics, lifecycle-sensitive behaviour needs integration and device tests, and released failures need privacy-conscious observability.

9. Limitations, Ethics, and Contemporary Practice

No test suite proves the absence of defects. Automated tests may encode incorrect requirements, omit rare device states, or become brittle. Flaky tests reduce trust and should be quarantined only temporarily while their causes are investigated. Contract tests, property-based testing, visual regression testing, and chaos experiments can supplement conventional tests when their cost is justified.

AI-assisted coding and test generation can suggest cases, but generated tests may repeat the implementation’s assumptions, contain insecure code, or assert superficial snapshots. Developers remain responsible for requirements, data protection, licensing, review, and validation. Synthetic or de-identified data should be used wherever possible.

Quality engineering is consequently a socio-technical discipline. It combines specifications, tools, human review, observability, and feedback from real users—including users with disabilities—while respecting privacy and fairness.

Chapter Summary

Mobile quality engineering uses layered evidence: unit tests for isolated rules, component or widget tests for interfaces, integration tests for collaboration, and E2E tests for realistic workflows. React Native Testing Library, Jest, Flutter widget tests, and integration frameworks provide practical mechanisms, but good test design depends on stable public behaviour, controlled dependencies, and deterministic data. Mocks, stubs, fakes, and spies serve different purposes and should not replace meaningful assertions.

Accessibility and device compatibility require explicit test dimensions and manual validation with assistive technologies. Systematic debugging proceeds from reproducible evidence to a falsifiable hypothesis and a regression test. Static analysis, code review, CI, crash reporting, and privacy-conscious observability prevent and diagnose defects across the delivery lifecycle. Coverage is evidence about tested paths, not a guarantee of quality. Risk-based selection, ethical data handling, and critical evaluation of AI-generated tests are essential professional practices.

Key Terms

Unit test: Test of a small isolated function or class.

Component test: Test of a React Native component through its rendered interface.

Widget test: Flutter test of a widget tree without a full device deployment.

Integration test: Test of cooperation among application modules or services.

End-to-end test: Test of a complete user workflow in a packaged application.

Test double: Substitute used to control or observe a dependency.

Stub: Test double returning predetermined responses.

Mock: Test double that records and verifies interactions.

Fake: Lightweight working implementation used in tests.

Fixture: Controlled data or setup used by a test.

Regression test: Test preventing a previously fixed defect from returning.

Flaky test: Test whose result changes without a relevant product change.

Accessibility: Usability of software by people with varied abilities.

Static analysis: Examination of code without executing it.

Symbolication: Mapping compiled addresses back to readable source locations.

Crash reporting: Collection and analysis of failure information from released applications.

Test pyramid: Model allocating many fast lower-level tests and fewer slower higher-level tests.

Test oracle: Rule or expected result used to determine correctness.

Observability: Ability to infer internal system behaviour from outputs and diagnostic signals.

Risk-based testing: Selecting test effort according to potential impact and likelihood.

Review Questions

Short Answer Questions

1. What is the principal difference between a unit test and an E2E test?
2. Distinguish a stub, mock, and fake.
3. Why should test locators avoid screen coordinates?
4. Name three accessibility properties that should be considered in a mobile interface.
5. Why is crash reporting not a substitute for pre-release testing?

Long Answer Questions

1. Explain the test pyramid and evaluate when a mobile project might deliberately depart from it.
2. Describe a systematic debugging process for an intermittent lifecycle crash.
3. Compare React Native component testing with Flutter widget testing, including suitable assertions.
4. Design a CI quality pipeline for a production mobile application and justify its stages.
5. Discuss the ethical and privacy risks of crash reporting and test data management.

Analytical / Application Questions

1. A checkout has a subtotal of 120 units, a fee of 8 units, and a discount of 150 units. Using the chapter’s clamping rule, calculate the payable total and state which unit tests should cover the boundary.
2. A test passes on an emulator but fails on a physical device after the app is backgrounded. Develop a fault-localisation plan covering evidence, hypotheses, tools, and regression testing.
3. Design a layered test strategy for a Flutter payment screen that uses a repository, local persistence, accessibility semantics, and a native payment SDK. Identify what should be real, fake, or device-tested at each layer.

Multiple Choice Questions with Answers

Question 1

For subtotal 50, fee 5, and discount 70, what is the final total under T = max(0, S + F − D)?

A. −15

B. 0

C. 15

D. 125

Correct Answer: B. 0

Question 2

Which test level is most appropriate for verifying a pure currency-calculation function in isolation?

A. Unit test

B. End-to-end test

C. Device compatibility test

D. Crash-report review

Correct Answer: A. Unit test

Question 3

In a React Native component test, which practice best reflects user-centred testing?

A. Inspecting a private component variable

B. Selecting a button by its screen coordinate

C. Finding a button by accessible role and name

D. Calling the component’s internal callback directly

Correct Answer: C. Finding a button by accessible role and name

Question 4

What is the primary purpose of a fake repository?

A. To provide a lightweight working dependency under controlled conditions

B. To compile native source code

C. To replace accessibility testing

D. To collect production crash reports

Correct Answer: A. To provide a lightweight working dependency under controlled conditions

Question 5

Which Flutter method advances the widget tree after scheduled frames and asynchronous work?

A. compile

B. pump

C. serialize

D. symbolicate

Correct Answer: B. pump

Question 6

Which activity is static analysis?

A. Running a user journey on a physical phone

B. Inspecting source with a type checker without executing the application

C. Interviewing users after release

D. Restoring a database backup

Correct Answer: B. Inspecting source with a type checker without executing the application

Question 7

Why should a device matrix include the minimum supported operating-system version?

A. It guarantees every possible device is covered

B. It tests an important compatibility boundary

C. It eliminates the need for unit tests

D. It makes network requests faster

Correct Answer: B. It tests an important compatibility boundary

Question 8

What should normally happen after a defect is fixed?

A. Delete all tests associated with the defect

B. Add a regression test that demonstrates the corrected behaviour

C. Disable crash reporting

D. Increase animation duration

Correct Answer: B. Add a regression test that demonstrates the corrected behaviour

Question 9

Which item should not ordinarily be written to crash-reporting logs?

A. Application release version

B. Operating-system version

C. Authentication password

D. De-obfuscated stack trace

Correct Answer: C. Authentication password

Question 10

What does risk-based testing primarily use to prioritise effort?

A. Potential impact and likelihood

B. File-name length and screen colour

C. Number of developers only

D. Random selection of test cases

Correct Answer: A. Potential impact and likelihood