Usability Testing & UX Optimization

Chapter 1
Foundations of Usability and UX Optimization

Certificate in UI/UX
Usability Testing & UX Optimization   •   Certificate

Foundations of Usability and UX Optimization

Introduction

Usability and user experience (UX) are central concerns in the design of websites, mobile applications, software, digital services, and interactive products. A product may contain sophisticated technology and attractive visual design yet still fail if users cannot understand it, operate it, or achieve their goals. Usability testing and UX optimization provide systematic ways to identify such problems and improve the quality of interaction.

This chapter introduces the conceptual foundations required for later study of usability evaluation and testing. It distinguishes usability from utility, user experience, accessibility, and optimization. It also explains the three central dimensions of usability identified in ISO 9241-11: effectiveness, efficiency, and satisfaction. These concepts are considered within a human-centred design process, in which design decisions are based on users, their goals, their environments, and evidence from evaluation.

Usability is not an optional feature added after development. It is a quality characteristic that influences adoption, productivity, safety, trust, inclusion, customer satisfaction, and organizational performance.

Learning Objectives

By the end of this chapter, you should be able to:

  • define usability, utility, user experience, accessibility, and UX optimization;
  • explain the ISO 9241-11 view of usability;
  • distinguish effectiveness, efficiency, and satisfaction;
  • describe the main principles of human-centred design;
  • identify how context of use affects usability;
  • select appropriate introductory usability measures;
  • recognize the relationship between usability and accessibility;
  • apply usability concepts to a practical design problem; and
  • identify limitations and ethical responsibilities in usability work.

1. What Is Usability?

Usability describes the extent to which specified users can use a product, system, or service to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. This definition is associated with ISO 9241-11 and is important because it makes usability conditional rather than absolute.

A design is not simply “usable” or “unusable” in every situation. Usability depends on:

  1. Users — their knowledge, abilities, language, expectations, and experience;
  2. Goals — what users are trying to accomplish;
  3. Product or system — the interface, content, functions, and feedback;
  4. Context of use — physical, social, organizational, technical, and environmental conditions; and
  5. Evaluation criteria — how success, effort, and user judgement are assessed.

For example, a professional audio-editing application may be highly usable for trained sound engineers but difficult for first-time users. A navigation system that works well in a quiet office may be less usable in a moving vehicle, where attention and screen time are restricted.

Usability therefore involves a relationship among people, technology, tasks, and circumstances. It cannot be judged reliably from visual appearance alone.

Usability as a Quality Attribute

Usability is often treated as a quality attribute alongside reliability, security, performance, maintainability, and compatibility. These attributes can conflict. Strong security controls may add interaction steps; extensive functionality may increase complexity; visual animation may affect performance or accessibility. UX professionals must therefore balance usability with other product requirements rather than optimizing one characteristic in isolation.

Usability and Usefulness

A product can be easy to use but have little value. A beautifully designed application that does not solve a meaningful problem has good interaction quality but poor usefulness. Conversely, a valuable service may be difficult to use, causing users to abandon it.

Usefulness is commonly understood as the combination of utility and usability:

  • Utility asks whether the system provides the functions users need.
  • Usability asks whether users can use those functions successfully and comfortably.

A tax platform may have the correct tax-calculation function, giving it utility, but users may still fail because the form structure is confusing. Both dimensions are necessary for a successful product.

2. Utility, User Experience, Accessibility, and Optimization

Utility

Utility is the extent to which a system provides appropriate functions for users’ goals. It concerns what the product enables users to do. A calendar application has utility if it allows users to create, modify, share, and receive reminders for events they need to manage.

Utility is investigated through requirements analysis, user research, task analysis, stakeholder consultation, and observation of real work. A usability test cannot compensate for missing functionality. If users need to export data but the system has no export capability, improving labels and layout will not solve the fundamental problem.

User Experience

User experience is the broader set of perceptions, responses, emotions, behaviours, and outcomes that arise before, during, and after interaction with a product, system, or service. It includes usability but extends beyond task performance.

UX may involve:

  • confidence and trust;
  • perceived value;
  • emotional response;
  • brand impressions;
  • accessibility and inclusion;
  • anticipation before use;
  • support and recovery after problems; and
  • the quality of the complete service journey.

Usability commonly focuses on interaction quality and task outcomes. UX also considers how the product feels, what users expect, and how the interaction fits into a wider experience. A banking application may be usable for transferring money but still produce a poor UX if fees are unclear, confirmation messages are alarming, or customer support is difficult to reach.

Accessibility

Accessibility is the degree to which products, services, environments, and information can be used by people with diverse abilities. It includes people with visual, auditory, motor, speech, cognitive, and neurological disabilities, as well as people affected temporarily by injury, illness, ageing, poor lighting, noise, or limited connectivity.

Accessibility is not identical to usability. Accessibility asks whether people with diverse abilities can access and operate the product at all; usability asks how effectively, efficiently, and satisfactorily specified users can achieve goals. The two concepts overlap and should be addressed together.

Important accessibility practices include:

  • providing meaningful text alternatives for informative images;
  • ensuring keyboard operation;
  • maintaining sufficient colour contrast;
  • using headings and semantic structure;
  • providing captions and transcripts;
  • avoiding reliance on colour alone;
  • supporting zoom and text resizing;
  • giving users enough time to read and respond; and
  • communicating errors in clear, actionable language.

Accessibility work should consider relevant standards and laws in the target market. The Web Content Accessibility Guidelines (WCAG) are widely used for web content, but compliance with a checklist does not guarantee a satisfactory experience. Testing with people with disabilities remains valuable.

UX Optimization

UX optimization is the systematic improvement of a product or service using evidence about user needs, behaviour, performance, and satisfaction. It is not merely visual polishing. Optimization may involve revising information architecture, simplifying a workflow, improving content, changing functionality, reducing errors, or redesigning an entire service.

Optimization should be goal-directed. A team may optimize for:

  • higher task completion;
  • fewer user errors;
  • shorter time to complete a task;
  • improved accessibility;
  • greater user confidence;
  • increased adoption or retention; or
  • reduced support requests.

Optimization without a clear objective can produce local improvements that damage the overall experience. For example, reducing the number of screens may shorten a process but increase cognitive load if each screen becomes overcrowded.

3. ISO 9241-11: Effectiveness, Efficiency, and Satisfaction

ISO 9241-11 provides a widely used conceptual framework for usability. It identifies three primary outcomes.

Effectiveness

Effectiveness is the accuracy and completeness with which users achieve specified goals. A user who enters the correct delivery address and successfully places an order has achieved the task effectively. A user who completes the process but enters an incorrect address has not achieved the goal completely.

Possible measures include:

  • task completion rate;
  • proportion of required subtasks completed;
  • number or severity of errors;
  • accuracy of entered information; and
  • successful completion without facilitator intervention.

Effectiveness must be defined in relation to the goal. Completing a form is not effective if the submitted information is wrong.

A simple completion-rate measure is:

Completion rate = (Number of successfully completed tasks ÷ Total number of attempted tasks) × 100%

This measure assumes that tasks have a clearly defined success criterion and that all attempts are classified consistently. It does not, by itself, indicate how difficult the task was or how much assistance users received.

Efficiency

Efficiency concerns the resources used in relation to the accuracy and completeness of goal achievement. Resources may include time, mental effort, physical effort, attention, or financial cost. In introductory usability testing, time on task is often used as a practical indicator, but time alone is not a complete measure.

A user who completes a task quickly by making several errors may be less efficient than a user who takes slightly longer and completes it accurately. Efficiency must therefore be interpreted alongside effectiveness.

A basic time-per-successful-task measure is:

Average successful-task time = Total time spent on successful attempts ÷ Number of successful attempts

The unit is usually seconds or minutes per successful task. The measure is meaningful only when task definitions, start and end points, and timing procedures are consistent.

Efficiency may also be compared across designs. For example, if the average successful-task time falls from 120 seconds to 90 seconds, the reduction is 30 seconds per successful task. However, a conclusion that the new design is better requires checking whether completion accuracy and satisfaction have remained acceptable.

Satisfaction

Satisfaction is the extent to which users’ physical, cognitive, and emotional responses meet their needs and expectations. It is subjective, but it can be studied systematically using interviews, questionnaires, rating scales, and observed behaviour.

Satisfaction may include perceptions of:

  • ease of use;
  • confidence;
  • comfort;
  • control;
  • trust;
  • attractiveness;
  • perceived effort; and
  • willingness to use the product again.

Satisfaction should not be confused with preference alone. Users may prefer a visually attractive design while still making more errors with it. Similarly, users may report satisfaction because they received assistance from a facilitator. Qualitative comments and performance evidence help interpret satisfaction data.

The Three Dimensions Together

Effectiveness, efficiency, and satisfaction are related but distinct. A product may be effective but inefficient, efficient but inaccurate, or efficient and effective but frustrating. Sound usability evaluation considers all three.

ISO 9241-11 usability dimensions Usability Effectiveness Accuracy and completeness Efficiency Resources required Satisfaction User responses and comfort All dimensions depend on users, goals, and context of use
Conceptual relationship among usability, effectiveness, efficiency, and satisfaction
Figure: Conceptual relationship among usability, effectiveness, efficiency, and satisfaction

Figure: The three principal usability outcomes described by ISO 9241-11.

4. Human-Centred Design

Human-centred design is an approach to interactive-system development that focuses on people, their needs, and the contexts in which they use products. It is iterative: teams investigate users and contexts, define requirements, produce design solutions, and evaluate those solutions. Findings are used to refine the design.

A typical cycle contains four connected activities:

  1. Understand and specify the context of use. Identify users, goals, tasks, environments, constraints, and stakeholders.
  2. Specify user requirements. Translate evidence into functional, content, accessibility, and experience requirements.
  3. Produce design solutions. Develop concepts, flows, wireframes, prototypes, content, and visual designs.
  4. Evaluate designs. Use inspections, usability tests, analytics, interviews, and other methods to identify problems and opportunities.

The cycle repeats because requirements and design assumptions become clearer through evaluation. Human-centred design does not mean accepting every user request without analysis. It means making design decisions with disciplined attention to human needs and evidence.

Context of Use

Context includes more than the screen. It may include:

  • physical location and lighting;
  • noise, movement, or interruptions;
  • device, network, and input method;
  • organizational policies;
  • social expectations and privacy;
  • user expertise and language;
  • frequency and urgency of tasks; and
  • consequences of error.

A hospital medication interface requires different safety considerations from a recreational game. A public kiosk must accommodate unfamiliar users, while an internal professional tool may support expert workflows but still needs clear error recovery.

Iteration and Prototyping

Early prototypes allow teams to test concepts before expensive development. Low-fidelity sketches can reveal navigation problems; interactive prototypes can reveal workflow problems; production systems can reveal performance, accessibility, and real-world adoption issues. Testing should occur throughout the product lifecycle rather than only at the end.

5. Usability Measures and Evidence

A usability measure is useful when it is connected to a defined question. Common introductory measures include:

DimensionExample measureTypical interpretation
EffectivenessCompletion rateWhether users achieve the goal
EffectivenessError count or severityWhere accuracy breaks down
EfficiencyTime on taskResources used during interaction
EfficiencyNumber of steps or actionsInteraction burden, when tasks are comparable
SatisfactionRating or questionnaire responseUser perception and confidence
AccessibilitySuccessful use with assistive technologyWhether barriers prevent access

Measurements require operational definitions. For example, “time on task” should specify when timing begins and ends. “Error” should distinguish a minor hesitation from an incorrect submission with serious consequences.

Quantitative results describe what happened; qualitative evidence helps explain why. A low completion rate may result from unclear terminology, missing information, technical failure, or a mismatch between the workflow and users’ mental models.

Baselines and Comparisons

A baseline is a reference condition against which a new design is compared. Comparisons should use equivalent tasks, comparable participants, and consistent procedures. Teams should avoid claiming improvement from small or uncontrolled differences. A faster completion time may be caused by participant learning, facilitator prompting, or technical performance rather than a better interface.

Qualitative and Quantitative Evidence

Quantitative evidence is useful for estimating frequency, comparing designs, and tracking targets. Qualitative evidence is useful for discovering unexpected problems, understanding reasoning, and identifying language or emotional issues. Good usability practice combines both when resources permit.

6. Applied Case: Improving a Community Clinic Booking Service

The following is an illustrative case.

Background

A community clinic introduced an online appointment service. The service allowed patients to choose a department, select a clinician, choose a time, and provide contact information. Analytics showed that many users began booking but did not complete the process.

Problem

The clinic initially assumed that users abandoned the service because they disliked online booking. A preliminary usability study with six representative participants revealed a different pattern. Users could find the service, but several struggled to distinguish “appointment type” from “clinic department.” Some available times disappeared after users entered personal information. One participant using a screen reader could not determine which fields were required.

Analysis

The team classified findings according to usability dimensions:

  • Effectiveness: Some users selected the wrong appointment type or failed to complete booking.
  • Efficiency: Users repeatedly moved between screens to compare times.
  • Satisfaction: Participants expressed uncertainty about whether their request had been submitted.
  • Accessibility: Required-field information and status feedback were not communicated adequately to assistive technology.

The team also examined utility. The service provided the necessary booking function, so its basic utility was present. The principal problems concerned information structure, feedback, and accessibility.

Approach and Solution

The team:

  1. replaced internal clinical terminology with patient-centred labels;
  2. displayed appointment type explanations beside the choices;
  3. showed available times before collecting nonessential personal information;
  4. preserved selected information when users moved backwards;
  5. marked required fields programmatically and visually;
  6. added a clear confirmation page and confirmation message; and
  7. tested the revised prototype with keyboard users and a screen-reader user.

Outcome

In a follow-up evaluation, more participants completed the booking task without assistance, and participants reported greater confidence. The team did not treat this result as final proof of success. It planned to monitor live completion, cancellation, support requests, and accessibility feedback after launch.

Learning Points

The case illustrates several principles:

  • abandonment does not automatically indicate a lack of interest;
  • utility and usability must be investigated separately;
  • effectiveness, efficiency, and satisfaction identify different problems;
  • accessibility should be included from the beginning; and
  • optimization is an iterative process requiring continued evidence.

7. Challenges, Limitations, and Ethical Responsibilities

Limitations of Usability Testing

A usability test is a sample of behaviour under particular conditions. It cannot guarantee that every user will succeed in every context. Participants may behave differently in a laboratory than in their normal environment. Small studies are often effective for discovering problems but are not necessarily suitable for estimating population-wide performance.

Usability findings may also be affected by:

  • participant recruitment bias;
  • unrealistic tasks;
  • facilitator influence;
  • learning and order effects;
  • technical instability;
  • insufficient accessibility coverage; and
  • overreliance on self-reported satisfaction.

These limitations do not make testing useless. They require careful interpretation and, where appropriate, complementary methods such as field observation, analytics, support-data analysis, and accessibility evaluation.

Ethical Practice

Usability professionals have responsibilities to participants. Studies should provide understandable information about the activity, obtain informed consent, protect privacy, and allow participants to withdraw. Researchers should avoid blaming users for design problems and should be especially careful when recruiting children, people with disabilities, patients, employees, or other groups with limited power.

Recorded screen activity, audio, video, and interaction logs may contain personal or sensitive information. Data should be collected only for legitimate purposes, minimized, secured, and retained according to applicable requirements. Reports should avoid unnecessary identification of individuals.

Avoiding Dark Patterns

Optimization can be misused to manipulate users. Dark patterns include confusing cancellation, hidden costs, disguised advertisements, coercive consent, and interfaces that make refusal harder than acceptance. A rise in conversion is not a positive UX outcome if it results from deception or undermines informed choice.

Ethical optimization respects autonomy, transparency, privacy, accessibility, and safety. Teams should define success measures that include user welfare rather than relying only on business metrics.

8. Contemporary Practice and Responsible Use of Technology

Modern UX teams use analytics, session recordings, remote testing, automated accessibility checks, and artificial intelligence. These tools can increase the volume and speed of evidence, but they do not replace human judgement.

Automated tools can identify missing labels, contrast problems, or unusual interaction patterns. They may not determine whether language is respectful, whether a workflow supports a real-world goal, or whether a person with a disability can complete a meaningful task. AI-generated design suggestions may reproduce bias, overlook minority users, or optimize for engagement instead of user wellbeing.

Responsible practice requires:

  • validating automated findings with human review;
  • protecting personal data in recordings and datasets;
  • checking outputs for bias and exclusion;
  • testing with diverse participants; and
  • documenting assumptions, limitations, and decisions.

The enduring principle is that evidence should inform design, while professional judgement and ethical responsibility remain essential.

Best-Practice Checklist

Before evaluating or optimizing an interface, confirm that you can answer the following questions:

  • Who are the users?
  • What goals are they trying to achieve?
  • What is the context of use?
  • What functions are necessary, and therefore what is the product’s utility?
  • How will effectiveness, efficiency, and satisfaction be defined?
  • Which accessibility needs must be included?
  • What evidence supports the current design assumptions?
  • What task or service outcome will indicate improvement?
  • How will privacy, consent, and participant welfare be protected?
  • How will the design be monitored after release?

Chapter Summary

Usability is the extent to which specified users achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. It is a relationship among users, goals, technology, tasks, and circumstances rather than a permanent property of an interface.

Utility concerns whether a product provides the functions users need, while usability concerns how successfully and comfortably they can use those functions. User experience is broader than usability and includes perceptions, emotions, trust, value, and the complete service journey. Accessibility focuses on enabling people with diverse abilities to access and operate products; it overlaps with usability and must be addressed throughout design.

ISO 9241-11 identifies effectiveness, efficiency, and satisfaction as central usability outcomes. Human-centred design uses an iterative cycle of understanding context, specifying requirements, producing solutions, and evaluating them. Reliable UX optimization combines performance evidence, user feedback, accessibility consideration, and ethical judgement. Usability testing is powerful but has limitations, so findings should be interpreted in context and supported by appropriate complementary evidence.

Key Terms

Usability: The extent to which specified users achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.

Utility: The extent to which a system provides functions needed to achieve users’ goals.

User experience: Users’ perceptions, responses, emotions, behaviours, and outcomes before, during, and after interaction with a product or service.

Accessibility: The degree to which people with diverse abilities can access and use a product, service, or environment.

UX optimization: Systematic improvement of user experience using evidence about needs, behaviour, performance, and satisfaction.

Effectiveness: The accuracy and completeness with which users achieve goals.

Efficiency: The resources required in relation to accurate and complete goal achievement.

Satisfaction: Users’ physical, cognitive, and emotional responses to the interaction.

Context of use: The users, goals, tasks, environments, technologies, and circumstances surrounding use.

Human-centred design: An iterative design approach focused on users, their needs, and their contexts.

Usability test: An evaluation in which representative users attempt defined tasks while evidence is collected.

Task completion rate: The proportion of attempted tasks completed successfully.

Error: An action or outcome that prevents or threatens accurate task completion.

Prototype: A representation of a product or interaction used for exploration and evaluation.

Baseline: A reference condition used for comparison.

Mental model: A user’s internal understanding of how a system works.

Assistive technology: Hardware or software that supports access for people with disabilities.

Dark pattern: An interface practice that manipulates or deceives users into actions they may not freely choose.

Human-centred evaluation: Assessment grounded in users’ goals, contexts, experiences, and evidence.

Review Questions

Short Answer Questions

  1. Define usability using the ISO 9241-11 perspective.
  2. What is the difference between utility and usability?
  3. Name and briefly explain the three principal dimensions of usability.
  4. Why is accessibility related to, but not identical to, usability?
  5. What is meant by context of use?

Long Answer Questions

  1. Explain how effectiveness, efficiency, and satisfaction should be considered together in usability evaluation.
  2. Describe the main activities in a human-centred design cycle.
  3. Compare usability, user experience, utility, and accessibility.
  4. Discuss the strengths and limitations of quantitative and qualitative usability evidence.
  5. Explain the ethical responsibilities involved in usability testing and UX optimization.

Analytical / Application Questions

  1. A mobile ticketing application allows users to purchase tickets, but users frequently select the wrong journey and abandon payment. Analyse the problem using utility, effectiveness, efficiency, and satisfaction.
  2. Design a short evaluation plan for an online public-service form used by people with different abilities and levels of digital experience. Identify users, tasks, measures, and ethical safeguards.
  3. A redesign reduces average task time but increases errors and decreases user confidence. Should it be considered an improvement? Justify your answer using the ISO 9241-11 framework.

Multiple Choice Questions with Answers

Question 1

Which statement best defines usability?

A. The visual attractiveness of an interface

B. The extent to which specified users achieve specified goals effectively, efficiently, and satisfactorily in a specified context

C. The number of functions included in a product

D. The speed at which software loads

Correct Answer: B. The extent to which specified users achieve specified goals effectively, efficiently, and satisfactorily in a specified context

Question 2

What does utility primarily concern?

A. Whether the system provides functions users need

B. Whether users enjoy the colour scheme

C. Whether the system uses modern technology

D. Whether users need training

Correct Answer: A. Whether the system provides functions users need

Question 3

Which usability dimension concerns accuracy and completeness of goal achievement?

A. Satisfaction

B. Accessibility

C. Effectiveness

D. Efficiency

Correct Answer: C. Effectiveness

Question 4

Which measure is most directly associated with efficiency?

A. Time required for a successful task

B. Number of available features

C. User age

D. Brand recognition

Correct Answer: A. Time required for a successful task

Question 5

Why is context of use important in usability evaluation?

A. Usability is identical for all users and environments

B. User performance and experience depend on circumstances, goals, users, and technology

C. Context determines only visual styling

D. Context replaces the need for user research

Correct Answer: B. User performance and experience depend on circumstances, goals, users, and technology

Question 6

Which example best represents an accessibility practice?

A. Adding more decorative animation

B. Hiding error messages to reduce visual clutter

C. Providing keyboard operation and meaningful text alternatives

D. Removing headings from long pages

Correct Answer: C. Providing keyboard operation and meaningful text alternatives

Question 7

What is the principal purpose of human-centred design?

A. To eliminate iteration

B. To focus design decisions on people, their needs, and their contexts

C. To prioritize developer convenience over user needs

D. To test only the finished product

Correct Answer: B. To focus design decisions on people, their needs, and their contexts

Question 8

A user completes a booking quickly but enters the wrong date. Which conclusion is most appropriate?

A. The task was fully effective because it was fast

B. The task was effective but not efficient

C. The task was not fully effective despite being fast

D. Satisfaction can be inferred as high

Correct Answer: C. The task was not fully effective despite being fast

Question 9

Which statement about usability testing is accurate?

A. A small test guarantees success for every future user

B. Testing can reveal problems but findings must be interpreted in relation to its participants and conditions

C. Testing should occur only after launch

D. Participant opinions are always more reliable than observed behaviour

Correct Answer: B. Testing can reveal problems but findings must be interpreted in relation to its participants and conditions

Question 10

What is a dark pattern?

A. A low-fidelity prototype

B. A method for measuring task time

C. An accessibility feature for dark environments

D. An interface practice that manipulates or deceives users

Correct Answer: D. An interface practice that manipulates or deceives users

Usability Testing & UX Optimization
Chapter 1 of 15