Digital Products, AI and Emerging Technology

Introduction

Digital products are no longer limited to screens and software features. A contemporary product may be a mobile application, an API platform, a connected appliance, an intelligent service, or an ecosystem combining all four. Product managers therefore design not only interfaces but also data flows, algorithms, service operations, trust mechanisms, and relationships among multiple participants.

Artificial intelligence (AI) intensifies this responsibility. AI-enabled products can classify, recommend, predict, generate, converse, and act. These capabilities create significant value, but they also introduce uncertainty, opacity, automation risks, privacy concerns, and new forms of exclusion. Design thinking remains useful because it places human needs, contextual understanding, experimentation, and ethical reflection at the centre of technological development. However, design thinking for AI must extend beyond the immediate user experience to include data subjects, operators, affected communities, regulators, and future users.

This chapter examines design principles for platforms, mobile products, connected services, and AI-enabled experiences. It also addresses human–AI interaction, explainability, automation boundaries, data quality, synthetic research, and responsible use of generative AI in discovery and prototyping.

Learning Objectives

By the end of this chapter, readers should be able to:

1. Distinguish the design challenges of platforms, mobile products, connected services, and AI-enabled products.
2. Apply appropriate models for human–AI interaction and automation design.
3. Evaluate explainability, transparency, uncertainty, and user control.
4. Diagnose data-quality risks and their effects on product performance.
5. Use synthetic research and generative AI responsibly during discovery and prototyping.
6. Develop governance practices for safe, inclusive, and accountable digital products.

1. Digital Products as Sociotechnical Systems

A digital product is a sociotechnical system: a combination of technical components, human behaviours, institutional rules, business processes, and social expectations. The visible interface is only one layer. A banking application, for example, depends on identity systems, payment networks, fraud models, customer-support processes, security controls, and legal obligations.

This perspective changes the unit of design. A product manager should ask not only, “Can the user complete the task?” but also:

• What data makes the task possible?
• Which people or organisations are affected indirectly?
• What happens when the system is wrong, unavailable, or misused?
• Who can challenge or reverse an automated decision?
• What operational capabilities are required after launch?

A useful service model contains five interacting layers:

The layers are interdependent. A highly accurate model is not valuable if users cannot interpret its output; an elegant interface is not trustworthy if the underlying data is incomplete; and a useful service is not sustainable if its operating cost exceeds its economic value.

2. Designing Platforms and Ecosystems

A platform enables interactions between two or more participant groups. Examples include marketplaces connecting buyers and sellers, operating systems connecting developers and users, and payment platforms connecting merchants, consumers, and financial institutions. Platform design differs from conventional product design because value is often created through participant interaction rather than by the platform alone.

2.1 Core platform design problems

Participant roles. A platform must define who can provide, consume, moderate, or govern value. Roles may overlap, as when a driver is both a service provider and a customer.

Network effects. The value of a platform may increase as participation grows. Direct network effects occur when users benefit from more users of the same type; indirect network effects occur when one participant group increases value for another. Growth is not automatically beneficial: poor-quality participants, fraud, congestion, or irrelevant content can reduce value.

Rules and incentives. Ranking, pricing, access, reputation, and dispute-resolution rules shape participant behaviour. These rules are product decisions, not merely policy details.

Trust and safety. Identity verification, reviews, content moderation, fraud detection, appeals, and payment protection are essential design components.

Governance and openness. A platform must decide who may join, what data can be accessed, how APIs are controlled, and how changes are communicated. Excessive openness may create abuse; excessive control may discourage innovation.

Platform discovery should therefore include interviews and observation across participant groups, analysis of failed interactions, and investigation of power asymmetries. A platform that satisfies its most powerful participants while imposing hidden costs on less powerful users may achieve short-term growth but produce long-term legitimacy and retention problems.

3. Mobile Products and Context-Aware Design

Mobile design is shaped by intermittent attention, variable connectivity, small screens, touch interaction, location, sensors, battery constraints, and privacy expectations. The mobile context is not simply a smaller version of desktop computing.

Effective mobile products generally provide:

• clear prioritisation of the primary task;
• short, recoverable interaction sequences;
• readable typography and adequate touch targets;
• graceful degradation when connectivity is poor;
• explicit permission requests at the moment of need;
• accessible alternatives to gestures, colour, or audio;
• protection against accidental actions and data loss.

Context-aware features should be proportionate to their benefit. Location access, for example, may be justified for navigation while the application is in use but excessive for a feature that does not require continuous tracking. Product teams should distinguish necessary data from merely convenient data and should communicate retention, sharing, and deletion practices clearly.

The key metric is not feature count but successful, appropriate task completion. A mobile product may be technically sophisticated yet fail because it interrupts users at unsuitable moments or demands more attention than the context permits.

4. Connected Services and the Internet of Things

Connected services combine physical objects, sensors, communications, cloud services, software interfaces, and human operations. Examples include smart energy systems, connected medical devices, industrial monitoring, and shared mobility.

Design must address the entire lifecycle:

1. acquisition and installation;
2. configuration and identity assignment;
3. normal use;
4. maintenance and updates;
5. failure, recovery, and support;
6. decommissioning and data deletion.

Connected products create distinctive risks. A device may be physically unsafe, unavailable during a network outage, vulnerable to attack, or difficult to update. Users may also misunderstand what is being sensed and where data is sent. Product managers should design local fallback behaviour, visible status indicators, secure update mechanisms, device ownership transfer, and end-of-life procedures.

Figure: A connected service is a chain of technical and human layers governed as one system.

5. Human–AI Interaction

AI changes interaction from direct manipulation to collaboration, delegation, supervision, and negotiation. A user may ask an assistant to draft a document, accept a recommendation, correct a classification, or approve an action. The interaction design must make the system’s role intelligible.

5.1 Levels of AI participation

A practical classification is:

The appropriate level depends on consequence, reversibility, uncertainty, user expertise, and speed requirements. Automation is more defensible for low-risk, reversible actions than for irreversible decisions affecting employment, credit, healthcare, safety, or liberty.

5.2 Automation boundaries

Product teams should define:

• what the system may do automatically;
• what requires confirmation;
• what is prohibited;
• when a human must be involved;
• how the user can pause, undo, or appeal an action;
• what evidence is recorded for later review.

A human-in-the-loop control is meaningful only when the human has sufficient time, authority, information, and competence to intervene. A nominal approval button placed after an opaque and time-pressured process is not genuine oversight.

5.3 Explainability and calibrated trust

Explainability has several meanings. A system may provide a global explanation of how it generally works, a local explanation of why it produced a particular output, a counterfactual explanation of what would need to change, or a process explanation showing data sources and decision stages. No single explanation serves every audience.

Good explanations are relevant, understandable, truthful, and actionable. They should communicate uncertainty rather than imply false precision. Product teams should test whether users form an appropriately calibrated level of trust: neither blindly accepting outputs nor rejecting useful assistance.

6. Data Quality and Product Performance

AI performance is constrained by the quality and suitability of data. Quality is multidimensional:

• Accuracy: Does the data represent the underlying fact?
• Completeness: Are required values present?
• Consistency: Do records agree across systems?
• Timeliness: Is the data current enough for the task?
• Validity: Does it conform to the intended format and meaning?
• Representativeness: Does it reflect relevant populations and contexts?
• Provenance: Can its origin, consent, transformation, and use be traced?

A useful conceptual relationship is:

This is not an engineering law or a calculable identity; it expresses dependence among factors. Strong model capability cannot compensate for systematically missing groups, ambiguous labels, or a shift between training and deployment contexts.

Data work should begin during discovery, not after model selection. Teams should create a data inventory, identify sensitive attributes, document label definitions, examine missingness, test representative coverage, and monitor drift after deployment. When a model is used to generate labels or prioritise cases, the resulting feedback loop may reinforce existing errors.

7. Synthetic Research and Generative AI in Discovery

Synthetic research uses generated personas, simulated responses, artificial datasets, or model-generated scenarios. It can help teams explore hypotheses, create edge cases, rehearse interviews, generate test content, and identify questions for real research. It cannot substitute for evidence from actual users when decisions affect real people.

Synthetic participants are especially risky because language models produce plausible but not necessarily valid representations of human attitudes. They may reproduce stereotypes, overrepresent articulate mainstream users, and conceal disagreement. Synthetic outputs should therefore be treated as hypotheses, not observations.

Responsible uses of generative AI

Generative AI can assist with:

• expanding an interview guide;
• clustering or summarising consented research data;
• generating alternative journey-map wording;
• producing low-fidelity interface variants;
• creating test cases and accessibility prompts;
• role-playing stakeholders before real interviews;
• identifying assumptions and unanswered questions.

It should not be used to fabricate participant evidence, infer sensitive attributes without justification, upload confidential research to an uncontrolled service, or present generated content as validated design knowledge.

A disciplined workflow is:

1. State the research question and decision to be informed.
2. Separate observed evidence, interpretation, and speculation.
3. Use AI to generate alternatives or questions, not to manufacture findings.
4. Preserve source traceability and human review.
5. Test important claims with real users or reliable data.
6. Record the tool, prompt purpose, model limitations, and approval decisions.

8. Prototyping AI-Enabled Experiences

Traditional prototypes represent screens and flows. AI prototypes must also represent uncertainty, latency, data dependencies, refusal behaviour, confidence, escalation, and failure recovery. A convincing demonstration can conceal a weak underlying service, so teams should distinguish appearance prototypes from capability prototypes.

A useful AI prototype includes:

• representative and adversarial inputs;
• expected outputs and unacceptable outputs;
• uncertainty or abstention states;
• correction and feedback mechanisms;
• human escalation;
• audit events;
• privacy and consent behaviour;
• performance thresholds appropriate to the use case.

For generative systems, evaluation should include factuality, relevance, harmful content, instruction following, consistency, privacy leakage, and resistance to manipulation. Evaluation sets should contain ordinary, ambiguous, rare, and adversarial cases. “Works in the demo” is not a sufficient acceptance criterion.

9. Governance, Ethics, and Regulation

Responsible product management requires governance throughout the lifecycle rather than a final ethics review. Key principles include:

• Purpose limitation: collect and use data for defined legitimate purposes.
• Data minimisation: avoid unnecessary collection and retention.
• Fairness: identify and mitigate unjustified disparities.
• Transparency: communicate relevant system behaviour and limitations.
• Accountability: assign named owners for decisions and incidents.
• Security: protect systems against unauthorised access and manipulation.
• Accessibility: include people with diverse abilities and contexts.
• Contestability: provide correction, appeal, and human review where appropriate.
• Sustainability: consider energy, hardware, labour, and environmental effects.

Regulatory obligations vary by jurisdiction and sector. Product managers should involve legal, privacy, security, accessibility, and risk specialists early, while recognising that compliance is a baseline rather than a complete ethical analysis. A product may be legal yet misleading, exclusionary, or socially harmful.

10. An Integrated Product Design Process

A practical process for emerging-technology products can be organised as follows:

Frame the system. Define users, non-users, affected parties, data flows, dependencies, and high-consequence outcomes.

Discover the context. Conduct interviews, observation, diary studies, technical investigation, and service-operations research. Identify workarounds and failure consequences.

Map risks and assumptions. Record assumptions about user intent, data availability, model capability, incentives, and acceptable error. Rank them by consequence and uncertainty.

Choose the least complex viable intervention. Do not introduce AI where a rule, search function, workflow improvement, or human service is more appropriate.

Prototype progressively. Test the interaction first, then data and model behaviour, then operational integration and governance.

Evaluate with real and edge-case evidence. Combine usability, accessibility, technical, fairness, security, and safety evaluation.

Launch with monitoring. Define leading indicators, incident thresholds, escalation routes, rollback procedures, and periodic review.

Learn and retire responsibly. Communicate changes, correct errors, delete data when justified, and decommission obsolete models and devices safely.

Illustrative Case: AI-Assisted Clinical Appointment Navigation

Background

A regional health service wants to reduce missed appointments and help patients find suitable clinics. It proposes a mobile assistant that answers questions, recommends appointment options, and sends reminders.

Problem

Early discovery suggests that patients use different terms for symptoms, have varying digital literacy, and may share phones with family members. Some patients have limited connectivity or require language and accessibility support. A wrong recommendation could delay care.

Analysis

The team separates low-risk navigation from clinical diagnosis. It identifies stakeholders beyond the app user: carers, clinicians, reception staff, interpreters, privacy officers, and patients whose data may be exposed through notifications. It also finds that historical attendance data reflects unequal transport access, so using it directly to prioritise patients could reproduce disadvantage.

Approach and solution

The first release uses structured information retrieval and clear escalation to staff rather than open-ended diagnosis. The assistant displays the source and date of service information, states when it is uncertain, and offers a call-back option. Appointment recommendations consider accessibility and transport needs only when users voluntarily provide them. Notifications use discreet wording, and the system supports offline confirmation codes. Generative AI is used internally to draft alternative plain-language messages, but all clinical and operational content is approved by qualified staff.

Testing includes older adults, screen-reader users, people with limited English, and patients with intermittent connectivity. Red-team testing examines prompt manipulation, disclosure of private information, and unsafe symptom advice.

Outcome

The service reduces routine enquiry volume while retaining a human route for uncertain or sensitive cases. More importantly, the team avoids presenting a general-purpose conversational model as a medical authority. Success is assessed through completed bookings, comprehension, inappropriate advice incidents, accessibility outcomes, staff workload, and patient trust—not merely the number of conversations.

Learning points

1. The safest AI product may be narrower than the original proposal.
2. Human escalation must be designed as part of the experience.
3. Data used for prediction may encode structural inequality.
4. Trust depends on useful transparency and reliable recovery, not confident language.
5. Generative AI is valuable for drafting and exploration, but validation remains human and evidential.

Challenges and Limitations

Emerging-technology design faces persistent constraints. Models can be probabilistic while users expect deterministic answers. Data may be legally available but ethically inappropriate. Explainability may simplify complex uncertainty or create a misleading story. Human oversight can become burdensome and ineffective at scale. Monitoring may identify performance degradation without revealing its social cause. Connected devices create long-term maintenance obligations, and generative systems introduce cost, energy use, copyright questions, and information-security risks.

Teams should also recognise organisational limitations. Responsible design requires time, diverse expertise, procurement controls, incident response, and leadership willingness to delay or withdraw a feature. A checklist cannot compensate for incentives that reward growth while externalising harm.

Best-Practice Checklist

Before launch, the product team should be able to answer:

• What user and societal problem is being addressed?
• Why is AI or connected technology necessary?
• Who benefits, who bears risk, and who is absent from the research?
• What data is used, and can its provenance and quality be demonstrated?
• What happens when the system is uncertain or wrong?
• Can users understand, correct, pause, or appeal the system?
• Are privacy, security, accessibility, and safety tested in context?
• Are synthetic outputs clearly separated from real evidence?
• Are monitoring, ownership, rollback, and retirement defined?
• What evidence would justify limiting, redesigning, or stopping the product?

Chapter Summary

Digital products should be designed as sociotechnical systems that include interfaces, data, models, infrastructure, operations, and governance. Platforms require attention to participant roles, incentives, network effects, trust, and power. Mobile products must respect context, attention, connectivity, accessibility, and privacy. Connected services require lifecycle thinking, secure updates, fallback behaviour, and clear data practices.

AI-enabled experiences require deliberate automation boundaries, calibrated trust, meaningful explanations, uncertainty communication, and genuine human control. Data quality involves accuracy, completeness, consistency, timeliness, validity, representativeness, and provenance. Synthetic research and generative AI can accelerate exploration and prototyping, but generated outputs are hypotheses rather than user evidence. Responsible product management integrates fairness, privacy, security, accessibility, accountability, contestability, and sustainability throughout discovery, development, deployment, and retirement.

Key Terms

Digital product: A product whose value depends substantially on software, data, digital interaction, or connected services.

Sociotechnical system: A system formed by technical components and the people, institutions, rules, and practices surrounding them.

Platform: A product or service that enables interaction among multiple participant groups.

Network effect: A change in value resulting from changes in the number or composition of participants.

Connected service: A service combining physical devices, connectivity, software, data, and human operations.

Human–AI interaction: The design of communication, collaboration, delegation, and control between people and AI systems.

Automation boundary: The defined limit between actions performed automatically and actions requiring human involvement.

Explainability: The degree to which a system’s behaviour or output can be meaningfully understood.

Calibrated trust: Trust proportionate to a system’s actual capabilities, uncertainty, and limitations.

Data provenance: A traceable record of data origin, transformation, consent, and use.

Data drift: A change in data characteristics between development and deployment or across time.

Synthetic research: Research-like material generated or simulated by computational systems rather than observed directly from participants.

Generative AI: AI capable of producing new text, images, audio, code, or other content.

Human-in-the-loop: A design in which a human reviews, authorises, corrects, or supervises system activity.

Contestability: The ability of affected people to challenge, correct, or appeal a system outcome.

Model monitoring: Ongoing observation of performance, safety, fairness, reliability, and changing conditions.

Data minimisation: Collecting and retaining no more personal data than is necessary for a defined purpose.

Responsible innovation: Developing technology while anticipating and addressing ethical, social, environmental, and governance consequences.

Review Questions

Short Answer Questions

1. Why should a digital product be treated as a sociotechnical system?
2. Distinguish direct and indirect network effects.
3. What is meant by an automation boundary?
4. Name four dimensions of data quality.
5. Why can synthetic research not replace research with real users?

Long Answer Questions

1. Explain the principal design challenges of digital platforms and propose ways to build trust among different participant groups.
2. Discuss how explainability and uncertainty communication influence human–AI interaction.
3. Compare the design requirements of mobile products and connected services.
4. Describe a responsible workflow for using generative AI in discovery and prototyping.
5. Explain why governance must continue after an AI-enabled product is launched.

Analytical / Application Questions

1. A recruitment platform uses an AI model to rank applicants. Develop an analysis covering data quality, fairness, explainability, human oversight, and appeal mechanisms.
2. Design a low-risk automation policy for a smart-home energy service. Specify which actions may be automatic, which require confirmation, and how failures should be handled.
3. A product team presents synthetic interviews as evidence that users want a new AI feature. Critique the approach and propose a valid mixed-method discovery plan.

Multiple Choice Questions with Answers

Question 1

Which characteristic most clearly distinguishes a platform from a conventional single-user digital product?

A. It always uses artificial intelligence.

B. It enables interactions among multiple participant groups.

C. It must be delivered through a mobile application.

D. It cannot operate without physical devices.

Correct Answer: B. It enables interactions among multiple participant groups.

Question 2

Which action best represents meaningful human oversight of an AI system?

A. Displaying an approval button after an irreversible action has already occurred.

B. Allowing a trained operator sufficient authority and time to review and reverse a proposed action.

C. Hiding uncertainty so that the operator acts more quickly.

D. Delegating all exceptions to an automated rule.

Correct Answer: B. Allowing a trained operator sufficient authority and time to review and reverse a proposed action.

Question 3

Which data-quality dimension concerns whether data reflects the relevant populations and contexts?

A. Representativeness

B. Latency

C. Compression

D. Encryption

Correct Answer: A. Representativeness

Question 4

What is the most appropriate use of synthetic research?

A. Presenting generated personas as verified customer evidence.

B. Replacing all interviews with model-generated responses.

C. Generating hypotheses and edge cases for later validation with real users.

D. Inferring sensitive attributes about actual participants without consent.

Correct Answer: C. Generating hypotheses and edge cases for later validation with real users.

Question 5

A counterfactual explanation primarily addresses which question?

A. Who owns the software licence?

B. What change might have produced a different outcome?

C. How many users downloaded the application?

D. Where is the physical server located?

Correct Answer: B. What change might have produced a different outcome?

Question 6

Why should a connected product include lifecycle design?

A. Devices never require maintenance after purchase.

B. Connectivity eliminates the need for support.

C. Installation, updates, failure, ownership transfer, and retirement create ongoing risks and responsibilities.

D. Lifecycle design applies only to industrial products.

Correct Answer: C. Installation, updates, failure, ownership transfer, and retirement create ongoing risks and responsibilities.

Question 7

Which principle recommends avoiding unnecessary collection and retention of personal data?

A. Data minimisation

B. Network externality

C. Model distillation

D. Interface consistency

Correct Answer: A. Data minimisation

Question 8

A medical assistant is uncertain about a patient’s request. Which response best follows the chapter’s guidance?

A. Provide a confident diagnosis to maintain conversational flow.

B. Refuse every request without offering alternatives.

C. State the limitation and provide an appropriate human escalation route.

D. Generate several diagnoses and ask the patient to select one.

Correct Answer: C. State the limitation and provide an appropriate human escalation route.

Question 9

Why is a successful product demonstration insufficient for accepting a generative AI feature?

A. Demonstrations cannot include user interfaces.

B. A demonstration may conceal failures involving uncertainty, adversarial inputs, privacy, and operational integration.

C. Generative AI can only be evaluated mathematically.

D. Users never interact with prototypes.

Correct Answer: B. A demonstration may conceal failures involving uncertainty, adversarial inputs, privacy, and operational integration.

Question 10

Which governance mechanism allows an affected person to challenge or correct an outcome?

A. Contestability

B. Caching

C. Interoperability

D. Compression

Correct Answer: A. Contestability