Digital Product Management & AI-Driven Products

Chapter 9
Foundations of AI-Driven Products

Post Graduate Program in Product Management
Digital Product Management & AI-Driven Products   •   Post Graduate Diploma

Foundations of AI-Driven Products

Introduction and Context

Artificial intelligence (AI) has moved from a specialist engineering discipline into the core of digital product management. Search engines, fraud detection systems, conversational assistants, medical-imaging tools, recommendation platforms and autonomous operations all use computational systems that infer patterns from data and produce outputs such as predictions, rankings, classifications, decisions or generated content.

For a product manager, the central question is not simply “Can we use AI?” It is: What user or business problem is sufficiently valuable, measurable, data-supported and operationally feasible for AI to solve better than simpler alternatives? Answering this question requires understanding the foundations of machine learning, deep learning, natural language processing, recommendation systems and generative AI.

AI-driven products differ from conventional software because their behaviour is often probabilistic rather than completely specified in advance. This affects product discovery, requirements, testing, user experience, monitoring, risk management and governance. A product manager must therefore manage not only features and workflows but also data quality, model performance, uncertainty, feedback loops and changing environments.

Learning Objectives

By the end of this chapter, you should be able to:

  1. Explain the relationships among artificial intelligence, machine learning, deep learning and generative AI.
  2. Distinguish deterministic software from probabilistic AI systems.
  3. Describe supervised, unsupervised and reinforcement learning.
  4. Explain the basic principles of natural language processing and recommendation systems.
  5. Assess whether an AI use case is supported by adequate data, measurable outcomes and feasible deployment conditions.
  6. Interpret basic classification and regression metrics.
  7. Identify uncertainty, bias, privacy, security, safety and governance risks.
  8. Apply an AI product-development framework to a realistic product problem.

1. The AI Product Landscape

Artificial intelligence refers broadly to computational systems designed to perform tasks associated with human intelligence, including perception, reasoning, language understanding, learning and decision support. The term is broad and includes both learning-based systems and systems based on explicit rules or symbolic reasoning.

Machine learning (ML) is a subset of AI in which a system learns a function or decision policy from examples, data or interaction rather than being programmed with every rule explicitly. Deep learning is a subset of ML that uses multi-layer neural networks to learn representations from data. Generative AI refers to models that create new outputs—such as text, images, audio, video, code or structured data—based on learned patterns and a prompt, instruction or conditioning input.

The relationship can be represented as:

AI ⊃ ML ⊃ Deep learning

Generative AI overlaps substantially with deep learning, although not every generative system must use a deep neural network in a strict conceptual sense.

Important terminology

  • Feature: An input variable used by a model, such as transaction amount or customer tenure.
  • Label or target: The outcome a supervised model is trained to predict.
  • Training data: Data used to estimate model parameters.
  • Validation data: Data used to tune choices such as model complexity or hyperparameters.
  • Test data: Held-out data used to estimate performance on unseen examples.
  • Inference: Applying a trained model to new input.
  • Model parameter: A value learned during training.
  • Hyperparameter: A configuration selected outside the ordinary learning process, such as learning rate or tree depth.
  • Embedding: A numerical vector representing an item, word, document, user or other entity in a learned space.
  • Inference latency: The time required to produce a model output.
  • Drift: A change in data, relationships or user behaviour that can reduce model performance.

2. Deterministic Software and Probabilistic AI

Traditional software is usually specified as rules and procedures. Given the same input, program version and operating conditions, a deterministic function produces the same output:

y = f(x)

Here, x is the input, y is the output and f is a fixed program. Determinism does not mean that conventional software is always correct; it means that its behaviour is specified and repeatable under the same conditions.

An ML system instead estimates a relationship from data. A classifier may produce a probability:

P(y = 1 | x) = p

The value p is the model’s estimated probability that an example with features x belongs to class 1. A business rule may then convert this probability into an action. For example, a transaction may be reviewed if p ≥ 0.80. The threshold is a product and risk decision, not merely a technical fact.

DimensionDeterministic softwareProbabilistic AI system
BehaviourExplicitly specifiedLearned or inferred from data
OutputUsually fixed for a given inputOften a prediction, score or distribution
TestingCheck rules and expected casesEvaluate statistical performance on representative data
Failure patternOften traceable to logic or integrationMay arise from bias, drift, ambiguity or unusual inputs
Change processCode and configuration changesCode, data, labels, prompts or model changes
Product requirementFunctional behaviourDesired performance, uncertainty and constraints

The distinction affects requirements. “The system shall identify every fraudulent transaction” is generally unrealistic. A better requirement specifies an operating point, such as a maximum review workload, minimum recall for a high-risk class, acceptable false-positive rate and response-time limit.

AI systems are not necessarily opaque, and traditional software is not necessarily transparent. The distinction concerns how behaviour is produced. A complex rule engine can be difficult to understand, while a simple linear model may be interpretable. Product teams should therefore assess transparency, traceability and controllability separately from the deterministic–probabilistic distinction.

3. Machine Learning Paradigms

3.1 Supervised learning

In supervised learning, the training data contain examples of inputs and known outcomes. The model learns a mapping from features x to a target y. Common tasks include:

  • Classification: Predicting categories, such as spam/not spam.
  • Regression: Predicting a numerical quantity, such as delivery time.
  • Ranking: Ordering alternatives, such as search results or products.

A general training objective is to minimise empirical loss:

θ̂ = arg min_θ (1/n) ∑i=1n L(f_θ(xi), yi)

Here, n is the number of training examples, xi is the input for example i, yi is its observed target, f_θ is the model with parameters θ, and L is a loss function. The formula assumes that examples and labels are sufficiently representative of the deployment environment. Minimising training loss alone can produce overfitting, so validation and test evaluation are essential.

For regression, mean absolute error is:

MAE = (1/n) ∑i=1n |yi − ŷi|

where ŷi is the prediction. If the target is measured in minutes, MAE is also measured in minutes. Mean squared error is:

MSE = (1/n) ∑i=1n (yi − ŷi)2

MSE is expressed in squared target units and penalises large errors more strongly than MAE.

For binary classification, a confusion matrix contains true positives (TP), false positives (FP), true negatives (TN) and false negatives (FN). Precision and recall are:

Precision = TP ÷ (TP + FP)
Recall = TP ÷ (TP + FN)

Precision answers: among predicted positives, how many were positive? Recall answers: among actual positives, how many were found? The appropriate metric depends on consequences. In disease screening, missing a true case may be more harmful than creating an additional referral; in customer communications, excessive false positives may damage trust.

Accuracy is:

Accuracy = (TP + TN) ÷ (TP + TN + FP + FN)

Accuracy can be misleading with imbalanced classes. If only 1% of transactions are fraudulent, a model that always predicts “not fraud” achieves 99% accuracy but detects no fraud.

3.2 Unsupervised learning

Unsupervised learning works without a target label. It can identify clusters, reduce dimensionality or detect unusual observations. For example, a subscription business might cluster customers according to usage patterns. Clusters are not automatically meaningful customer segments; they require interpretation, validation and a plausible action.

A common clustering method is k-means. It assigns observations to k clusters by minimising within-cluster squared distance:

J = ∑j=1k ∑xi ∈ Cj ||xi − μj||²

Here, Cj is cluster j, μj is its centroid and ||xi − μj||² is squared distance. The method is sensitive to scaling, the selected value of k and the assumption that centroid-based clusters are appropriate. Product teams should not treat an algorithmic segment as a discovered truth.

3.3 Reinforcement learning

In reinforcement learning, an agent takes actions in an environment and receives rewards. The agent seeks a policy that maximises expected cumulative reward. This paradigm is relevant to sequential decisions such as recommendation ordering, inventory control or adaptive experimentation.

A simplified discounted return is:

Gt = rt + γrt+1 + γ²rt+2 + …

where rt is the reward at time t and γ is a discount factor with 0 ≤ γ < 1. The discount factor expresses how strongly future rewards are valued. In real products, reward design is difficult: optimising clicks may reduce satisfaction, and optimising short-term revenue may increase cancellations later.

4. Deep Learning

Deep learning uses neural networks composed of layers that transform inputs into increasingly abstract representations. A basic neuron computes a weighted sum followed by an activation function:

h = σ(wᵀx + b)

Here, x is an input vector, w is a weight vector, b is a bias, σ is an activation function and h is the output. A deep network applies many such transformations. During training, optimisation methods adjust weights to reduce a loss function.

Deep learning has been particularly effective where large datasets and high-dimensional inputs are available, including images, speech, text and sensor data. Convolutional networks have historically been important for visual tasks; recurrent architectures were influential for sequences; and transformer architectures now underpin many language and multimodal systems.

Deep learning is not automatically superior. It may require substantial data, compute, specialised expertise and monitoring. A smaller model or rules-based approach may be preferable when the dataset is limited, the decision must be highly explainable, or latency and infrastructure costs are tightly constrained.

5. Natural Language Processing

Natural language processing (NLP) enables systems to process human language. Product applications include search, document classification, sentiment analysis, translation, summarisation, question answering, transcription and conversational interfaces.

Language presents several challenges:

  • Words can be ambiguous in context.
  • Meaning depends on syntax, discourse and world knowledge.
  • Users express the same intention in many ways.
  • Language changes across communities and time.
  • Training data may encode stereotypes, private information or historical exclusion.

Modern NLP systems often represent text as tokens and map tokens or sequences into embeddings. An embedding is not a dictionary definition; it is a learned numerical representation useful for tasks such as similarity search or classification. Retrieval systems can compare a user query with document embeddings and return relevant passages.

Large language models (LLMs) are trained to model sequences of tokens. Their apparent fluency arises from learning statistical regularities over very large corpora and from later alignment or instruction-tuning processes. Fluency does not guarantee factual accuracy, reasoning reliability or domain suitability. An LLM may produce a plausible but unsupported statement, commonly called a hallucination.

For product design, conversational quality should be decomposed into measurable dimensions: task completion, factuality, groundedness, response time, refusal behaviour, escalation quality, user effort and satisfaction. A chatbot that produces elegant text but cannot complete the intended task is not a successful product.

6. Recommendation Systems

Recommendation systems select or rank items for a user. They are used in commerce, media, education, advertising and professional platforms. A recommendation pipeline commonly includes:

  1. Candidate generation: Retrieve a manageable set of potentially relevant items.
  2. Ranking: Score and order candidates using user, item and contextual features.
  3. Policy and filtering: Apply inventory, safety, legal, diversity and business constraints.
  4. Presentation: Display recommendations in a user interface.
  5. Feedback collection: Observe clicks, purchases, dwell time, skips, ratings or complaints.

Two classic approaches are:

  • Content-based recommendation: Recommend items similar to those a user previously liked, using item attributes or embeddings.
  • Collaborative filtering: Use patterns of interaction among users and items. A user may receive an item because users with similar behaviour interacted with it.

The cold-start problem occurs when a new user or item has little interaction history. Solutions include onboarding preferences, content features, popularity priors, contextual signals and carefully designed exploration.

Recommendation objectives must be broader than immediate clicks. Important measures may include conversion, retention, satisfaction, content diversity, creator exposure, complaint rates and long-term value. Optimising a single proxy can create feedback loops: items shown more often receive more interactions, which makes them appear more popular, which causes them to be shown even more.

7. Generative AI

Generative AI creates outputs rather than merely selecting or classifying existing records. Major product patterns include:

  • Drafting and transformation: summarising, rewriting or translating.
  • Retrieval-augmented generation (RAG): retrieving trusted content and using it to support generation.
  • Copilots: assisting users within a professional workflow.
  • Content creation: generating text, images, audio, video or code.
  • Synthetic data: producing artificial examples for testing or augmentation, subject to privacy and fidelity risks.
  • Agentic workflows: allowing a model to plan or call tools under controlled permissions.

A generative AI product consists of more than a foundation model. It may include retrieval, prompt or policy logic, tools, databases, identity controls, interfaces, evaluation systems and human review. Model selection should consider quality, cost, latency, context limits, data handling, reliability, customisation and vendor dependence.

A useful distinction is between model capability and product reliability. A model may answer a question correctly in a demonstration while the product fails because the wrong document was retrieved, permissions were ignored, the output was not validated or the user misunderstood the confidence of the answer.

Generation can be represented abstractly as sampling from a conditional distribution:

ŷ ∼ P(y | x, c)

Here, x is the user input, c represents context such as retrieved documents or system instructions, and ŷ is a generated output. The symbol ∼ indicates sampling rather than guaranteed equality. Consequently, identical prompts may produce different outputs unless generation settings and system conditions constrain randomness.

8. Assessing AI Use-Case Feasibility

A promising AI use case should pass several tests.

Value and problem quality

The problem should be important, frequent or costly enough to justify AI investment. Product teams should define the user, the decision or task, the current workaround and the measurable outcome. “Add an AI assistant” is not a problem statement. “Reduce the time support agents spend locating approved troubleshooting steps while preserving escalation accuracy” is more actionable.

Data readiness

Data assessment includes:

  • Availability and access rights.
  • Relevance to the target task.
  • Completeness and consistency.
  • Label quality and label definitions.
  • Representativeness of users, contexts and edge cases.
  • Recency and expected drift.
  • Privacy, consent, retention and provenance.
  • Separation of training data from future evaluation data.

The quantity of data is not enough. A million records generated by a biased process may be less useful than a smaller, carefully labelled and representative dataset.

Technical and operational feasibility

Assess whether the system can meet latency, availability, security, integration, infrastructure and cost requirements. A model that is accurate but takes 20 seconds to respond may be unsuitable for an interactive workflow. A high-performing model that requires sending confidential data to an unauthorised service is not deployable.

Economic feasibility

A simple expected-value model can support early prioritisation:

Expected net value = Expected benefit − (development cost + operating cost + risk cost)

This is not a complete financial model. Benefits should include only plausible incremental outcomes, while operating cost may include inference, storage, human review, monitoring and model updates. Risk cost includes expected losses from errors, misuse, compliance failure and reputational damage.

Uncertainty and risk

Uncertainty can arise from noisy data, limited examples, ambiguous labels, distribution shift or model limitations. Product teams should define what happens when confidence is low: ask for clarification, abstain, route to a human, show sources or provide a safe fallback. A system that is not allowed to say “I do not know” may create avoidable harm.

9. Product Development for AI Systems

AI product development is iterative but requires explicit evaluation design.

Discovery

Define the user problem, decision context, affected stakeholders and unacceptable failure modes. Identify whether AI is necessary or whether a rule, search interface, workflow redesign or better data display would solve the problem more simply.

Data and label design

Create a data card or equivalent record describing sources, collection conditions, ownership, intended use, known gaps and restrictions. Define labels operationally. For example, “urgent support issue” needs criteria and adjudication procedures; otherwise model evaluation will be inconsistent.

Baselines

Establish a non-AI baseline, such as manual performance, a rules engine, popularity ranking or a simple statistical model. The AI system should demonstrate improvement on a meaningful product outcome, not merely on a benchmark score.

Offline evaluation

Use temporally and contextually appropriate splits. Random splits can leak future information when records are time-dependent. Evaluate overall performance and performance across important subgroups. Inspect false positives and false negatives, not only aggregate metrics.

Online evaluation

Controlled experiments can compare product variants, but guardrail metrics are essential. A recommendation change may increase clicks while reducing retention. A support assistant may reduce handling time while increasing incorrect resolutions. Monitor both primary outcomes and harm indicators.

Deployment and monitoring

Monitor data quality, feature distributions, prediction distributions, latency, cost, model errors, subgroup performance, abuse attempts and user feedback. Establish owners, incident procedures, rollback mechanisms and review schedules. Monitoring is part of the product, not a post-launch technical afterthought.

10. Illustrative Case: AI-Assisted Customer Support

Background

A fictional subscription software company receives 40,000 support requests per month. Agents spend substantial time searching internal documentation. The company proposes a generative AI assistant that drafts answers and cites relevant help-centre articles.

Problem

The initial proposal—“deploy an AI chatbot”—is narrowed to an agent-assist product. The assistant will recommend sources and draft responses, while the agent retains responsibility for sending the final answer. The objectives are to reduce search time and preserve or improve resolution quality.

Analysis

The company audits six months of anonymised tickets and discovers that documentation is strong for common configuration questions but weak for billing disputes and account-security incidents. Those categories require restricted handling. Historical resolution labels are inconsistent, so a sample is independently reviewed by trained adjudicators.

The team defines evaluation measures: median time to first useful draft, citation correctness, resolution rate, escalation accuracy, agent acceptance rate, customer recontact within seven days and error rates for sensitive categories.

Approach and solution

The first version uses retrieval-augmented generation. The system retrieves approved documents according to the agent’s permissions, presents source passages and generates a draft constrained to those passages. It refuses to answer when retrieval confidence is low or when the issue belongs to a restricted category. The interface clearly labels generated text as a draft.

A rules-based baseline and an internal keyword search tool are evaluated alongside the AI assistant. A staged rollout begins with a small group of trained agents. Human review is mandatory, and feedback is captured through structured error categories rather than an undifferentiated thumbs-up button.

Outcome

In this illustrative scenario, the assistant reduces median document-search time by 30% and improves draft speed, but it initially produces unsupported claims when retrieved documents are incomplete. The team responds by adding citation checks, improving document ownership and expanding refusal conditions. The product is judged successful only after sensitive-category error rates remain below the pre-agreed threshold and customer recontact does not increase.

Learning points

  1. The valuable product was agent assistance, not unrestricted automation.
  2. Retrieval quality and documentation governance were as important as model choice.
  3. Human review reduced risk but did not eliminate the need for monitoring.
  4. Success required product, operational and safety metrics together.
  5. A clearly defined fallback was a feature, not a failure.

11. Ethical, Regulatory and Governance Considerations

AI products can reproduce historical discrimination, expose personal information, create unsafe recommendations or make decisions that users cannot contest. Governance should be proportionate to risk but present throughout the lifecycle.

Key controls include:

  • Purpose limitation: Use data for defined, legitimate purposes.
  • Data minimisation: Collect and retain only what is necessary.
  • Access control: Restrict sensitive data and model tools according to role.
  • Transparency: Inform users when they are interacting with AI or receiving generated content where relevant.
  • Human oversight: Provide meaningful review for consequential decisions.
  • Auditability: Record model versions, data versions, prompts, outputs and decisions where appropriate and lawful.
  • Fairness assessment: Compare error patterns across relevant groups, recognising that fairness definitions may conflict.
  • Security: Address prompt injection, data exfiltration, model abuse and supply-chain risks.
  • Intellectual property and provenance: Assess rights to training, retrieved and generated content.
  • Contestability: Allow affected people to seek explanation, correction or review when appropriate.

Regulatory requirements vary by jurisdiction and application. Product managers should involve legal, privacy, security, compliance and domain experts early, especially for employment, credit, health, education, public services and safety-critical use cases. Compliance is not a substitute for responsible design; a formally compliant product can still be confusing, harmful or untrustworthy.

12. Contemporary Developments and Strategic Implications

AI product capabilities are increasingly delivered through foundation models, APIs, open-weight models, specialised small models and multimodal systems. Product teams are therefore making architectural choices about build versus buy, model routing, privacy boundaries, fine-tuning, retrieval and tool use.

Several strategic patterns are important:

  • Model commoditisation: Basic capabilities may become cheaper and widely available, shifting differentiation toward proprietary data, workflow integration, trust and distribution.
  • Small and specialised models: Smaller models can offer lower latency, lower cost and improved control for bounded tasks.
  • Multimodality: Systems increasingly combine text, images, audio, video and structured data.
  • AI agents: Tool-using systems can perform multi-step tasks, but autonomy increases the need for permissions, confirmations, observability and rollback.
  • Evaluation as a product capability: Static benchmarks are insufficient for changing, domain-specific systems. Teams need living test sets, adversarial cases and production feedback loops.
  • Human-AI collaboration: The strongest design may allocate different tasks to people and models rather than replacing one with the other.

The strategic discipline is to match capability with consequence. Low-risk drafting may tolerate occasional correction; financial or medical decisions require stronger evidence, controls and accountability.

Best-Practice Checklist

Before approving an AI initiative, ask:

  1. What specific user problem and measurable outcome justify the system?
  2. Is AI better than a simpler alternative?
  3. What data supports the task, and who has the right to use it?
  4. What are the important error types and who bears their cost?
  5. What level of uncertainty can users safely tolerate?
  6. What is the baseline, and how will incremental value be measured?
  7. How will the system behave when confidence is low or data are missing?
  8. What human review, permissions and audit records are required?
  9. How will drift, abuse and quality degradation be detected?
  10. Who owns the product after launch?

Chapter Summary

AI-driven products use computational systems to infer, predict, rank, decide or generate outputs. Machine learning learns patterns from data; deep learning uses multi-layer neural networks; NLP processes language; recommendation systems select or rank items; and generative AI creates new content conditioned on prompts, context or tools.

The defining product-management distinction is between deterministic software, whose behaviour is explicitly specified, and probabilistic AI, whose outputs are learned estimates subject to uncertainty. AI requirements must therefore include performance, error trade-offs, confidence handling, monitoring and fallback behaviour.

Supervised, unsupervised and reinforcement learning support different product problems. Metrics such as precision, recall, MAE and MSE are useful only when connected to business consequences and representative evaluation data. Recommendation systems require attention to cold start, feedback loops and long-term outcomes. Generative AI requires controls for factuality, grounding, privacy, security and human oversight.

A feasible AI use case combines meaningful user value, suitable data, technical and economic feasibility, measurable outcomes and manageable risk. Successful AI product management includes baseline comparison, careful label and data design, offline and online evaluation, lifecycle monitoring, governance and responsible handling of uncertainty.

Key Terms

Artificial intelligence: Computational capability associated with tasks such as learning, reasoning, perception or language processing.

Machine learning: A method in which systems learn patterns or functions from data or interaction.

Deep learning: Machine learning based on multi-layer neural networks.

Supervised learning: Learning from examples containing inputs and known targets.

Unsupervised learning: Learning patterns from data without predefined target labels.

Reinforcement learning: Learning actions through interaction and rewards.

Feature: An input variable supplied to a model.

Label: The target outcome used in supervised learning.

Inference: Applying a trained model to new data.

Overfitting: Learning training-specific patterns that do not generalise well.

Precision: The proportion of predicted positive cases that are actually positive.

Recall: The proportion of actual positive cases that are detected.

Natural language processing: Computational processing and generation of human language.

Embedding: A learned numerical representation of an entity or piece of content.

Recommendation system: A system that selects or ranks items for users.

Cold-start problem: Difficulty making recommendations with limited history for a new user or item.

Generative AI: AI that produces new content such as text, images, audio, code or video.

Hallucination: A plausible-looking but unsupported or incorrect generated output.

Data drift: Change in input data or relationships that may reduce model performance.

Retrieval-augmented generation: Generation supported by retrieving relevant external or organisational content.

Review Questions

Short Answer Questions

  1. What is the difference between machine learning and deep learning?
  2. Why can accuracy be misleading for an imbalanced classification problem?
  3. Define precision and recall.
  4. What is the cold-start problem in recommendation systems?
  5. Why does a generative AI product require more than a foundation model?

Long Answer Questions

  1. Compare deterministic software and probabilistic AI across behaviour, testing, requirements and failure management.
  2. Explain supervised, unsupervised and reinforcement learning, giving one product example for each.
  3. Describe the stages of a recommendation-system pipeline and discuss the risks of optimising clicks alone.
  4. Explain how data quality, labels, representativeness and drift affect AI product performance.
  5. Discuss the ethical, privacy, security and governance issues that should be addressed before launching a generative AI product.

Analytical / Application Questions

  1. A fraud model identifies 900 transactions as suspicious. Of these, 720 are genuinely fraudulent. There are 800 fraudulent transactions in total. Calculate precision and recall, and explain which operational consequences each metric represents.
  2. A university proposes an AI system to automatically reject scholarship applications predicted to be unlikely to succeed. Analyse the use case using value, data, uncertainty, fairness, governance and human-oversight criteria. Recommend a safer product design if appropriate.
  3. A commerce platform’s recommendation model increases click-through rate but reduces repeat purchases and increases complaints about repetitive content. Diagnose possible causes and propose a revised evaluation and product strategy.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes probabilistic AI from deterministic software?

A. Probabilistic AI always produces random outputs.

B. Probabilistic AI estimates outputs or distributions from data and may express uncertainty.

C. Deterministic software cannot process numerical data.

D. Deterministic software always produces correct results.

Correct Answer: B. Probabilistic AI estimates outputs or distributions from data and may express uncertainty.

Question 2

A classifier has TP = 80 and FP = 20. What is its precision?

A. 20%

B. 50%

C. 80%

D. 90%

Correct Answer: D. 90%

Question 3

Which learning paradigm uses labelled examples containing target outcomes?

A. Supervised learning

B. Unsupervised learning

C. Reinforcement learning

D. Random search

Correct Answer: A. Supervised learning

Question 4

What is the primary purpose of retrieval-augmented generation?

A. To remove the need for evaluation

B. To ground generated responses in retrieved relevant information

C. To guarantee that every generated statement is true

D. To replace access controls with prompts

Correct Answer: B. To ground generated responses in retrieved relevant information

Question 5

Which situation illustrates the cold-start problem?

A. A model responds too slowly during peak traffic.

B. A new item has no interaction history for recommendation.

C. A classifier has too many positive labels.

D. A document contains multiple languages.

Correct Answer: B. A new item has no interaction history for recommendation.

Question 6

Why may accuracy be a poor metric for detecting rare fraud?

A. Accuracy cannot be calculated for classification.

B. A model can achieve high accuracy by predicting the common non-fraud class.

C. Accuracy measures only response time.

D. Fraud labels are always numerical.

Correct Answer: B. A model can achieve high accuracy by predicting the common non-fraud class.

Question 7

Which is an example of unsupervised learning?

A. Predicting delivery time from historical labelled deliveries

B. Classifying emails as spam using reviewed examples

C. Grouping customers by behavioural similarity without predefined labels

D. Learning a game policy from rewards

Correct Answer: C. Grouping customers by behavioural similarity without predefined labels

Question 8

What is model drift?

A. A change in data or relationships that can reduce model performance

B. A deliberate increase in model size

C. A method for encrypting training data

D. A user interface redesign

Correct Answer: A. A change in data or relationships that can reduce model performance

Question 9

Which product requirement is most appropriate for a high-risk AI classifier?

A. The model must be intelligent.

B. The model must never make an error.

C. The model must meet specified error, review, latency and escalation constraints.

D. The model must use the newest available algorithm.

Correct Answer: C. The model must meet specified error, review, latency and escalation constraints.

Question 10

Why is a non-AI baseline valuable in AI product development?

A. It proves that AI is unnecessary in every case.

B. It provides a comparison for judging whether AI creates incremental value.

C. It eliminates the need for test data.

D. It guarantees regulatory compliance.

Correct Answer: B. It provides a comparison for judging whether AI creates incremental value.

Digital Product Management & AI-Driven Products
Chapter 9 of 14