Domain AI and Generative Models

Introduction and Context

Artificial intelligence is increasingly built as a collection of domain capabilities rather than as a single general-purpose program. Natural language processing (NLP), computer vision, speech technology, recommender systems and generative models address different forms of data, interaction and decision-making. Although these domains use related machine-learning principles, each has distinct representations, evaluation methods, failure modes and governance requirements.

Recent progress has been driven by deep neural networks, large-scale datasets, specialised hardware and transformer-based architectures. Foundation models are trained on broad data and adapted to many downstream tasks. Large language models (LLMs) generate and analyse text, while multimodal models process combinations of text, images, audio and video. These systems create powerful applications but also introduce risks involving hallucination, privacy, bias, copyright, security and accountability.

This chapter examines the technical foundations and practical use of these systems. It treats generative AI as an engineered component that must be evaluated, monitored and governed—not merely as a conversational interface.

Learning Objectives

After completing this chapter, a learner should be able to:

1. Explain the purposes and main architectures of NLP, computer vision, speech and recommender systems.
2. Distinguish discriminative, generative, foundation and large language models.
3. Describe tokenisation, embeddings, attention, prompting and retrieval-augmented generation.
4. Explain how multimodal systems combine information from different data types.
5. Select appropriate evaluation measures and identify important limitations.
6. Design a responsible domain-AI solution for a practical organisational problem.

1. Domain AI: Data, Representation and Task

A domain-AI system maps observations to predictions, decisions or generated outputs. An observation may be a sentence, image, waveform, user history or combination of modalities. The first design question is therefore: what information must be represented, and what task must be performed?

Common task types include:

• Classification: assigning a category, such as identifying whether an email is fraudulent.
• Regression: predicting a numerical quantity, such as demand.
• Ranking: ordering items, such as search results or products.
• Sequence modelling: predicting or transforming ordered data, such as words or speech frames.
• Generation: producing a new text, image, audio signal or structured object.
• Retrieval: locating relevant records from a collection.
• Decision and control: selecting actions subject to objectives and constraints.

A representation converts raw data into a form that a model can use. Traditional systems relied on manually designed features; modern deep-learning systems learn representations through training. Nevertheless, data quality, labels, domain knowledge and task definition remain decisive. A sophisticated model cannot reliably correct a poorly specified objective or systematically biased data.

Discriminative and Generative Models

A discriminative model estimates a decision boundary or conditional relationship such as P(y|x), where x is an input and y is a label. A generative model represents how data may be produced. It may estimate P(x) or P(x|c), where c is a condition such as a text prompt.

For classification, a common loss is cross-entropy:

Here, K is the number of classes, y_k is 1 for the correct class and 0 otherwise, and ŷ_k is the predicted probability for class k. This formula assumes that predictions are valid probabilities and is generally applied independently to training examples, often by averaging their losses. It penalises confident incorrect predictions more strongly than uncertain predictions.

2. Natural Language Processing

NLP enables computers to process human language in written or spoken form. Important tasks include text classification, named-entity recognition, sentiment analysis, information extraction, machine translation, question answering, summarisation and dialogue.

From Tokens to Contextual Representations

A computer does not directly operate on words as humans experience them. Text is divided into tokens. A token may be a word, subword or character sequence. Subword tokenisation is useful because it represents unfamiliar words by combining smaller units, although token counts depend on the tokenizer and language.

Each token is mapped to a vector called an embedding. Contextual models produce a representation whose meaning depends on surrounding tokens. Thus, the representation of “bank” can differ in “river bank” and “commercial bank.”

A transformer uses attention to relate tokens to one another. In scaled dot-product attention:

Q contains queries, K keys, and V values. d_k is the dimensionality of the key vectors. The transpose produces pairwise query–key scores; division by √d_k controls the scale of these scores; softmax converts each row into weights that sum to 1. The formula assumes compatible matrix dimensions and is applied to a sequence, often with masking so that a decoder cannot use future tokens.

A language model estimates the probability of a token sequence using the chain rule:

The model generates one token at a time by selecting or sampling from the next-token distribution. Temperature changes the sharpness of this distribution, but a higher temperature does not guarantee greater factual accuracy.

NLP Applications and Evaluation

For classification, accuracy can be misleading when classes are imbalanced. Precision and recall are defined as:

TP denotes true positives, FP false positives and FN false negatives. Precision answers “of the predicted positives, how many were correct?” Recall answers “of the actual positives, how many were found?” The appropriate balance depends on consequences. In medical screening, missing a condition may be more serious than creating additional referrals.

Text-generation evaluation requires multiple methods. Token-overlap measures such as ROUGE can help compare summaries, but they do not fully assess factuality, usefulness or style. Human evaluation, expert review, task success, groundedness and safety testing are often necessary.

3. Computer Vision

Computer vision analyses images and video. Core tasks include image classification, object detection, semantic segmentation, instance segmentation, optical character recognition and image generation.

A convolutional neural network (CNN) applies learnable filters across local regions. Early layers may detect edges and textures; deeper layers combine these into shapes and objects. Vision transformers instead divide an image into patches, represent patches as tokens and apply attention. Hybrid systems are also common.

Object detection predicts both a class and a bounding box. The intersection over union (IoU) of predicted box A and reference box B is:

IoU lies between 0 and 1 for ordinary non-empty boxes. It measures overlap, not whether the category is correct. Detection benchmarks commonly count a detection as correct only when its class is correct and IoU exceeds a selected threshold.

Segmentation assigns labels at pixel level. In medical or industrial applications, performance should be measured separately for important classes because a high overall pixel accuracy can conceal poor performance on small but critical objects.

Computer-vision systems are sensitive to lighting, camera position, occlusion, image quality and distribution shift. A model trained on clear daytime images may fail at night or when a camera is replaced. Data augmentation, calibration, representative testing and human escalation are therefore central engineering practices.

4. Speech and Audio Systems

Speech systems contain several related tasks:

• Automatic speech recognition (ASR): converts speech into text.
• Text-to-speech (TTS): converts text into a spoken waveform.
• Speaker identification: determines which enrolled speaker produced an utterance.
• Speaker verification: tests whether speech matches a claimed identity.
• Speech understanding: extracts intent, entities and conversational meaning.
• Audio event detection: identifies sounds such as alarms or machinery faults.

An ASR pipeline may include voice activity detection, acoustic representation, an encoder–decoder model and language decoding. Modern systems often use self-supervised learning on large quantities of unlabelled audio, followed by fine-tuning on transcribed speech.

Word error rate is a common ASR measure:

S is the number of substitutions, D deletions, I insertions and N the number of reference words. WER is a count ratio and may exceed 1 when many insertions occur. It should be reported by speaker group, accent, acoustic environment and language where fairness or operational reliability matters.

Speech technologies raise distinctive concerns. Voice can contain biometric information, private conversations and emotional or health-related signals. Consent, retention limits, access controls and disclosure that a synthetic voice is being used are important safeguards.

5. Recommender Systems

Recommender systems help users discover products, media, documents or actions. They may use:

• Content-based filtering: recommends items similar to those with features preferred by the user.
• Collaborative filtering: uses patterns of interaction among users and items.
• Hybrid recommendation: combines content, collaborative and contextual signals.
• Context-aware recommendation: incorporates time, location, device or situation.

In matrix factorisation, an interaction matrix R is approximated by user and item latent factors:

R may contain ratings or implicit interactions, U represents users and V represents items. The dimensions must be compatible. For explicit ratings, a typical optimisation objective is:

Here, Ω is the set of observed user–item pairs, r_ui is an observed rating, u_u and v_i are latent vectors, and λ ≥ 0 controls L2 regularisation. The objective assumes squared error is a suitable loss and does not by itself solve cold-start, popularity bias or fairness problems.

Recommenders should be evaluated beyond click-through rate. Useful measures include precision@k, recall@k, normalized discounted cumulative gain, coverage, diversity, novelty, user satisfaction and long-term outcomes. Optimising clicks alone may promote sensational content, reduce diversity or create harmful feedback loops.

6. Foundation Models and Large Language Models

A foundation model is trained on broad data at scale and adapted to many tasks. It is not necessarily universally capable, unbiased or safe. Adaptation methods include supervised fine-tuning, instruction tuning, parameter-efficient fine-tuning, retrieval augmentation and tool use.

An LLM is generally a transformer-based language model trained to predict tokens. Its capabilities may include summarisation, code generation, translation, extraction and question answering. These capabilities arise from learned statistical structure, not from guaranteed understanding or a built-in database of verified facts.

Embeddings and Vector Search

An embedding is a numerical vector representing an item such as a sentence, document, image or audio segment. Semantically related items may be close in vector space, depending on training and the embedding model.

Cosine similarity is commonly used:

Vectors a and b must be non-zero. The dot product is divided by their Euclidean norms, so the result is independent of overall vector magnitude. Values near 1 indicate similar direction, 0 orthogonality and −1 opposite direction, although practical embedding spaces may not use the full range meaningfully.

A vector database stores embeddings and supports approximate nearest-neighbour search. A production retrieval system also needs metadata filtering, access control, versioning, chunking strategy, duplicate handling and evaluation of retrieved evidence. Similarity is not identical to truth: a semantically similar document can still be outdated or incorrect.

Prompt Engineering

Prompt engineering is the deliberate design of instructions and context supplied to a model. Effective prompts commonly specify:

1. The role or task.
2. Relevant context and source boundaries.
3. Constraints, audience and desired format.
4. Examples where the output pattern is difficult.
5. A verification or uncertainty instruction.

For example, a support prompt may request: “Using only the supplied policy excerpts, classify the request as eligible or ineligible, quote the supporting clause, and state ‘insufficient evidence’ when no clause applies.” This is safer than requesting an unrestricted answer.

Zero-shot prompting provides instructions without examples. Few-shot prompting provides examples. Structured output, such as a JSON schema, improves integration but does not guarantee valid or truthful content. System prompts are not a security boundary; sensitive actions require application-level authorisation and validation.

7. Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) combines search with generation. A typical pipeline is:

1. Ingest and clean source documents.
2. Split them into meaningful passages.
3. Create embeddings and index passages.
4. Retrieve candidates for a user query.
5. Optionally rerank candidates using a stronger relevance model.
6. Construct a prompt containing selected evidence.
7. Generate an answer with citations or source references.
8. Validate, log and monitor the result.

RAG can reduce reliance on a model’s stale parameters and can expose organisational knowledge without retraining the entire model. It does not eliminate hallucination. Failures occur when the answer is not in the collection, retrieval misses the relevant passage, access permissions are ignored, documents conflict or the model misinterprets evidence.

A useful evaluation separates retrieval from generation. Retrieval recall asks whether relevant evidence appears among the top results. Answer evaluation examines correctness, completeness, citation support, refusal behaviour and latency. Test sets should include ordinary queries, ambiguous queries, adversarial instructions, outdated documents and unauthorised requests.

RAG also creates a prompt-injection risk. A retrieved document may contain instructions intended to manipulate the model. Retrieved text must be treated as data, not automatically as an instruction. Tools and database actions should be mediated by explicit policies, least privilege and deterministic validation.

8. Multimodal AI

Multimodal AI processes two or more modalities, such as text and images or audio and video. Three broad design strategies are common:

• Early fusion: combine representations before substantial task processing.
• Late fusion: process modalities separately and combine decisions or representations later.
• Cross-modal attention: allow one modality to attend to another, such as text tokens attending to image regions.

Contrastive learning can align representations from different modalities. Given a batch of matched image–text pairs, the training objective encourages the matched pair to have higher similarity than mismatched pairs. Such models support image search, visual question answering and zero-shot classification.

Multimodal systems may fail through modality imbalance, poor alignment, missing information or incorrect visual grounding. An image-language model can produce a fluent answer that is not supported by the image. Evaluation should therefore test grounding: whether claims correspond to identifiable content in the relevant modality.

Applications include accessibility tools, industrial inspection, document understanding, clinical image assistance, educational tutoring and creative production. High-impact applications require domain experts, traceable evidence and clear limits on autonomous action.

9. Generative Model Families

Different generative architectures have different strengths:

Generative quality has several dimensions: fidelity to the input or condition, diversity, coherence, controllability, factuality and safety. A visually realistic output may be factually wrong; a fluent answer may be unsupported; a diverse generator may reproduce harmful stereotypes.

10. A Practical Architecture for Domain AI

A robust implementation separates model capability from application responsibility. A typical architecture contains:

• Data layer: governed documents, images, audio, interaction records and metadata.
• Representation layer: tokenisers, embeddings, vision encoders and speech features.
• Model layer: classifiers, rankers, foundation models or generative models.
• Orchestration layer: prompts, retrieval, routing, tool calls and workflow logic.
• Control layer: identity, authorisation, content filtering, validation and audit logs.
• Evaluation layer: offline test sets, red-team tests, human review and production monitoring.
• User layer: explanations, citations, feedback and escalation to a human.

The choice between a conventional model, a fine-tuned model and RAG should follow the problem. RAG is appropriate when answers must reflect changing private documents. Fine-tuning is useful when the desired behaviour or format must be learned consistently. A conventional supervised model may be preferable for a narrow, stable classification task because it is cheaper and easier to validate.

11. Worked Example: Evaluating a Support Classifier

Suppose a support classifier labels 100 requests as urgent or non-urgent. It identifies 18 urgent cases correctly, incorrectly labels 6 non-urgent cases as urgent, and misses 2 urgent cases. Therefore TP = 18, FP = 6 and FN = 2.

The classifier finds 90% of urgent cases, but only 75% of its urgent predictions are correct. If missing an urgent case is substantially more costly than creating an unnecessary escalation, this operating point may be reasonable. The organisation should also measure workload created by false positives, performance across customer groups and performance after distribution shift.

12. Illustrative Case: University Knowledge Assistant

Background

A university wishes to provide students with an assistant that answers questions about enrolment, assessment, fees and academic integrity. The policy collection changes each semester and contains access-controlled documents.

Problem

A general LLM can answer fluently but may use outdated rules, invent deadlines or expose information intended only for staff. The university needs current, cited and appropriately limited answers.

Analysis

The problem is primarily one of governed information access, not simply language generation. A model’s internal knowledge cannot guarantee current policy accuracy. The system must distinguish public policy from restricted records and must recognise when evidence is absent or conflicting.

Approach and Solution

The university creates a versioned document repository with policy owners, effective dates and access labels. Documents are split into passages while preserving section headings and references. An embedding index supports semantic retrieval, and a metadata filter enforces user permissions before passages reach the model. A reranker improves selection for precise policy questions.

The assistant prompt requires answers to use only retrieved passages, cite policy identifiers and effective dates, state uncertainty when evidence is insufficient and direct students to a human office for exceptional cases. It cannot change enrolment records or issue disciplinary decisions. Such actions require authenticated workflows and human approval.

Evaluation uses a curated question set containing ordinary, ambiguous, outdated and adversarial questions. Reviewers score factual correctness, citation support, completeness, refusal quality and fairness across student groups. Logs record document versions, retrieval results, model version and final response, while excluding unnecessary personal information.

Outcome

In this illustrative scenario, the assistant reduces routine enquiries while routing exceptional cases to staff. Its success is not measured only by response fluency. The decisive measures are supported-answer rate, stale-policy rate, inappropriate disclosure rate, escalation appropriateness and student satisfaction.

Learning Points

1. RAG improves access to current information only when retrieval and document governance are sound.
2. Authorisation must occur outside the language model.
3. Citations and refusal behaviour are useful controls but require testing.
4. Domain experts must define acceptable risk and escalation conditions.
5. Monitoring must continue after deployment because policies, users and attacks change.

13. Limitations, Ethics and Governance

Reliability and Robustness

Generative models can hallucinate, overgeneralise and fail unpredictably on rare inputs. Benchmark performance may not represent local users or operational conditions. Models should be tested on realistic, representative and adversarial cases, with confidence or uncertainty handled carefully rather than inferred from fluent wording.

Bias and Fairness

Bias may enter through historical data, sampling, labels, measurement, model behaviour or deployment context. Fairness cannot be reduced to one universal metric. Teams should identify affected groups, select context-appropriate measures, involve stakeholders and assess both error rates and downstream consequences.

Privacy and Security

Training and inference may expose personal or confidential information. Controls include data minimisation, consent where required, retention limits, encryption, access management, redaction and secure logging. Threats include prompt injection, data poisoning, model extraction, membership inference and insecure tool use.

Transparency, Copyright and Accountability

Users should know when they interact with an automated system and when content is synthetic, subject to applicable law and sector rules. Organisations should document data provenance, model versions, evaluation results, intended use, prohibited use and responsibility for decisions. Copyright and licensing issues require review of training data, retrieved materials and generated outputs.

Environmental and Economic Considerations

Large models can require substantial energy, hardware and operational cost. A smaller model, retrieval system or conventional classifier may meet requirements more efficiently. Model selection should consider total cost of ownership, latency, accessibility, maintenance and environmental impact—not only benchmark scores.

14. Best Practices for Professional Implementation

1. Define the task, user, decision authority and harm boundaries before selecting a model.
2. Establish a representative, versioned evaluation set before deployment.
3. Separate retrieval, generation, validation and authorisation components.
4. Prefer grounded answers and explicit uncertainty for knowledge-intensive applications.
5. Use least-privilege tool access and validate all model-generated actions.
6. Measure subgroup performance, robustness, latency, cost and safety.
7. Keep human review for high-impact, irreversible or contested decisions.
8. Monitor drift, policy changes, user feedback, incidents and attack patterns.
9. Document data provenance, model limitations and change history.
10. Provide users with correction, appeal and escalation mechanisms.

Contemporary Developments

Current research is moving toward smaller efficient models, mixture-of-experts architectures, long-context processing, multimodal agents, synthetic-data workflows, on-device inference and retrieval systems that combine vectors with symbolic or graph-based search. Tool-using agents can plan and execute multi-step tasks, but increased autonomy also increases the need for permissions, observability and interruption mechanisms.

Another important direction is evaluation beyond static benchmarks. Researchers increasingly study real-world robustness, calibration, data contamination, model editing, provenance, interpretability and societal impact. The central professional challenge is to convert impressive general capability into reliable domain performance under explicit constraints.

Chapter Summary

Domain AI applies machine learning to language, vision, speech, recommendation and multimodal data. NLP systems represent token sequences and use attention to model context; computer-vision systems analyse spatial structure; speech systems transform between acoustic signals and language; recommender systems rank items using content, collaborative and contextual evidence. Foundation models generalise across tasks, while LLMs generate sequences through next-token prediction.

Embeddings support semantic comparison and vector retrieval. Prompt engineering specifies task, context and output constraints. RAG connects generation to governed, current evidence, but requires careful chunking, retrieval evaluation, access control and protection against prompt injection. Multimodal models align information across text, images, audio and video, yet require grounding evaluation.

Successful professional deployment depends on more than model quality. It requires representative evaluation, human oversight, privacy and security controls, fairness analysis, provenance, monitoring and clear accountability. The most appropriate solution may be a conventional model, a foundation model, RAG or a hybrid architecture, selected according to risk, data, cost and operational requirements.

Key Terms

Attention: A mechanism that weights relationships among elements in a sequence or across modalities.

Automatic speech recognition: Conversion of spoken audio into text.

Embedding: A vector representation of an item in a learned semantic space.

Foundation model: A broadly trained model adapted to multiple downstream tasks.

Generative model: A model that learns to produce new data or sequences.

Hallucination: An unsupported or incorrect generated claim presented as an answer.

Intersection over union: The area of overlap divided by the area of union between two regions.

Large language model: A large neural language model, commonly transformer-based, that predicts and generates token sequences.

Multimodal AI: AI that processes or generates more than one data modality.

Natural language processing: Computational processing of human language.

Prompt engineering: Designing instructions and context for a model.

RAG: Retrieval-augmented generation, combining information retrieval with generation.

Recall: The proportion of actual positive cases correctly identified.

Recommender system: A system that predicts or ranks items for users or contexts.

Semantic search: Retrieval based on meaning represented in vectors rather than exact words alone.

Token: A unit of text processed by a language model.

Transformer: A neural architecture based substantially on attention mechanisms.

Vector database: A system that stores and searches numerical representations.

Review Questions

Short Answer Questions

1. What is the difference between a discriminative model and a generative model?
2. Why are embeddings useful in semantic search?
3. Define precision and recall and explain when recall may be prioritised.
4. What are the principal stages of a RAG pipeline?
5. Why must authorisation be implemented outside a language model?

Long Answer Questions

1. Explain how transformer attention supports contextual language representation. Include the role of queries, keys, values and scaling.
2. Compare content-based, collaborative and hybrid recommender systems, including their strengths and limitations.
3. Discuss the main evaluation challenges for computer-vision systems deployed under distribution shift.
4. Explain how foundation models can be adapted through prompting, fine-tuning and retrieval augmentation.
5. Analyse the ethical, privacy and security risks of speech and multimodal AI in an organisational setting.

Analytical / Application Questions

1. Design a RAG architecture for a regulated organisation. Specify document governance, retrieval, authorisation, evaluation and escalation controls.
2. A classifier produces TP = 45, FP = 15 and FN = 5. Calculate precision and recall, then recommend whether the operating threshold should change if missed positives are very costly.
3. A recommender increases click-through rate but decreases content diversity and causes complaints from a minority user group. Propose a revised evaluation and governance framework that addresses both utility and harm.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes retrieval-augmented generation from ordinary prompting?

A. RAG changes the model’s neural weights during every query.

B. RAG retrieves external evidence and supplies it as context for generation.

C. RAG converts all text into speech before answering.

D. RAG guarantees that every generated statement is true.

Correct Answer: B. RAG retrieves external evidence and supplies it as context for generation.

Question 2

In the attention expression softmax(QKᵀ / √d_k)V, what is the principal purpose of dividing by √d_k?

A. To convert tokens into images

B. To make all values equal to zero

C. To control the scale of query–key scores

D. To remove the need for training data

Correct Answer: C. To control the scale of query–key scores

Question 3

A system has TP = 18, FP = 6 and FN = 2. What is its recall?

A. 0.60

B. 0.75

C. 0.80

D. 0.90

Correct Answer: D. 0.90

Question 4

Which measure evaluates overlap between a predicted and reference bounding box?

A. Word error rate

B. Intersection over union

C. Cosine temperature

D. Click-through rate

Correct Answer: B. Intersection over union

Question 5

What does an embedding primarily provide?

A. A numerical representation useful for similarity comparison

B. A legal guarantee of data ownership

C. A deterministic proof that a document is correct

D. A replacement for access control

Correct Answer: A. A numerical representation useful for similarity comparison

Question 6

Which recommender approach uses patterns of interaction among users and items?

A. Collaborative filtering

B. Optical character recognition

C. Speech synthesis

D. Image segmentation

Correct Answer: A. Collaborative filtering

Question 7

What does word error rate measure in automatic speech recognition?

A. The energy consumed by a microphone

B. Substitutions, deletions and insertions relative to reference words

C. The number of speakers in an audio file

D. The visual quality of generated images

Correct Answer: B. Substitutions, deletions and insertions relative to reference words

Question 8

Which practice is most appropriate for model-generated tool actions?

A. Allowing unrestricted access because prompts are trusted

B. Removing all logs to protect privacy

C. Using least privilege and deterministic validation

D. Treating retrieved instructions as authoritative commands

Correct Answer: C. Using least privilege and deterministic validation

Question 9

What is a major limitation of increasing temperature during language generation?

A. It guarantees factual accuracy.

B. It may increase variation but does not guarantee truthfulness.

C. It prevents tokenisation.

D. It converts a generative model into a classifier.

Correct Answer: B. It may increase variation but does not guarantee truthfulness.

Question 10

Which design combines information from more than one data type, such as text and images?

A. Multimodal AI

B. Single-variable regression

C. Content-only filtering

D. Binary tokenisation

Correct Answer: A. Multimodal AI