Generative AI and Foundation Models

Introduction and Context

Generative artificial intelligence (AI) refers to systems that produce new content—such as text, images, audio, video, software code, or scientific structures—rather than merely classifying or retrieving existing data. Recent generative systems are commonly built as foundation models: large models trained on broad, diverse datasets and adapted to many downstream tasks.

Large language models (LLMs), including transformer-based systems, have made natural-language generation widely accessible. They can summarize documents, answer questions, generate code, translate languages, and support decision-making. However, fluent output is not equivalent to factual accuracy, understanding, or trustworthy reasoning. Effective use therefore requires knowledge of model architecture, training, prompting, retrieval, evaluation, governance, and human oversight.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain foundation models, LLMs, tokens, embeddings, and transformers.

2. Describe pretraining, instruction tuning, alignment, and inference.

3. Compare major generative architectures.

4. Design effective prompts and evaluate generated output.

5. Explain retrieval-augmented generation (RAG) and fine-tuning.

6. Describe multimodal AI and its applications.

7. Analyze hallucination, reliability, grounding, copyright, privacy, and governance risks.

8. Recommend responsible deployment practices for professional applications.

1. Foundation Models and Generative Modeling

A foundation model is a large model trained on broad data at scale and subsequently adapted for multiple tasks. The term emphasizes reuse: the same underlying model may support question answering, classification, summarization, coding, or image analysis.

A generative model estimates a distribution over possible data. For text, an autoregressive language model estimates the probability of a sequence of tokens:

P(x₁, x₂, …, xₙ) = ∏ᵢ₌₁ⁿ P(xᵢ | x₁, …, xᵢ₋₁).

Here, xᵢ is the token at position i. The equation expresses the chain rule of probability: the probability of a complete sequence is the product of each token’s conditional probability given preceding tokens. Autoregressive generation predicts one token at a time. The formula describes the model objective conceptually; practical systems use approximations, finite context windows, and numerical optimization.

A token is a unit processed by a language model. It may be a complete word, part of a word, punctuation mark, or character sequence. Tokenization affects cost, context capacity, multilingual performance, and handling of technical vocabulary. An embedding is a numerical vector representing a token, sentence, image, or other object in a learned space. Similar meanings often produce nearby vectors, although similarity is learned from data and is not a guarantee of truth.

Foundation models differ from traditional task-specific models in scale, breadth, and adaptability. Their generality is useful, but it also makes behavior difficult to predict and evaluate across all contexts.

2. Transformer Architecture

Most modern LLMs use the transformer architecture. Its central mechanism is self-attention, which allows each token representation to incorporate information from other tokens in a sequence.

For query matrix Q, key matrix K, and value matrix V, scaled dot-product attention is:

Attention(Q, K, V) = softmax(QKᵀ / √dₖ)V.

The division by √dₖ, where dₖ is the key-vector dimension, reduces excessively large dot products and helps maintain stable gradients. Softmax converts scores into non-negative weights that sum to one. Each query therefore forms a weighted combination of value vectors. In multi-head attention, several attention operations learn different relationships, such as syntax, reference, or positional patterns.

A transformer commonly includes attention layers, feed-forward networks, residual connections, normalization, and positional information. Decoder-only transformers use causal masking so that a token cannot attend to future tokens during autoregressive training. Encoder–decoder transformers contain an encoder for an input sequence and a decoder for output generation; they are useful for tasks such as translation.

Transformers are powerful because they process relationships in parallel during training and can represent long-range dependencies better than many earlier recurrent architectures. Their limitations include substantial computational cost, context-window constraints, sensitivity to data quality, and imperfect reasoning.

3. Training a Large Language Model

3.1 Pretraining

During pretraining, a model is exposed to large collections of text, code, or other data. In causal language modeling, the model predicts the next token. A common loss is cross-entropy:

L = −∑ᵢ log pθ(xᵢ | x₍<ᵢ₎),

where θ denotes model parameters, x₍<ᵢ₎ denotes preceding tokens, and pθ is the model’s predicted probability. Training seeks parameters that reduce average loss over the training data. Lower loss generally indicates better prediction of the training distribution, but it does not by itself guarantee factuality, fairness, safety, or usefulness.

Pretraining requires data collection, filtering, deduplication, tokenization, distributed optimization, hardware, monitoring, and evaluation. Data may include public websites, books, code, licensed material, and human-created datasets. Data provenance and legal permissions are important because training data can contain personal information, copyrighted works, errors, and social bias.

3.2 Adaptation and Alignment

A pretrained model can continue training on a specialized dataset. Fine-tuning updates some or all parameters using task-specific examples. Instruction tuning trains on prompts paired with desired responses, improving compliance with user instructions. Human-feedback methods and preference optimization attempt to make outputs more helpful, harmless, and aligned with stated objectives.

Alignment is not identical to truth. A model may be polite, safe-sounding, and instruction-following while still producing an unsupported claim. Alignment also reflects the values and judgments embedded in data, reward models, policies, and evaluation procedures.

3.3 Inference and Decoding

At inference time, the model produces a probability distribution over possible next tokens. Greedy decoding selects the highest-probability token at each step. Temperature modifies the sharpness of the distribution: lower values usually make output more deterministic, while higher values increase diversity. Top-k sampling limits choices to k likely tokens, and top-p sampling chooses from the smallest probability mass whose cumulative probability reaches p. These methods affect style and variability; they do not independently establish correctness.

4. Prompting as Interaction Design

A prompt is an instruction or input supplied to a generative model. Effective prompting is not magic wording; it is specification design. A strong prompt typically identifies:

• the task and intended audience;
• relevant context and source material;
• constraints, such as length, format, or terminology;
• the desired output structure;
• quality criteria and uncertainty requirements;
• examples when the task is difficult or specialized.

Zero-shot prompting gives an instruction without examples. Few-shot prompting includes representative examples. Role framing can clarify perspective, but it does not give the model genuine expertise or authority. Chain-of-thought-style prompting may encourage intermediate reasoning, although generated reasoning should not automatically be treated as a faithful record of internal computation. For high-stakes work, requesting concise evidence, assumptions, and uncertainty is generally more useful than accepting unsupported confidence.

Prompt injection is an attack in which untrusted content attempts to override system or user instructions. For example, a webpage retrieved by an assistant may contain text saying “ignore previous instructions.” Robust systems separate instructions from data, restrict tool permissions, validate outputs, and treat external content as untrusted.

5. Retrieval-Augmented Generation

A standalone LLM answers from patterns encoded during training and from the current prompt. Its knowledge may be outdated, incomplete, or inappropriate for a specific organization. Retrieval-augmented generation (RAG) connects a model to an external knowledge base.

A typical RAG pipeline is:

1. Collect and authorize documents.

2. Clean, segment, and attach metadata to them.

3. Convert passages into embeddings and store them in a vector index.

4. Retrieve passages similar to the user query, often with keyword and semantic search.

5. Place selected passages in the model context.

6. Generate an answer constrained by the retrieved evidence.

7. Cite sources and evaluate both retrieval and generation.

RAG improves freshness and traceability, but it does not guarantee accuracy. Retrieval may return irrelevant, outdated, contradictory, or malicious content. A context window may be too small, and the model may ignore or misinterpret evidence. Evaluation should therefore distinguish retrieval quality from answer quality. Useful measures include precision of retrieved passages, recall of necessary evidence, citation correctness, groundedness, completeness, latency, and cost.

A practical grounding instruction is: “Answer only from the supplied sources. Cite the passage supporting each substantive claim. If the sources do not establish the answer, state that it is unavailable.” This is a control, not a mathematical guarantee.

6. Fine-Tuning and Other Adaptation Methods

Fine-tuning is appropriate when a model must consistently follow a domain style, output schema, classification policy, or specialized task pattern. It is less suitable when the main problem is frequently changing factual knowledge; RAG is usually better for that purpose.

Full fine-tuning updates all or most parameters and can be expensive. Parameter-efficient methods, such as adapters and low-rank adaptation, update a much smaller set of parameters. They reduce storage and computational requirements but still require careful data design and evaluation.

Fine-tuning risks include overfitting, catastrophic forgetting, leakage of sensitive examples, amplification of biased labels, and unwanted memorization. A good dataset contains clear examples, representative edge cases, refusal cases, and explicit output expectations. Train, validation, and test data should be separated, with duplicate or near-duplicate examples controlled. Evaluation must include both task performance and safety behavior.

7. Multimodal Generative AI

Multimodal models process two or more modalities, such as text, images, audio, video, or sensor data. A system may encode an image into a representation, combine it with text representations, and generate a textual or visual response. Applications include document understanding, visual question answering, accessibility tools, medical-image assistance, design, robotics, and speech interfaces.

Multimodality introduces additional failure modes. Image models may misread small text, spatial relationships, or unusual objects. Audio systems may confuse accents or speakers. Generated images can contain distorted anatomy, inconsistent text, or misleading visual evidence. Evaluation must be modality-specific and should test robustness to noise, compression, missing information, and adversarial inputs.

8. Hallucination, Grounding, and Reliability

A hallucination is an output that is unsupported, fabricated, or materially incorrect while appearing plausible. It can arise because the model optimizes likely continuation rather than truth, lacks current information, receives ambiguous instructions, or is asked to answer beyond available evidence.

Hallucination is not limited to invented facts. It includes fabricated citations, incorrect calculations, false quotations, misattributed sources, and confident interpretations of ambiguous images. Fluency is therefore a poor proxy for reliability.

Grounding means linking an output to verifiable evidence, a trusted database, a tool result, or an observed input. Grounding strategies include retrieval, database queries, calculators, code execution, citations, constrained generation, and human review. A grounded system can still fail if the source is wrong or the model draws an invalid conclusion.

Reliability should be evaluated against the intended use case. Relevant tests include factual accuracy, calibration, robustness, consistency, bias, refusal behavior, privacy leakage, and performance on rare but consequential cases. A useful operational principle is to match autonomy to risk: low-risk drafting may require light review, whereas medical, legal, financial, safety, or employment decisions require qualified human oversight and documented controls.

9. Copyright, Privacy, and Governance

Generative AI raises copyright questions at several stages: whether training material was lawfully obtained, whether outputs reproduce protected expression, and whether users have rights to generated or transformed content. The legal status of training and outputs varies across jurisdictions and continues to develop. Organizations should obtain legal advice for material deployments, use licensed or permitted data where possible, retain provenance records, and avoid assuming that “generated” means “free of rights restrictions.”

Privacy risks include exposure of personal data in prompts, memorization of training examples, inference of sensitive attributes, and insecure logging. Users should minimize personal data, apply access controls, encrypt sensitive information, establish retention limits, and prohibit confidential material from unapproved services.

Governance should define approved uses, prohibited uses, accountability, incident reporting, model documentation, vendor requirements, evaluation procedures, and change management. Impact assessments are especially important for systems affecting rights, opportunities, safety, or essential services. Human review must be meaningful: reviewers need authority, time, training, and access to evidence rather than merely approving automated recommendations.

10. Illustrative Case: University Policy Assistant

Background

A university wants an assistant that answers questions about admissions, assessment rules, library services, and student support. The policy documents change each semester, and inaccurate answers could cause academic or financial harm.

Problem

A general LLM produces fluent answers but sometimes cites outdated rules or invents deadlines. Simply fine-tuning it on old policy documents would not reliably solve the freshness problem.

Analysis

The central requirement is current, traceable institutional knowledge. A RAG design is therefore more appropriate than relying only on model memory. Documents must be versioned, approved, segmented, and indexed with metadata such as faculty, effective date, and policy status.

Approach and Solution

The assistant retrieves current passages using hybrid keyword and semantic search. Its prompt requires citations, prohibits unsupported claims, and instructs it to say when evidence is insufficient. Access controls prevent students from seeing restricted staff documents. For high-risk questions—appeals, disability accommodations, disciplinary matters, or financial eligibility—the system directs the student to an authorized human office. Test sets include ordinary questions, outdated-policy traps, ambiguous requests, prompt-injection attempts, and questions for which no answer exists.

Outcome

In this illustrative case, the system improves traceability and reduces outdated answers, but it does not eliminate errors. Staff periodically review failed queries, update documents, monitor citation correctness, and suspend the assistant if a serious incident occurs.

Learning Points

The case demonstrates that model choice is only one component of an AI system. Data governance, retrieval quality, access control, escalation, evaluation, and accountability are equally important. Fine-tuning may improve tone or format, while retrieval supplies changing facts.

11. Contemporary Developments and Best Practices

Current development is moving toward smaller specialized models, multimodal systems, tool-using agents, long-context processing, synthetic data, and on-device inference. Agentic systems can plan and call tools, but increased autonomy expands the consequences of faulty plans, prompt injection, and excessive permissions. Long context does not guarantee that every supplied fact will be used correctly.

Best practices include:

• define the use case, users, risks, and success criteria before selecting a model;
• use authoritative, permissioned, versioned data;
• separate trusted instructions from untrusted content;
• apply least-privilege access to tools and documents;
• require citations or structured evidence where appropriate;
• test normal, adversarial, ambiguous, and out-of-distribution cases;
• log inputs, outputs, retrieved sources, model versions, and reviewer actions subject to privacy rules;
• monitor performance after deployment and provide an incident-response process;
• disclose AI assistance when users may reasonably need to know;
• preserve human accountability for consequential decisions.

Chapter Summary

Generative AI systems produce new content by modeling patterns in data. Foundation models provide reusable capabilities across many tasks, while LLMs commonly use transformer architectures and autoregressive token prediction. Pretraining establishes broad representations; instruction tuning, preference optimization, prompting, retrieval, and fine-tuning adapt behavior for particular uses.

RAG connects generation to external evidence and is valuable for current, domain-specific knowledge, but retrieval and generation must be evaluated separately. Fine-tuning is useful for stable behaviors and formats, whereas changing facts are generally better handled through retrieval. Multimodal models extend generative AI beyond text but introduce modality-specific errors.

Fluency does not establish truth. Hallucination, weak grounding, privacy leakage, copyright uncertainty, bias, prompt injection, and overreliance are central risks. Responsible deployment requires evidence, testing, access control, documentation, monitoring, human oversight, and governance proportionate to the consequences of failure.

Key Terms

Foundation model: A broadly trained model adapted to many downstream tasks.

Generative AI: AI that produces new text, images, audio, code, or other content.

Large language model: A large neural model trained to represent and generate language.

Token: A unit of text processed by a language model.

Embedding: A learned numerical representation of an item in a vector space.

Transformer: A neural architecture based primarily on attention mechanisms.

Self-attention: A mechanism that weights relationships among elements in a sequence.

Pretraining: Broad initial training on large datasets.

Instruction tuning: Training on instructions paired with desired responses.

Fine-tuning: Further training for a specialized task, domain, or behavior.

Inference: Using a trained model to produce an output.

Prompt: User or system input that specifies a task or context.

Hallucination: Unsupported or incorrect output presented as plausible information.

Grounding: Connecting an output to verifiable evidence or tool results.

Retrieval-augmented generation: Generation supported by retrieved external information.

Multimodal AI: AI that processes or generates more than one data modality.

Prompt injection: An attempt by untrusted content to alter model instructions.

Calibration: The relationship between stated confidence and actual correctness.

Parameter-efficient adaptation: Adaptation that updates only a small portion of model parameters.

Model governance: Policies and controls governing responsible model development and use.

Review Questions

Short Answer Questions

1. What distinguishes a foundation model from a traditional task-specific model?

2. What is the purpose of the scaling factor √dₖ in scaled dot-product attention?

3. Why does RAG help with changing institutional information?

4. Define hallucination and give one example.

5. Why is fluency not sufficient evidence of reliability?

Long Answer Questions

1. Explain the stages from pretraining to instruction tuning and inference in an LLM.

2. Compare prompting, RAG, and fine-tuning as methods for adapting a generative AI system.

3. Describe a transformer’s self-attention mechanism and its main practical benefits and limitations.

4. Discuss copyright, privacy, and accountability issues in an organizational generative AI deployment.

5. Explain how a multimodal system can fail differently from a text-only system.

Analytical / Application Questions

1. A company needs an assistant that answers questions from policies updated weekly. Should it rely primarily on fine-tuning, RAG, or both? Justify your design.

2. Design an evaluation plan for an AI assistant used in student financial-aid inquiries. Identify risks, test cases, metrics, and escalation rules.

3. An assistant retrieves a malicious webpage containing instructions to ignore its system prompt. Analyze the attack and propose technical and procedural defenses.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a foundation model?

A. A model trained for only one fixed classification task

B. A broadly trained model that can be adapted to multiple tasks

C. A database containing only verified facts

D. A rule-based program with no learned parameters

Correct Answer: B. A broadly trained model that can be adapted to multiple tasks

Question 2

In an autoregressive language model, what does the model generally predict during generation?

A. The next token conditioned on preceding tokens

B. All future documents at once

C. Only the final token of a paragraph

D. A database record selected by a human

Correct Answer: A. The next token conditioned on preceding tokens

Question 3

What is the main purpose of dividing QKᵀ by √dₖ in scaled dot-product attention?

A. To remove all positional information

B. To make every token equally important

C. To control score magnitude and improve numerical stability

D. To convert text directly into images

Correct Answer: C. To control score magnitude and improve numerical stability

Question 4

Which approach is generally most appropriate for frequently changing factual information?

A. Using retrieval-augmented generation with versioned sources

B. Increasing output temperature

C. Removing all source documents

D. Using greedy decoding alone

Correct Answer: A. Using retrieval-augmented generation with versioned sources

Question 5

What is a hallucination in generative AI?

A. A model refusing every user request

B. A plausible but unsupported or incorrect output

C. A reduction in the number of model parameters

D. A documented citation linked to its source

Correct Answer: B. A plausible but unsupported or incorrect output

Question 6

Which statement about fine-tuning is correct?

A. It always guarantees factual accuracy

B. It is useful for stable specialized behavior and formats

C. It eliminates the need for evaluation

D. It is the only way to provide current information

Correct Answer: B. It is useful for stable specialized behavior and formats

Question 7

What is prompt injection?

A. Compressing a prompt to reduce token cost

B. An attempt by untrusted content to override intended instructions

C. Converting an image into an embedding

D. Increasing the number of training epochs

Correct Answer: B. An attempt by untrusted content to override intended instructions

Question 8

Which practice best supports reliable high-stakes deployment?

A. Treating fluent output as automatically correct

B. Allowing unrestricted tool access

C. Combining evidence, testing, monitoring, and meaningful human oversight

D. Disabling all logging and review

Correct Answer: C. Combining evidence, testing, monitoring, and meaningful human oversight

Question 9

What does grounding primarily seek to achieve?

A. Connecting generated claims to verifiable evidence

B. Making outputs longer

C. Increasing randomness in decoding

D. Replacing all human decisions

Correct Answer: A. Connecting generated claims to verifiable evidence

Question 10

Why can multimodal systems require modality-specific evaluation?

A. Text and images always have identical error patterns

B. Different modalities introduce different recognition and generation failures

C. Multimodal systems cannot use embeddings

D. Multimodal systems never generate content

Correct Answer: B. Different modalities introduce different recognition and generation failures