7. Generative AI, Foundation Models, and Multimodal Intelligence

Introduction and Context

Generative artificial intelligence refers to computational systems that learn statistical regularities from data and use those regularities to produce new content. The content may be text, images, audio, video, software code, molecular structures, or combinations of these modalities. Unlike conventional predictive systems, which generally map inputs to predefined labels or numerical outputs, generative systems model a distribution over possible observations and sample from, or decode, that distribution.

The contemporary importance of generative AI arises from the convergence of several developments: large-scale datasets, distributed computing, specialized hardware, transformer architectures, self-supervised learning, and improved optimization methods. These developments have produced foundation models: broad models trained on diverse data and adaptable to many downstream tasks. Large language models are a prominent example, but the foundation-model paradigm also includes vision, speech, video, and multimodal systems.

Generative models are not simply larger versions of traditional machine-learning models. Their training objectives, failure modes, evaluation criteria, and organizational consequences differ substantially. A model may produce fluent text without possessing reliable factual knowledge; generate a convincing image with physically impossible details; or combine correct information with an unsupported inference. Consequently, the study of generative AI requires both technical understanding and critical analysis of data provenance, system architecture, human oversight, security, governance, and social impact.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Explain the probabilistic foundations of generative modeling.

2. Compare autoregressive models, variational autoencoders, generative adversarial networks, and diffusion models.

3. Describe the architecture and training principles of foundation models and large language models.

4. Design retrieval-augmented generation systems and distinguish them from parametric model knowledge.

5. Analyze prompting, fine-tuning, parameter-efficient adaptation, and alignment methods.

6. Explain multimodal representation learning and cross-modal generation.

7. Select suitable evaluation methods for quality, factuality, robustness, safety, and fairness.

8. Diagnose hallucinations and propose technical, organizational, and human-centred mitigation strategies.

9. Critically assess the professional, ethical, legal, and research implications of generative AI.

From Density Estimation to Generative Systems

Let observed data be represented by a random variable X. A generative model attempts to learn the data distribution p(x), or a conditional distribution p(x|c) when c represents a class, prompt, context, or another modality. Once such a distribution has been learned, the model can generate a new sample x that resembles the training data. In contrast, a discriminative model estimates p(y|x) or a direct decision function for predicting a label y from x.

The distinction is conceptual rather than absolute. A language model predicts the next token, which appears to be a discriminative local task, but repeated conditional predictions define a distribution over complete sequences. Similarly, an image generator may learn latent representations and then decode them into samples. Generation is therefore closely related to probability, representation learning, optimization, and sequential decision-making.

Several properties are useful when analyzing a generative model. Expressiveness concerns whether the model family can represent the relevant data distribution. Tractability concerns whether likelihoods, gradients, or samples can be computed efficiently. Controllability concerns the ability to direct generation using labels, prompts, constraints, or retrieved evidence. Calibration concerns whether confidence corresponds to correctness. Robustness concerns performance under distribution shift, adversarial input, incomplete context, or unusual requests.

Four Generative Paradigms

Autoregressive generation

An autoregressive model factorizes a joint distribution into a product of conditional distributions:

p(x1, x2, ..., xn) = product of p(xi | x1, ..., xi-1).

For language, xi is usually a token. During training, the model receives the preceding ground-truth tokens and learns to predict the next one, commonly through cross-entropy loss. During generation, it produces one token at a time and feeds each generated token back as context.

Transformers have made autoregressive modeling highly effective. Self-attention allows each position to integrate information from other positions, while positional mechanisms represent sequence order. Causal masking prevents a position from attending to future tokens during next-token prediction. Decoding may use greedy selection, beam search, temperature scaling, top-k sampling, or nucleus sampling. Greedy decoding is deterministic but can be repetitive; stochastic sampling can improve diversity but may reduce factual consistency.

Autoregressive models have a natural likelihood objective and are effective for language, code, music, and sequential data. Their major limitations are sequential generation cost, exposure bias between training and inference, sensitivity to context limits, and the possibility of producing fluent but false continuations.

Variational autoencoders

A variational autoencoder, or VAE, consists of an encoder and decoder connected through a latent variable z. The encoder approximates a posterior q(z|x), and the decoder models p(x|z). Because direct posterior inference is often difficult, the model optimizes the evidence lower bound:

ELBO = E_q[log p(x|z)] - KL(q(z|x) || p(z)).

The first term rewards reconstruction quality. The Kullback–Leibler divergence term regularizes the latent distribution toward a prior, often a standard normal distribution. The reparameterization trick expresses z as a differentiable transformation of random noise and encoder outputs, allowing gradient-based training.

VAEs provide structured latent spaces useful for interpolation, representation learning, anomaly detection, and controllable generation. Their outputs can be smoother or blurrier than those of adversarial models because the likelihood objective averages over uncertainty. Posterior collapse may occur when the decoder ignores the latent variable, particularly when the decoder is very powerful.

Generative adversarial networks

A GAN contains a generator G and discriminator D. The generator maps random noise to synthetic samples, while the discriminator distinguishes real samples from generated ones. In the idealized minimax formulation, the discriminator seeks to maximize discrimination and the generator seeks to minimize it. Training is a competitive game rather than direct likelihood maximization.

GANs can generate sharp and realistic images, but optimization may be unstable. Mode collapse occurs when the generator produces a narrow set of outputs and fails to represent the diversity of the data distribution. Discriminator overconfidence, gradient instability, and sensitivity to architecture and regularization complicate training. GANs remain influential in image synthesis, super-resolution, style transfer, and data augmentation, although diffusion models have become dominant in many high-quality image-generation settings.

Diffusion models

Diffusion models define a forward process that gradually adds noise to data and a learned reverse process that removes noise. In a simple formulation, the forward process produces x_t from x_0 through a schedule of increasingly noisy transformations. A neural network learns to estimate the noise or denoised state needed to reverse the process.

Generation begins with random noise and repeatedly applies the learned reverse transitions. Conditioning can be introduced through class labels, text embeddings, image features, or control signals. Classifier-free guidance increases adherence to conditioning by combining conditional and unconditional predictions, although excessive guidance can reduce diversity or create artifacts.

Diffusion models offer strong sample quality, broad mode coverage, and flexible conditioning. Their disadvantage is computational cost: many denoising steps may be required, although distillation, improved samplers, latent diffusion, and consistency methods reduce inference time. Diffusion architectures are now used for images, audio, video, three-dimensional assets, and scientific design.

Foundation Models and Large Language Models

A foundation model is trained on broad data at scale and subsequently adapted to multiple tasks. The defining idea is not merely size; it is the reuse of general representations across applications. Pretraining may involve self-supervised objectives, such as predicting masked content, the next token, or a missing part of an image. The pretraining corpus establishes what patterns, associations, styles, and biases the model can represent.

Large language models generally use transformer networks. A tokenization procedure maps text into discrete units. Embedding layers represent tokens as vectors, and stacked attention and feed-forward blocks transform these representations. The output layer produces a probability distribution over the vocabulary. Training usually minimizes next-token cross-entropy over very large corpora.

Scale can improve capability, but it does not guarantee reliability. Performance depends on data quality, optimization, architecture, inference procedures, and evaluation design. Emergent-looking capabilities may reflect nonlinear interactions among these factors, while benchmark gains may sometimes result from data contamination or memorization. A doctoral analysis should therefore distinguish parameter count from capability, and capability from dependable performance in a particular domain.

Post-training commonly includes supervised instruction tuning, preference optimization, safety training, and domain adaptation. Instruction tuning improves compliance with task descriptions. Preference-based methods use human or machine preferences to encourage helpful, harmless, or stylistically appropriate responses. These methods shape behavior but do not transform the model into a database or guarantee truthfulness.

Retrieval-Augmented Generation

Retrieval-augmented generation, or RAG, combines a generative model with an external information-retrieval system. A typical pipeline contains document ingestion, cleaning, chunking, embedding, indexing, query formulation, retrieval, reranking, context assembly, generation, and response evaluation.

When a user submits a question, the system converts the query into a representation and retrieves relevant passages from a vector index, keyword index, or hybrid system. The language model then generates an answer conditioned on the retrieved context. The principal advantage is that information can be updated without retraining model parameters, and answers can be grounded in organization-specific documents.

RAG is not automatically reliable. Poor chunking may separate a claim from its qualification. Embedding search may retrieve semantically similar but incorrect passages. Long contexts may dilute relevant evidence. A model may ignore retrieved evidence, cite an irrelevant source, or infer more than the documents support. Effective systems therefore use metadata filters, access control, reranking, source attribution, quotation or evidence constraints, and refusal rules when retrieval is insufficient.

RAG and fine-tuning solve different problems. Retrieval changes the information available at inference time; fine-tuning changes model behavior or representations. Retrieval is often preferable for frequently changing policies, private documents, and traceable factual answers. Fine-tuning is more appropriate for consistent formats, specialized language, or task behaviours that must be applied repeatedly.

Prompting and Adaptation

Prompting is the design of instructions and context supplied to a model. Zero-shot prompting provides a task description without examples. Few-shot prompting includes demonstrations. Role, format, constraints, decomposition, and evaluation criteria can all be specified. Chain-of-thought-style methods encourage intermediate reasoning, but exposing or requiring detailed internal reasoning is not always appropriate; concise rationales, structured verification, or tool-based checking may be safer and easier to audit.

Prompt engineering should be treated as system design rather than a collection of magic phrases. A robust prompt defines the task, audience, input boundaries, desired output schema, evidence requirements, uncertainty policy, and prohibited actions. Prompt injection is a security threat in which untrusted content attempts to override system instructions. This is especially important in RAG and tool-using systems, where documents, web pages, or emails may contain adversarial instructions.

Adaptation methods include full fine-tuning, parameter-efficient fine-tuning, adapters, prompt tuning, and low-rank adaptation. Full fine-tuning updates most parameters and can achieve strong specialization but requires substantial compute and may cause catastrophic forgetting. Low-rank methods update a small number of additional parameters, reducing storage and deployment costs. Quantization reduces numerical precision and memory requirements, but may affect quality. Domain adaptation must be evaluated for both target-task improvement and unintended degradation of general capabilities.

Tool augmentation extends a model with calculators, databases, search engines, code interpreters, workflow systems, or robotic interfaces. A model may decide when to call a tool and incorporate the result into its response. This improves arithmetic, current-information access, and structured operations, but creates new risks involving authorization, data leakage, unsafe actions, and tool-output manipulation.

Multimodal Intelligence

Multimodal models process or generate more than one form of data, such as text, images, speech, video, and sensor streams. A central challenge is learning representations that preserve modality-specific structure while supporting cross-modal alignment. Contrastive learning can bring corresponding image and text embeddings closer while separating unrelated pairs. Joint encoders combine modalities early or within shared transformer layers. Cross-attention allows one modality to condition another, as in text-guided image generation or visual question answering.

Multimodal intelligence involves at least three capabilities: perception, alignment, and generation. Perception identifies entities, attributes, events, or spatial relationships. Alignment associates information across modalities. Generation produces an answer or artifact consistent with the available evidence. Errors may arise at any stage. A model can correctly recognize an object but misunderstand a question, align a caption with the wrong region, or generate a plausible but unsupported explanation.

Applications include accessibility tools, medical-image assistance, industrial inspection, educational tutoring, design, robotics, remote sensing, and human–computer interaction. In high-stakes environments, multimodal systems require modality-specific validation. A text-only safety test cannot establish that a model handles manipulated images, ambiguous speech, visual privacy, or sensor failures appropriately.

Evaluation: Quality Is Multidimensional

No single metric adequately evaluates a generative system. Automatic text metrics based on token overlap may reward lexical similarity but miss factuality and usefulness. Perplexity measures predictive fit but does not directly measure helpfulness or truth. Image metrics can capture distributional similarity while failing to detect semantic errors or unsafe content.

Evaluation should be aligned with the intended use. Important dimensions include:

• Task success: whether the output solves the stated problem.
• Factuality and groundedness: whether claims are supported by reliable evidence.
• Faithfulness: whether an answer reflects the provided source rather than unsupported model knowledge.
• Diversity and novelty: whether outputs avoid unnecessary duplication while remaining valid.
• Robustness: whether performance persists under paraphrase, noise, ambiguity, and distribution shift.
• Calibration: whether confidence and uncertainty correspond to actual reliability.
• Safety and fairness: whether the system avoids harmful, discriminatory, privacy-invasive, or abusive outputs.
• Efficiency: latency, memory, energy use, and cost per request.

Human evaluation remains important, but it must use clear rubrics, representative samples, blinded comparisons, and inter-rater reliability analysis. Red-team testing examines adversarial prompts, prompt injection, data exfiltration, harmful content, and misuse pathways. Continuous monitoring is necessary because users, documents, models, and threat environments change after deployment.

Hallucination and Its Mitigation

Hallucination refers to generated content that is presented as plausible but is unsupported, false, or inconsistent with available evidence. It can result from incomplete training data, ambiguous prompts, next-token optimization, retrieval failure, decoding choices, or pressure to answer every question. Hallucination is not a single technical defect; it is an interaction between model uncertainty, system design, user expectations, and evaluation practices.

Mitigation should operate at several layers. At the model layer, additional training data, preference optimization, uncertainty estimation, and abstention objectives may help. At the system layer, retrieval, source verification, constrained decoding, structured outputs, tool calls, and post-generation fact checking can reduce errors. At the interface layer, citations, confidence indicators, clarification questions, and explicit limitations improve user judgment. At the organizational layer, approved use cases, human review, incident reporting, audit logs, and domain-specific validation are essential.

A refusal is not always the correct response. The system should distinguish between insufficient evidence, prohibited content, uncertain interpretation, and a request that can be answered safely with qualifications. The objective is calibrated assistance: answer when evidence is adequate, identify uncertainty when it is material, and decline or escalate when risk is unacceptable.

An Illustrative Case: A University Research-Policy Assistant

Background

A large university develops an internal assistant to answer questions about research ethics procedures, grant administration, data-management rules, and institutional policies. The assistant is intended for researchers and administrative staff, not as a substitute for formal approval bodies.

Situation and problem

An initial prototype uses a general language model with a long system prompt containing selected policy excerpts. Users report that answers are fluent but occasionally cite outdated procedures or combine rules from different faculties. The university must improve accuracy while protecting confidential research information.

Analysis

The problem is not solved by asking the model to be more confident or by adding more prose to the prompt. Policies change over time, documents have different authorities, and the assistant must distinguish general guidance from binding requirements. The risk of an incorrect answer is uneven: a minor formatting question differs from advice about human-participant approval or export-controlled data.

Approach

The university implements a permission-aware hybrid RAG architecture. Official documents are versioned and classified by authority, effective date, faculty, and access level. Queries are routed to appropriate collections, and retrieval combines keyword matching with semantic search. A reranker prioritizes current authoritative passages. The response template requires a short answer, supporting citations, the policy effective date, and an escalation instruction for high-risk questions.

The model is adapted with parameter-efficient training for institutional terminology and output format, but policy facts remain in the retrieval layer. Evaluation uses a test set developed by research-office experts, including ambiguous questions, outdated-policy traps, missing-evidence cases, and unauthorized-document requests. The system is instructed to state when no authoritative passage is found rather than fill the gap from general knowledge. Access logs and review workflows are established for incidents.

Outcome

The redesigned system produces more traceable answers and makes uncertainty visible. It does not eliminate errors: retrieval can still fail, and users may misunderstand a citation. However, the separation between stable behaviour and changing policy content makes maintenance easier, while escalation rules reduce the likelihood that the assistant is treated as an approval authority.

Learning points

The case demonstrates that generative AI quality depends on the complete socio-technical system. RAG, adaptation, metadata, access control, evaluation, interface design, and governance must be designed together. It also illustrates why factuality, authorization, and accountability cannot be inferred from linguistic fluency.

Risks, Governance, and Professional Responsibility

Generative systems create risks involving privacy, copyright, confidential information, discrimination, misinformation, cyber abuse, labor displacement, environmental cost, and concentration of technological power. Training data may contain personal information or protected works. Outputs may reproduce memorized content or generate stereotypes. Synthetic media can support accessibility and creativity but can also enable impersonation and fraud.

Governance should include data documentation, model cards or system documentation, impact assessments, security testing, access controls, retention rules, provenance mechanisms, and clear accountability. Organizations should classify applications according to consequence and exposure. A low-risk brainstorming tool may require basic monitoring, whereas a system supporting medical, legal, educational, employment, or public-sector decisions requires stronger validation and human authority.

Professional users must not treat generated content as automatically original, accurate, or ethically permissible. They should verify consequential claims, protect sensitive inputs, disclose meaningful AI assistance where required, preserve human responsibility, and follow applicable institutional and legal rules. Researchers should report datasets, preprocessing, evaluation populations, prompts or protocols, limitations, and failure cases sufficiently for independent scrutiny.

Current Directions and Research Questions

Current research is moving toward smaller specialized models, efficient inference, mixture-of-experts architectures, long-context systems, multimodal agents, synthetic-data pipelines, and models that use external tools. Diffusion and transformer hybrids are expanding into video, robotics, scientific simulation, and three-dimensional generation. Retrieval systems increasingly incorporate knowledge graphs, structured databases, and agentic query planning.

Important unresolved questions include how to measure genuine reasoning, how to establish provenance for generated media, how to prevent capability loss during adaptation, how to align models under conflicting values, and how to evaluate systems under real-world distribution shift. Research is also needed on energy-efficient training, multilingual and low-resource performance, accessibility, privacy-preserving learning, and governance mechanisms that are effective without suppressing beneficial innovation.

Best Practices for Designing Generative AI Systems

A responsible development process begins with a clearly bounded use case and an explicit failure-cost analysis. Teams should establish what the system is allowed to do, what evidence it must use, when it must abstain, and who remains accountable. Data should be documented and quality-checked before training or indexing.

Architecture should be selected according to the task rather than fashion. Autoregressive models are natural for sequential generation, VAEs for structured latent representations, GANs for certain high-fidelity transformations, and diffusion models for controllable synthesis. RAG is valuable when current, private, or auditable knowledge is required. Fine-tuning should be used when behavior or format must change consistently.

Evaluation should include ordinary cases, edge cases, adversarial cases, and realistic user workflows. Testing should measure not only average quality but also severe failures, subgroup performance, latency, cost, and uncertainty. Deployment should be staged, monitored, and reversible. Finally, users need training on verification, privacy, prompt injection, source interpretation, and appropriate escalation.

Chapter Summary

Generative AI models learn distributions that enable the production of new content. Autoregressive models generate sequentially, VAEs learn probabilistic latent spaces, GANs use adversarial competition, and diffusion models learn to reverse progressive corruption. Foundation models generalize across tasks through broad pretraining, while large language models use transformer-based sequence prediction and post-training adaptation.

RAG connects generation to external, updateable evidence, whereas fine-tuning changes model behaviour or representations. Prompting, parameter-efficient adaptation, and tool use extend model capability but introduce security and reliability concerns. Multimodal systems must align perception, representation, and generation across different data types. Evaluation must address task success, factuality, grounding, robustness, safety, fairness, calibration, and efficiency. Hallucination mitigation requires model, system, interface, and governance interventions. The central professional lesson is that generative AI is a socio-technical system whose reliability depends on evidence, controls, evaluation, and accountable use rather than fluency alone.

Key Terms

Generative model: A model that learns a data distribution and produces new samples from it.

Autoregressive model: A model that factorizes a joint distribution into sequential conditional predictions.

Variational autoencoder: A latent-variable generative model trained using reconstruction and distribution-regularization objectives.

Generative adversarial network: A system in which a generator and discriminator compete during training.

Diffusion model: A generative model that learns to reverse a gradual process of adding noise.

Foundation model: A broadly pretrained model adaptable to many downstream tasks.

Large language model: A large neural language model trained primarily to predict token sequences.

Transformer: An architecture based largely on attention mechanisms for processing sequences or multimodal representations.

Tokenization: The conversion of raw text or other input into model-processing units.

Latent representation: A compact internal representation that captures underlying patterns in data.

Prompt engineering: The deliberate design of instructions, context, constraints, and examples for a model.

Fine-tuning: Further training of a pretrained model for a particular task, domain, or behaviour.

Parameter-efficient fine-tuning: Adaptation that updates a small parameter set rather than the full model.

Retrieval-augmented generation: Generation conditioned on information retrieved from an external knowledge source.

Embedding: A vector representation that encodes semantic or structural properties.

Multimodal learning: Learning from or across multiple data modalities.

Hallucination: Unsupported, false, or inconsistent generated content presented as plausible.

Calibration: The degree to which model confidence corresponds to empirical correctness.

Prompt injection: An attack in which untrusted content attempts to alter system instructions or behaviour.

Classifier-free guidance: A diffusion-generation technique that strengthens adherence to conditioning information.

Review Questions

Short Answer Questions

1. What is the difference between a generative model and a discriminative model?

2. Why can autoregressive language generation produce fluent but factually incorrect text?

3. What two terms comprise the principal VAE evidence lower bound?

4. How does retrieval-augmented generation differ from fine-tuning?

5. Define hallucination in the context of generative AI.

Long Answer Questions

1. Compare autoregressive models, VAEs, GANs, and diffusion models in terms of objective, generation process, strengths, and limitations.

2. Explain the architecture and training lifecycle of a large language model from pretraining through post-training adaptation.

3. Describe a complete RAG pipeline and analyze the main sources of retrieval and generation failure.

4. Explain why multimodal intelligence requires more than simply combining text and image inputs.

5. Discuss a multidimensional evaluation framework for a high-stakes generative AI application.

Analytical / Application Questions

1. An organization needs an assistant that answers questions from frequently changing internal policies and must cite authoritative sources. Should it prioritize RAG, fine-tuning, or both? Justify the architecture and controls.

2. A text-to-image system produces highly realistic images but repeatedly omits important objects specified in prompts. Analyze possible causes and propose an evaluation and mitigation plan.

3. Design a hallucination-mitigation strategy for a research assistant used by doctoral students. Include evidence handling, abstention, human review, monitoring, and governance.

22. Multiple Choice Questions with Answers

Question 1

Which characteristic most directly distinguishes a generative model from a conventional discriminative classifier?

A. It always uses unsupervised learning

B. It models a distribution from which new observations can be produced

C. It cannot use labels during training

D. It produces only numerical outputs

Correct Answer: B. It models a distribution from which new observations can be produced

Question 2

During autoregressive language generation, what does the model normally condition on when predicting the next token?

A. Only the final token in the vocabulary

B. Future ground-truth tokens

C. Previously available context and generated tokens

D. A discriminator score

Correct Answer: C. Previously available context and generated tokens

Question 3

What is the principal role of the KL-divergence term in a standard variational autoencoder objective?

A. It forces the discriminator to classify real images

B. It regularizes the approximate latent posterior toward a prior

C. It removes the need for a decoder

D. It guarantees perfect reconstruction

Correct Answer: B. It regularizes the approximate latent posterior toward a prior

Question 4

A generator produces excellent-looking images but represents only a small fraction of the training distribution. Which problem is most likely occurring?

A. Posterior collapse

B. Mode collapse

C. Tokenization failure

D. Exposure bias

Correct Answer: B. Mode collapse

Question 5

Why is retrieval-augmented generation particularly useful for organizational policy assistants?

A. It makes model parameters unnecessary

B. It allows changing source information to be incorporated without retraining the whole model

C. It guarantees that every retrieved document is correct

D. It prevents all prompt-injection attacks

Correct Answer: B. It allows changing source information to be incorporated without retraining the whole model

Question 6

Which adaptation method usually changes a relatively small number of additional parameters while leaving most pretrained parameters fixed?

A. Full retraining from random initialization

B. Parameter-efficient fine-tuning

C. Greedy decoding

D. Forward diffusion

Correct Answer: B. Parameter-efficient fine-tuning

Question 7

A retrieved document contains instructions telling the model to reveal confidential system data. What threat does this illustrate?

A. Mode collapse

B. Prompt injection

C. Posterior regularization

D. Calibration

Correct Answer: B. Prompt injection

Question 8

Which evaluation dimension asks whether an answer is supported by the documents supplied to the model?

A. Groundedness or faithfulness

B. Tokenization rate

C. Sample latency only

D. Parameter count

Correct Answer: A. Groundedness or faithfulness

Question 9

A model gives a confident answer even though its retrieval system found no authoritative evidence. Which mitigation is most appropriate?

A. Increase temperature without other changes

B. Require evidence checks and permit abstention when evidence is insufficient

C. Remove all source citations

D. Use only greedy decoding

Correct Answer: B. Require evidence checks and permit abstention when evidence is insufficient

Question 10

Why does a multimodal system require modality-specific validation?

A. All modalities have identical error patterns

B. Text-only evaluation fully measures visual and audio reliability

C. Errors can arise separately in perception, cross-modal alignment, and generation

D. Multimodal systems cannot be evaluated by humans

Correct Answer: C. Errors can arise separately in perception, cross-modal alignment, and generation