Generative AI and Foundation Models

Introduction

Generative artificial intelligence (generative AI) refers to computational systems that produce new text, images, audio, video, code, or combinations of these media from learned patterns. Unlike conventional automation, which follows explicitly programmed rules, generative systems infer relationships from large datasets and use those relationships to create outputs that resemble human-produced media.

For mass media managers, generative AI is not merely a production tool. It is an infrastructure technology that affects research, commissioning, writing, editing, localization, audience engagement, advertising, archives, and distribution. Its value depends on the relationship between model capability, editorial purpose, cost, workflow design, and governance. A powerful model can still be unsuitable if it produces unverifiable claims, violates rights, exposes confidential data, or creates more correction work than it saves.

A foundation model is a large, general-purpose model trained on broad data and adapted for many downstream tasks. Large language models (LLMs) are one important class of foundation model, but foundation models may also process images, sound, video, or several modalities simultaneously.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain foundation models, large language models, tokens, embeddings, and transformers.
2. Distinguish autoregressive language generation from diffusion-based generation.
3. Describe how multimodal models process and connect text, images, audio, and video.
4. Design effective prompts and evaluate generated outputs.
5. Explain retrieval-augmented generation (RAG), fine-tuning, and AI-agent architectures.
6. Evaluate media applications according to quality, cost, rights, safety, and editorial risk.
7. Recommend governance and human-supervision practices for media organizations.

1. Foundation Models and Generative AI

A foundation model is trained on a broad corpus before being adapted through prompting, supervised training, retrieval, or fine-tuning. The term “foundation” emphasizes reuse: one base model can support summarization, translation, question answering, script drafting, classification, image analysis, or conversational interfaces.

A model does not contain a searchable copy of every training document in a simple database-like form. Instead, training adjusts numerical parameters so that the system learns statistical and semantic regularities. This distinction matters. A model may reproduce memorized passages, but it may also generate plausible statements without possessing a reliable, up-to-date source for them.

Generative AI systems normally include four layers:

The model is therefore only one component of a professional media system.

Important terminology

A token is a unit processed by a language model. It may correspond to a word, part of a word, punctuation mark, or other symbol. Tokenization differs between languages and model families. A context window is the maximum amount of input and generated material the model can consider in one interaction. A larger context window does not guarantee accurate use of every included passage.

An embedding is a numerical vector representing semantic or other learned relationships. Similar meanings often occupy nearby regions in embedding space, although similarity depends on the training data and embedding method. Embeddings are widely used for search, recommendation, clustering, and RAG.

Inference is the process of using a trained model to generate an output. Parameters are learned numerical values inside the model. Weights is a common term for parameters, although some systems also contain other learned components.

2. Large Language Models

An LLM predicts sequences of tokens. In an autoregressive model, the probability of a sequence is factorized as:

Here, x_i is the token at position i, n is the sequence length, and the product expresses the probability of each token conditioned on preceding tokens. The equation describes the modeling objective; it does not mean that the model consciously reasons one word at a time. During generation, the system repeatedly estimates the next-token distribution and selects a token according to a decoding strategy.

Training commonly minimizes cross-entropy loss. For a target sequence, a simplified form is:

L is loss, P is the model’s predicted probability, and log is the natural logarithm. Lower loss generally indicates better prediction on the evaluated data, but it does not by itself prove factual accuracy, fairness, originality, or editorial usefulness. The formula assumes a defined token sequence and a probability distribution produced by the model.

Transformers and attention

Most contemporary LLMs use the transformer architecture. Its central mechanism, self-attention, enables a representation of one token to incorporate information from other tokens in the sequence. A standard attention expression is:

Q represents queries, K keys, V values, and d_k the dimensionality of the key vectors. QK^T produces compatibility scores; division by √d_k controls their scale; softmax converts scores into normalized weights. In practice, multiple attention heads learn different relationships, such as syntax, reference, or long-range associations.

Transformers also use positional information because attention alone does not inherently encode word order. Feed-forward layers, residual connections, normalization, and stacked blocks transform the representations. The architecture is powerful because it processes relationships in parallel during training and can be scaled across large datasets and computing systems.

Why LLMs can hallucinate

A hallucination is an output that is fluent but unsupported, inaccurate, or fabricated. It can occur because the model is optimized to predict likely language, not to guarantee truth. Causes include incomplete training data, ambiguous prompts, outdated knowledge, weak source grounding, instruction conflicts, and pressure to answer rather than abstain.

For media work, hallucinations are especially serious in names, quotations, dates, statistics, legal claims, medical information, and breaking news. Fluency is not evidence. Every consequential claim requires source verification.

3. Multimodal Models

A multimodal model accepts or generates more than one media type. It may combine text with images, speech, audio, video, or structured data. One design uses separate encoders for different modalities and maps their representations into a shared space. Another uses a unified token-like representation for several media types. The exact architecture varies by system.

Typical media capabilities include:

• describing an image or video frame;
• transcribing and translating speech;
• identifying visual elements for accessibility metadata;
• answering questions about a document or infographic;
• generating storyboards, voice, images, or video;
• aligning captions with scenes and sound.

Multimodality does not remove the need for domain expertise. A model may identify a visible object but misunderstand its cultural significance, misread sarcasm, confuse a person, or infer an event not actually shown. Media organizations should test performance across accents, languages, disability contexts, skin tones, regional visual conventions, and low-quality footage.

4. Diffusion Systems and Synthetic Media

Diffusion models are widely used for image, audio, and video generation. In a simplified description, training gradually adds noise to data and teaches a model to reverse the process. At generation time, the system begins with noise and performs repeated denoising steps conditioned on a prompt or other input.

A forward noising process may be written conceptually as:

Here, x₀ is the original data, x_t is the noisy version at step t, ε is random noise, and α̅_t is a schedule-dependent cumulative signal coefficient between 0 and 1. This equation is a simplified representation; implementations differ in parameterization and sampling method. The terms must have compatible numerical dimensions, and the same data representation must be used for x₀, x_t, and ε.

Diffusion systems are useful for concept art, background generation, visual prototyping, restoration experiments, and synthetic advertising variations. However, they can reproduce stylistic features from training data, generate inaccurate text in images, distort hands or objects, and create deceptive depictions. Newsrooms should label illustrative or synthetic imagery and should not present fabricated scenes as documentary evidence.

5. Prompting as Media Direction

Prompting is the design of instructions, context, constraints, examples, and output requirements. A professional prompt is closer to a creative brief than a magic phrase. It should state:

1. Role or task: what the system must do.
2. Source material: what it may use.
3. Audience and purpose: who will use the result and why.
4. Constraints: length, tone, reading level, prohibited claims, and format.
5. Evaluation criteria: accuracy, completeness, neutrality, accessibility, or style.
6. Uncertainty behavior: when to state “insufficient evidence” or request clarification.

For example: “Using only the supplied interview transcript, produce a 150-word web summary for a general audience. Preserve names and quotations exactly, identify claims that require fact-checking, and do not add information from outside the transcript.” This prompt reduces unsupported invention by restricting the evidence base and specifying verification behavior.

Prompting methods

Zero-shot prompting gives instructions without examples. Few-shot prompting supplies examples of desired inputs and outputs. Chain-of-thought-style prompting asks for structured reasoning, but organizations should avoid treating hidden reasoning text as proof of correctness. A safer practice is to request concise conclusions, evidence references, assumptions, and checks. Structured output prompting requires a schema such as fields for headline, source, confidence, and unresolved issues.

Prompt testing should use a fixed evaluation set rather than a few impressive demonstrations. Managers should compare outputs for accuracy, consistency, bias, latency, cost, and correction time.

6. Retrieval-Augmented Generation

RAG connects a generative model to an external collection. A typical pipeline is:

1. collect and permission the documents;
2. clean, segment, and attach metadata;
3. create embeddings for the segments;
4. retrieve relevant segments for a query;
5. place the retrieved evidence in the model context;
6. generate an answer with citations or source links;
7. evaluate both retrieval and generation.

RAG is valuable for archives, policy manuals, newsroom style guides, program catalogs, and current organizational information. It can reduce outdated answers without retraining the base model. It does not guarantee truth: if retrieval returns irrelevant, incomplete, or biased documents, the generated answer may still be wrong.

Retrieval quality can be measured with precision and recall. If relevant retrieved items are denoted by true positives, irrelevant retrieved items by false positives, and relevant but unretrieved items by false negatives:

Precision measures the proportion of retrieved items that are relevant; recall measures the proportion of all relevant items that were retrieved. These measures require a defined relevance standard and labeled evaluation set. A media archive may prefer high recall during discovery but higher precision when generating a final factual answer.

RAG designs should preserve document identity, publication date, rights status, page or timestamp references, and access controls. Citations must be checked against the generated claim rather than added cosmetically.

7. Fine-Tuning and Model Adaptation

Fine-tuning updates some or all model parameters using a task-specific dataset. It can improve consistent style, classification, formatting, or domain behavior. It is appropriate when the organization has a stable task and a sufficiently representative, legally usable dataset.

Fine-tuning is not a substitute for current information. A model fine-tuned on last year’s news does not automatically know today’s events. Current facts are generally better supplied through retrieval or controlled databases.

Adaptation options include:

Fine-tuning data must be reviewed for copyright, personal data, bias, confidential information, and duplicates. Evaluation should compare the adapted model with the base model and include failure cases, not only average performance.

8. AI Agents and Workflow Automation

An AI agent is a system that uses a model to interpret a goal, plan or select steps, call tools, observe results, and continue until a stopping condition is reached. A media agent might search an archive, retrieve a transcript, draft metadata, check a style guide, and send a package for human approval.

A safe agent architecture separates:

• planning: deciding which steps may be useful;
• tools: narrowly defined functions such as search or transcription;
• state: task information and intermediate results;
• permissions: what the agent may read, change, or publish;
• approval: human review at consequential decision points;
• logging: records of prompts, sources, actions, and outputs.

Agents introduce risks beyond ordinary chat. They can take incorrect actions, follow malicious instructions embedded in documents, expose secrets, repeat tasks indefinitely, or make unauthorized publishing decisions. Least-privilege access, sandboxing, rate limits, confirmation gates, and rollback mechanisms are essential.

9. Media Capabilities, Costs, and Evaluation

Generative AI can support the media value chain:

• Research: archive discovery, interview preparation, trend clustering.
• Production: outlines, rough cuts, subtitles, translation, storyboards.
• Post-production: noise reduction, caption alignment, metadata, versioning.
• Distribution: headlines, summaries, personalization, accessibility formats.
• Commercial work: campaign variants, audience segmentation, creative testing.
• Management: schedule analysis, rights tracking, resource forecasting.

A useful evaluation framework considers five dimensions:

1. Quality: factuality, relevance, coherence, visual or audio fidelity.
2. Efficiency: time saved, throughput, latency, and correction burden.
3. Economics: model charges, computing, storage, integration, staff review, and failure costs.
4. Risk: privacy, rights, bias, misinformation, security, and reputational harm.
5. Strategic fit: compatibility with brand, audience, workforce, and editorial values.

A simplified total-cost model is:

The terms represent model usage, data preparation and storage, integration and infrastructure, human review, and expected failure-related cost. This is a managerial accounting model rather than a universal law. Costs should be measured over a defined period and include both fixed and variable components where relevant.

Worked example: comparing automation options

A publisher processes 10,000 article summaries per month. A model costs $0.004 per summary. Integration and monitoring cost $1,200 per month. Human review takes 30 seconds per summary at an effective labor cost of $24 per hour. If automation reduces review to 12 seconds, the monthly review cost is:

30 seconds = 0.5 minutes; 12 seconds = 0.2 minutes.

Model cost is:

Therefore, estimated monthly operating cost is:

This estimate excludes failure costs, training, and taxes. If the automated system creates expensive legal or reputational errors, the apparent saving may disappear. The correct decision therefore depends on quality and risk evidence, not on model price alone.

10. Rights, Ethics, and Governance

Media organizations must address copyright and licensing for training data, input materials, generated outputs, voice and likeness, music, footage, and archival content. Legal rules differ by jurisdiction and continue to develop. Contractual permissions and documented provenance are therefore important even where a use appears technically possible.

Privacy risks arise when staff submit unpublished scripts, source identities, audience records, or personal data to external services. Organizations should classify data, minimize collection, encrypt sensitive material, define retention periods, and establish approved tools.

Important governance practices include:

• maintain an inventory of AI systems and use cases;
• assign accountable owners;
• conduct risk assessments before deployment;
• document data sources and permissions;
• test for demographic and linguistic bias;
• require human review for consequential outputs;
• label synthetic or materially AI-altered media when audiences could be misled;
• retain logs and version information;
• provide correction and appeal procedures;
• train employees in verification and secure use.

Human oversight should be meaningful. A person who merely clicks “approve” without time, expertise, or authority is not an effective safeguard. Editorial accountability remains with the organization and its responsible professionals.

11. Illustrative Case: Public-Service Video Localization

Background

A fictional public-service broadcaster publishes emergency-preparedness videos in four languages. Manual transcription, translation, dubbing, captioning, and quality checks take several days, delaying distribution.

Problem

Management proposes an AI pipeline that transcribes the master video, translates the script, generates synthetic voices, creates captions, and prepares platform-specific versions. The content includes safety instructions, place names, and numbers, so errors could cause harm.

Analysis

The broadcaster identifies different risk levels. Transcription is relatively low risk if checked against the audio. Translation of evacuation instructions and numbers is high risk. Synthetic voices may improve access but could be confused with an official human speaker. Personal data in interviews requires controlled processing.

Approach and solution

The organization uses an approved transcription model, then retrieves terminology from a maintained multilingual glossary. A language specialist reviews every safety instruction, proper noun, number, and caption timing. The synthetic voice is clearly disclosed, and a human producer approves the final video. The system logs source versions, translation changes, reviewers, and publication dates.

Outcome

Production time falls substantially, but the organization rejects full automation. The greatest benefit comes from automating repetitive preparation while reserving human judgment for meaning, safety, tone, and public trust.

Learning points

The case illustrates that workflow design matters more than model novelty. RAG supports terminology consistency; prompting defines constraints; human review manages high-impact errors; and disclosure protects audience understanding. Performance should be measured by accurate, timely, accessible communication—not merely by the number of automated steps.

12. Contemporary Developments and Best Practices

Foundation models are increasingly moving toward longer context, tool use, multimodal interaction, smaller specialized models, on-device inference, and systems that combine generation with structured data. These developments may reduce latency or improve privacy, but they also create new governance challenges, including complex supply chains and less transparent model updates.

Best practice for media managers is to begin with a specific user problem rather than an attractive demonstration. Establish a baseline workflow, define measurable success criteria, classify risks, test with representative data, and pilot under supervision. Compare total cost and correction effort with the existing process. Keep humans responsible for factual claims, public-interest judgments, rights decisions, and publication of consequential material.

Chapter Summary

Generative AI creates media by learning patterns in large datasets. Foundation models provide reusable capabilities, while LLMs generate token sequences through transformer-based representations and attention. Multimodal models connect text, image, audio, and video; diffusion systems commonly generate media through iterative denoising.

Prompting shapes model behavior through tasks, context, constraints, and evaluation criteria. RAG supplies external evidence and can improve freshness, but retrieval quality and citation accuracy must be tested. Fine-tuning adapts behavior for stable tasks but requires careful data governance. AI agents combine models with tools and permissions, making security, logging, and approval controls essential.

Media value depends on quality, efficiency, total cost, rights, privacy, bias, safety, and public trust. Human oversight, provenance, disclosure, and accountable governance are necessary because fluent output is not equivalent to truth or editorial responsibility.

Key Terms

Foundation model: A broad, reusable model adapted to many downstream tasks.

Generative AI: AI that produces new content such as text, images, audio, or video.

Large language model: A model trained to represent and generate language sequences.

Token: A unit processed by a language model.

Context window: The input and output range a model can consider in one interaction.

Parameter: A learned numerical value in a model.

Embedding: A numerical representation used to encode relationships or similarity.

Transformer: A neural architecture based substantially on attention mechanisms.

Self-attention: A mechanism that relates elements within a sequence.

Hallucination: Fluent but unsupported, inaccurate, or fabricated output.

Multimodal model: A model that processes or generates multiple media types.

Diffusion model: A generative model commonly based on learning to reverse a noising process.

Prompt: An instruction and context supplied to a model.

Retrieval-augmented generation: Generation grounded in documents retrieved from an external collection.

Fine-tuning: Further training of a model on task- or domain-specific data.

AI agent: A model-based system that plans, uses tools, and performs actions toward a goal.

Inference: Use of a trained model to produce an output.

Provenance: Information about the origin, history, and modification of media or data.

Human-in-the-loop: A workflow in which people review, guide, or approve AI outputs.

Review Questions

5 Short Answer Questions

1. What is a foundation model, and why is it reusable across media tasks?
2. Define token, embedding, and context window.
3. Why can an LLM produce a hallucination?
4. What is the principal purpose of retrieval-augmented generation?
5. Name two safeguards required for AI agents.

5 Long Answer Questions

1. Explain how transformers and self-attention support language generation.
2. Compare prompting, RAG, and fine-tuning for a media archive application.
3. Discuss the opportunities and limitations of multimodal models in accessibility production.
4. Explain why total cost of ownership is more informative than model usage price alone.
5. Develop a governance framework for synthetic images used by a news organization.

3 Analytical / Application Questions

1. A broadcaster wants to automate translation of emergency announcements. Design a workflow that balances speed, accuracy, disclosure, and human approval.
2. An archive chatbot gives confident answers with no sources. Diagnose possible retrieval and generation failures and propose an evaluation plan.
3. A media company claims that an AI agent can publish audience-targeted headlines without review. Assess the operational, ethical, and reputational risks and recommend permission controls.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a foundation model?

A. A model trained only for one narrowly defined output

B. A broad model that can be adapted to multiple downstream tasks

C. A database containing every document used in training

D. A rule-based system that never generates novel content

Correct Answer: B. A broad model that can be adapted to multiple downstream tasks

Question 2

In an autoregressive language model, the next token is primarily predicted from:

A. Only the final character of the input

B. A fixed human-written dictionary

C. The preceding token sequence and learned parameters

D. A random selection independent of context

Correct Answer: C. The preceding token sequence and learned parameters

Question 3

What is the main function of self-attention in a transformer?

A. To delete all repeated words

B. To relate elements within a sequence when forming representations

C. To convert every text into an image

D. To guarantee factual correctness

Correct Answer: B. To relate elements within a sequence when forming representations

Question 4

Which technique is most directly intended to provide a model with current organizational documents at answer time?

A. Retrieval-augmented generation

B. Random decoding

C. Image diffusion

D. Tokenization alone

Correct Answer: A. Retrieval-augmented generation

Question 5

If a retrieval system returns 80 relevant items and 20 irrelevant items, its precision is:

A. 20%

B. 50%

C. 80%

D. 100%

Correct Answer: C. 80%

Question 6

What is a major risk of treating fluent model output as verified journalism?

A. The output may contain unsupported or fabricated claims

B. The output will always be shorter than the source

C. The model cannot produce punctuation

D. The output cannot be translated

Correct Answer: A. The output may contain unsupported or fabricated claims

Question 7

Which use is generally most suitable for fine-tuning?

A. Supplying live breaking-news facts without retrieval

B. Improving consistent behavior for a stable, well-defined task

C. Replacing all rights and privacy reviews

D. Guaranteeing that an agent takes no actions

Correct Answer: B. Improving consistent behavior for a stable, well-defined task

Question 8

What distinguishes an AI agent from a simple text-generation interaction?

A. An agent may plan, use tools, and take authorized actions

B. An agent cannot access any information

C. An agent is always human-operated manually

D. An agent produces only images

Correct Answer: A. An agent may plan, use tools, and take authorized actions

Question 9

Which cost is included in a realistic total-cost assessment of media AI?

A. Only the advertised token price

B. Only the purchase price of a computer

C. Model, integration, review, data, and failure-related costs

D. Only the number of prompts written

Correct Answer: C. Model, integration, review, data, and failure-related costs

Question 10

Why should synthetic documentary-style images generally be labeled?

A. To make them impossible to edit

B. To prevent audiences from confusing fabricated scenes with documentary evidence

C. To increase the model context window

D. To eliminate the need for captions

Correct Answer: B. To prevent audiences from confusing fabricated scenes with documentary evidence