10. Natural Language, Perception, and Multimodal Intelligence

Introduction and Context

Artificial intelligence becomes most useful when it can operate through the forms of communication and perception used by people. Human interaction with the world is not limited to structured tables or explicit commands. People speak, write, observe, gesture, interpret images, combine sensory evidence, and use context to infer meaning. Natural language processing (NLP), speech technologies, computer vision, and multimodal intelligence address these capabilities.

This chapter examines how AI systems understand language, perceive visual and auditory information, extract knowledge, conduct conversations, and combine multiple modalities for reasoning. The emphasis is not merely on individual algorithms, but on the design of complete systems: how data are represented, how models are trained, how quality is evaluated, and how technical decisions affect users and organizations.

A system that classifies an image, transcribes a meeting, answers a question, or assists a person with a disability must be assessed differently from a system that predicts a numerical value. Accuracy alone may be inadequate. Robustness, fairness, interpretability, latency, privacy, safety, and user satisfaction may determine whether an intelligent application is fit for purpose.

Learning Objectives

After studying this chapter, students should be able to:

1. Explain the principal concepts and architectures used in language, speech, vision, and multimodal AI.

2. Distinguish language understanding, information extraction, dialogue management, and language generation.

3. Describe major approaches to speech recognition and computer vision.

4. Analyse how multimodal systems align and fuse text, images, audio, and other signals.

5. Select suitable quality measures for language, speech, vision, and conversational applications.

6. Design an applied AI solution while considering data quality, bias, privacy, security, and human oversight.

7. Evaluate emerging developments such as foundation models, retrieval-augmented generation, and embodied interaction.

1. From Signals to Meaning

An AI perception system receives signals; an intelligent system must convert those signals into useful representations and decisions. A camera produces pixels, a microphone produces a time-varying waveform, and a document contains sequences of symbols. None of these signals is meaning by itself. Meaning emerges through representation, context, inference, and interaction.

A useful abstraction is a pipeline:

sensing → preprocessing → representation → inference → grounding → action or response

Preprocessing may include noise reduction, image resizing, tokenization, or speech segmentation. Representation converts raw input into features or learned embeddings. Inference maps representations to labels, entities, intentions, answers, or generated content. Grounding connects an output to an external object, document, location, event, or user goal. Finally, the system responds, recommends, retrieves, controls, or asks for clarification.

Modern deep learning often combines these stages in a single trainable architecture. Nevertheless, the conceptual distinction remains valuable. A model can produce fluent language without being grounded in reliable evidence; it can recognize an object without understanding its role in a scene; and it can transcribe speech accurately while misunderstanding the speaker's intention.

Important terminology

Token: A unit processed by a language model, such as a word, subword, character, or punctuation mark.

Embedding: A dense numerical representation intended to capture semantic or perceptual relationships.

Context: Information surrounding an input that influences interpretation, including prior dialogue, visual content, or domain knowledge.

Ontology: A structured specification of concepts, relationships, and constraints in a domain.

Grounding: Linking a model's representation or output to evidence, real-world entities, or an external knowledge source.

Inference: The process of deriving a prediction, interpretation, or decision from input and learned knowledge.

Modality: A type or channel of information, such as text, image, audio, video, touch, or sensor data.

Multimodal intelligence: The ability to represent, align, reason over, and generate information across two or more modalities.

2. Language Intelligence

Language as a computational object

Human language is sequential, structured, ambiguous, and dependent on social and physical context. The sentence “The bank is closed” may refer to a financial institution or the side of a river. Pronouns require discourse context, and the intended meaning of a request may differ from its literal wording. Consequently, language systems must address syntax, semantics, pragmatics, discourse, and world knowledge.

Traditional NLP systems used hand-written grammars, dictionaries, symbolic rules, and statistical models. These remain useful in controlled domains, but contemporary systems generally use neural representations learned from large corpora. Transformer architectures are especially important because self-attention allows a model to relate a token to other tokens across a sequence. An encoder commonly produces contextual representations for understanding tasks, while a decoder generates sequences; encoder–decoder systems support tasks such as translation and summarization.

A language model estimates the probability of a sequence or the next token given preceding context. Pretraining on large collections allows models to acquire broad linguistic regularities. Fine-tuning, instruction tuning, retrieval, and tool use can adapt such models to particular tasks. However, learned statistical association is not equivalent to verified knowledge or reliable reasoning.

Core language-processing tasks

Tokenization and linguistic analysis. Tokenization divides text into computational units. Part-of-speech tagging assigns grammatical categories, while parsing represents syntactic relationships. These operations support downstream interpretation, although modern neural systems may learn them implicitly.

Text classification. A classifier assigns one or more labels, such as topic, sentiment, urgency, toxicity, or document type. Class imbalance and ambiguous labels can make accuracy misleading; precision, recall, macro-F1, and calibration are often more informative.

Named-entity recognition and relation extraction. Named-entity recognition identifies people, organizations, places, dates, products, or domain-specific entities. Relation extraction determines connections such as “works-for,” “located-in,” or “causes.” Together they transform unstructured text into structured knowledge.

Information extraction. Information extraction may include entity recognition, relation extraction, event extraction, temporal normalization, and field extraction from documents. In an insurance workflow, for example, a system may identify a claimant, accident date, damaged component, and estimated amount. Human review is appropriate when extraction affects legal, financial, or safety decisions.

Question answering and retrieval. An open-domain system searches a collection and produces an answer. Retrieval-augmented generation (RAG) first retrieves relevant passages and then conditions generation on them. This can improve traceability and domain freshness, but retrieval errors, incomplete documents, and unsupported synthesis remain possible.

Summarization and translation. Summarization compresses content while preserving important meaning. Translation maps meaning between languages rather than merely substituting words. Evaluation must consider factuality, omissions, terminology, fluency, and cultural appropriateness.

Dialogue and conversational systems

A conversational system normally contains several conceptual components:

1. Input understanding: speech recognition, language identification, intent classification, and entity extraction.

2. Dialogue state tracking: representation of the user's goal, filled slots, unresolved references, and conversation history.

3. Dialogue policy: selection of the next action, such as answering, asking a question, confirming information, or transferring to a human.

4. Response generation: construction of a natural-language response, often using templates, retrieval, or a generative model.

5. Execution and monitoring: interaction with databases, APIs, tools, and safety controls.

A banking assistant should not simply generate a plausible answer. It must authenticate a user, apply authorization rules, distinguish informational requests from transactions, confirm consequential actions, and provide a reliable escalation path. This illustrates the difference between a conversational model and a dependable conversational service.

Dialogue quality includes task success, completion rate, appropriate clarification, response relevance, latency, turn-taking, robustness to paraphrase, and user trust. Excessive confidence is a serious failure mode. An assistant should express uncertainty or request clarification rather than inventing a policy or transaction result.

3. Speech and Auditory Perception

Speech processing connects acoustic signals with linguistic and paralinguistic information. Automatic speech recognition (ASR) converts speech to text; text-to-speech (TTS) converts text into an audio waveform; speaker recognition identifies or verifies a speaker; and speech emotion or prosody analysis estimates characteristics such as emphasis, pace, or affect.

An ASR system traditionally used acoustic, pronunciation, and language models. End-to-end neural systems learn a direct mapping from audio features to tokens, often using convolutional, recurrent, or Transformer-based architectures. Important practical variables include microphone quality, background noise, reverberation, accents, code-switching, speech rate, domain terminology, and overlapping speakers.

The principal ASR metric is word error rate (WER):

WER = (substitutions + deletions + insertions) / number of reference words.

A low overall WER may conceal poor performance for a particular accent or group. Evaluation should therefore be stratified by speaker characteristics, environment, language variety, and task-critical vocabulary. In a medical transcription system, a small error in a drug name may be more consequential than several errors in function words.

TTS systems are assessed through intelligibility, naturalness, pronunciation, prosody, latency, and speaker similarity where relevant. Voice cloning creates additional risks involving consent, impersonation, fraud, and unauthorized use of biometric identity. Systems should disclose synthetic voices when users could reasonably mistake them for human speech.

4. Computer Vision: Perception in Images and Video

Computer vision enables machines to interpret visual data. The task may be recognition, localization, segmentation, tracking, reconstruction, or generation.

Image classification assigns one or more labels to an image. Object detection identifies object categories and bounding boxes. Semantic segmentation assigns a class to each pixel, while instance segmentation distinguishes separate objects of the same class. Object tracking links detections across video frames. Optical character recognition (OCR) converts visual text into machine-readable text.

Convolutional neural networks introduced effective hierarchical feature learning for vision, while vision Transformers use attention over image patches or related visual tokens. Pretrained vision–language models can associate images and text, enabling image retrieval, captioning, visual question answering, and zero-shot classification.

Evaluation depends on the task. Classification commonly uses accuracy, precision, recall, F1, and confusion matrices. Detection uses intersection over union (IoU) and average precision. Segmentation may use mean IoU or Dice similarity. OCR requires character or word error rate. In safety-critical inspection, missed defects may be far more costly than false alarms, so threshold selection should reflect the cost of errors.

Vision systems are vulnerable to changes in lighting, viewpoint, occlusion, weather, camera placement, and sensor quality. They may also inherit demographic or geographic bias from training data. A model that performs well on a benchmark may fail after deployment because the operational environment differs from the data distribution. Continuous monitoring and representative validation are therefore essential.

5. Information Extraction from Documents and the World

Many organizational processes depend on information distributed across emails, forms, reports, images, audio, and databases. Intelligent document processing combines OCR, layout analysis, language models, entity extraction, classification, and validation rules. A robust workflow does not treat extraction as an isolated prediction. It records provenance, confidence, extracted fields, validation results, and human corrections.

For example, a procurement system may identify supplier names, purchase-order numbers, tax values, dates, and line items. It can compare extracted values with business rules and flag discrepancies. Structured validation reduces the danger that a fluent but incorrect model output will enter a transaction system.

Knowledge graphs provide another approach. They represent entities as nodes and relationships as edges, often with types, timestamps, and sources. Graphs support entity resolution, semantic search, recommendation, and reasoning over linked facts. Their quality depends on schema design, identity matching, update processes, and provenance. A knowledge graph should distinguish observed facts, inferred relations, and uncertain claims.

6. Multimodal Representation and Reasoning

Multimodal intelligence requires more than placing different inputs beside one another. The system must determine how modalities correspond, which source is reliable, and how evidence should influence a conclusion.

A common framework includes three levels of fusion:

• Early fusion: features from several modalities are combined before substantial task-specific processing. This can capture fine-grained relationships but requires synchronized, compatible representations.
• Intermediate fusion: each modality is encoded separately and representations interact through attention or cross-modal layers. This provides a balance between specialization and integration.
• Late fusion: independent models produce decisions that are combined through voting, weighted averaging, or a learned decision layer. It is easier to implement and may be robust when modalities are missing, but it can lose detailed cross-modal relationships.

Cross-attention allows one modality to select relevant information from another. In image captioning, language generation may attend to regions of an image. In visual question answering, the question guides visual attention. In video understanding, temporal attention connects frames, audio, and subtitles.

Multimodal systems must address alignment, synchronization, missing modalities, conflicting evidence, and modality dominance. If a spoken statement conflicts with an image, the system needs a task-specific policy rather than an arbitrary preference. In an industrial setting, a sensor reading may be more reliable than a noisy photograph; in a social interaction, prosody may modify the literal text.

A multimodal model can be viewed as performing evidence integration:

P(hypothesis | text, image, audio) ∝ P(text, image, audio | hypothesis) P(hypothesis).

In practice, neural systems learn an approximation to such integration. The equation is useful conceptually because it highlights prior knowledge, evidence quality, and uncertainty. A system should not treat every modality as equally informative in every situation.

Human–AI interaction and grounding

Human–AI interaction concerns how people understand, control, correct, and collaborate with intelligent systems. Effective interaction requires discoverable capabilities, comprehensible feedback, appropriate timing, and meaningful recovery from errors. Multimodal interfaces may combine speech, text, gaze, gesture, touch, and visual displays.

Grounding occurs when a system and user establish what an expression refers to. If a user says “move that file,” the system must identify the referent, verify the intended operation, and perhaps request confirmation. Shared visual context can reduce ambiguity, but it can also create privacy risks. For consequential actions, confirmation and reversibility are important design principles.

7. Quality Assessment: Beyond a Single Accuracy Number

Evaluation should begin with the intended use, possible harms, and acceptable error trade-offs. A useful evaluation plan combines offline metrics, stress tests, human assessment, and post-deployment monitoring.

For classification, precision measures the proportion of predicted positives that are correct, while recall measures the proportion of actual positives that are detected. F1 combines them through their harmonic mean. Calibration measures whether predicted probabilities correspond to observed frequencies. A system with high accuracy but poor calibration may be unsuitable for risk-sensitive decisions.

Generative language evaluation can include n-gram overlap measures, semantic similarity, factuality checks, citation verification, human judgments, and task success. Automatic metrics are useful for comparison but do not fully measure usefulness, originality, cultural appropriateness, or hallucination. Human evaluation should use explicit rubrics and multiple evaluators where feasible.

Robustness evaluation tests perturbations such as spelling errors, accent variation, paraphrase, image blur, lighting changes, missing modalities, adversarial inputs, and domain shift. Fairness evaluation compares performance and error types across relevant groups. Privacy evaluation examines memorization, data leakage, access control, and the possibility of inferring sensitive attributes.

Operational metrics include response latency, throughput, availability, cost per interaction, energy use, escalation rate, and correction effort. A model that is slightly more accurate but too slow for a live conversation may be inferior to a smaller model. Quality is therefore a multidimensional engineering and governance decision.

8. Applications Across Organizations

In healthcare, language models can support clinical-document search, coding assistance, patient communication, and summarization. Vision models can assist image analysis, but clinical responsibility, data protection, validation, and human review remain essential. In manufacturing, vision inspection, speech-enabled maintenance, and multimodal fault diagnosis can reduce downtime. In education, intelligent tutoring systems can combine written answers, speech, diagrams, and learner interaction, provided that assessment fairness and student privacy are protected.

Retail and logistics use conversational agents, document extraction, visual product search, demand-related information services, and warehouse perception. Public-sector applications include multilingual access, document processing, emergency-call transcription, and accessibility tools. In each case, deployment should begin with a clearly defined task, decision authority, evidence requirements, and escalation process.

9. Illustrative Case Study: Multimodal Maintenance Assistant

Background

Consider a fictional manufacturing organization, Orion Components, operating automated assembly lines. Technicians diagnose faults using sensor dashboards, maintenance manuals, photographs, and spoken notes. The organization proposes a multimodal assistant to reduce diagnostic time.

Situation and problem

A technician may report, “The motor is making a high-pitched sound,” attach a photograph of a damaged cable, and provide a machine identifier. Relevant information is scattered across manuals and previous service records. Existing keyword search performs poorly when terminology varies. However, an incorrect recommendation could cause equipment damage or injury.

Analysis

The task is not simply question answering. The system must transcribe speech, identify equipment and symptoms, interpret the photograph, retrieve authorized maintenance documents, compare possible causes, and distinguish evidence from hypothesis. It must also handle missing images, poor audio, outdated manuals, and conflicting sensor readings.

The team defines success as reduced time to locate relevant procedures, high recall for safety-critical warnings, traceable citations, low hallucination rate, and a mandatory human decision for repair authorization. It selects a speech recognizer adapted to plant vocabulary, an OCR and vision pipeline for equipment labels, a retrieval system over version-controlled manuals, and a language model constrained to cite retrieved evidence.

Approach and safeguards

The assistant performs the following sequence:

1. Transcribes the technician's note and displays the transcript for correction.

2. Extracts the machine identifier, symptom, location, and urgency.

3. Uses the image to identify visible cable damage, while reporting confidence and limitations.

4. Retrieves relevant procedures, safety warnings, and prior resolved cases.

5. Produces a ranked list of possible causes, each linked to evidence.

6. Asks clarifying questions when the machine identity or symptom is uncertain.

7. Requires technician confirmation before creating a work order or recommending a hazardous action.

Evaluation uses WER for technical speech, field-level extraction accuracy, retrieval recall, citation correctness, diagnostic ranking quality, response latency, and technician task-completion time. Safety tests include deliberately misleading images, contradictory sensor data, missing manuals, and attempts to bypass authorization.

Outcome

In this illustrative scenario, the pilot does not measure success solely by the fluency of responses. It finds that evidence-linked answers and effective clarification are more valuable than unrestricted generation. The assistant is approved for search and documentation support but not for autonomous repair decisions. A feedback process records corrections, updates the knowledge base, and monitors whether performance changes after equipment or manual revisions.

Learning points

The case demonstrates that multimodal intelligence is a system-design problem. Model selection, document governance, user interface, uncertainty communication, access control, and evaluation must be designed together. Human oversight is not an afterthought; it is part of the operating model.

10. Challenges, Ethics, and Governance

Language and perception systems can reproduce social bias, misrepresent minority dialects, or perform unevenly across demographic groups. Fairness cannot be guaranteed by removing sensitive attributes because proxies may remain in language, images, location, or behavior. Evaluation should examine representation, subgroup performance, error severity, and the consequences of deployment.

Privacy is especially important for voice, face, location, medical documents, and workplace video. Organizations should define lawful purpose, data minimization, retention periods, access controls, encryption, audit logs, and procedures for deletion or correction. Biometric identification requires particularly careful justification and governance.

Security threats include prompt injection, data poisoning, model extraction, voice spoofing, adversarial images, unauthorized tool use, and leakage of confidential documents. A model connected to external systems should operate with least privilege, input validation, output filtering, rate limits, and transaction confirmation.

Generative systems may hallucinate facts, imitate individuals, create misleading media, or expose copyrighted or confidential material. Provenance indicators, source citations, content labeling, retrieval controls, and human review reduce but do not eliminate these risks. Organizations should assign accountability: someone must own the data, model, decision, incident response, and user communication.

A common mistake is to deploy a general model without defining the decision boundary. Another is to optimize an average benchmark score while ignoring rare but severe errors. Further mistakes include treating user corrections as optional, failing to version documents, and assuming that a model's confidence score is a reliable probability without calibration.

11. Current Developments and Emerging Directions

Foundation models increasingly unify text, image, audio, and video processing. Multimodal assistants can interpret documents, discuss images, analyze recorded meetings, and invoke external tools. Smaller specialized models are also important because they may provide lower latency, lower cost, improved privacy, and easier deployment at the edge.

Retrieval-augmented generation is developing toward richer evidence management, including structured databases, knowledge graphs, tables, and temporal information. Agentic systems combine language reasoning with planning and tool execution, but their autonomy increases the need for authorization, verification, and observability.

On-device and edge intelligence can reduce transmission of sensitive data and improve responsiveness. Synthetic data and simulation support training when real data are scarce, although synthetic data may reproduce the assumptions and biases of its generators. Research continues on causal and grounded multimodal reasoning, continual learning, efficient adaptation, uncertainty estimation, and interfaces that support accessibility without excessive surveillance.

Best Practices for Design and Deployment

A professionally managed language or multimodal system should:

• define the user task, decision authority, and unacceptable failures before selecting a model;
• use representative, documented, and version-controlled data;
• establish modality-specific and end-to-end evaluation measures;
• test subgroup performance, robustness, privacy, security, and domain shift;
• ground consequential answers in authoritative sources;
• communicate uncertainty and preserve a human escalation path;
• constrain tool access and require confirmation for irreversible actions;
• log inputs, outputs, evidence, corrections, and model versions subject to privacy requirements;
• monitor performance after deployment and provide mechanisms for correction;
• review accessibility, cultural context, and environmental cost.

Chapter Summary

Language, perception, and multimodal intelligence enable AI systems to work with text, speech, images, video, and other sensory information. Language intelligence includes representation, classification, extraction, retrieval, generation, and dialogue management. Speech systems transform audio and text while facing challenges from noise, accents, vocabulary, and identity misuse. Computer vision supports classification, detection, segmentation, tracking, OCR, and visual reasoning.

Multimodal systems align and fuse information through early, intermediate, or late fusion. Their reliability depends on synchronization, grounding, missing-modality handling, evidence quality, and uncertainty management. Evaluation must combine task metrics with human judgments, robustness, fairness, privacy, latency, cost, and operational outcomes. Practical deployment requires document governance, secure tool use, human oversight, provenance, and continuous monitoring. The central professional lesson is that intelligent interaction is not achieved by a model alone; it emerges from the coordinated design of data, models, interfaces, workflows, and governance.

Key Terms

Automatic Speech Recognition (ASR): Conversion of spoken audio into written text.

Computer Vision: AI methods for interpreting images, video, and visual scenes.

Dialogue State Tracking: Representation of goals, entities, history, and unresolved information in a conversation.

Embedding: A learned numerical representation of an object, token, image, or other input.

Entity Extraction: Identification of meaningful objects such as people, dates, organizations, or products in data.

Grounding: Connecting an AI output to external evidence, real-world entities, or a specific context.

Intersection over Union: The overlap between predicted and reference regions divided by their union.

Knowledge Graph: A graph representation of entities and typed relationships, often with provenance.

Large Language Model: A neural language model trained on large text collections to predict and generate language.

Modality: A channel or form of information, such as text, image, audio, or video.

Multimodal Fusion: Combination of representations or decisions from multiple information modalities.

Named-Entity Recognition: Identification and classification of named entities in text.

Precision: The proportion of positive predictions that are correct.

Recall: The proportion of actual positive cases detected by a system.

Retrieval-Augmented Generation: Generation conditioned on information retrieved from an external collection.

Semantic Segmentation: Assignment of a semantic class to each pixel in an image.

Tokenization: Division of text into computational units processed by a language model.

Word Error Rate: Speech-recognition errors divided by the number of words in the reference transcript.

Review Questions

Short Answer Questions

1. What is grounding, and why is it important in conversational AI?

2. Distinguish semantic segmentation from object detection.

3. State the components of word error rate.

4. What is the purpose of dialogue state tracking?

5. Define intermediate multimodal fusion.

Long Answer Questions

1. Explain the architecture and evaluation of a conversational AI system.

2. Discuss the major tasks in computer vision and the metrics appropriate to them.

3. Explain how retrieval-augmented generation can improve enterprise question answering and identify its limitations.

4. Compare early, intermediate, and late multimodal fusion.

5. Analyse the ethical, privacy, and security issues in speech and vision applications.

Analytical / Application Questions

1. Design an evaluation plan for a multilingual customer-service assistant that handles text and speech. Justify the metrics, subgroup tests, and escalation criteria.

2. A visual inspection system has high overall accuracy but misses defects under poor lighting. Diagnose the likely causes and propose a deployment and monitoring strategy.

3. Design a multimodal AI assistant for a safety-sensitive workplace. Specify its data sources, workflow, human controls, evidence requirements, and failure-handling mechanisms.

22. Multiple Choice Questions with Answers

Question 1

A conversational system receives a vague request and asks a targeted follow-up question before acting. Which component is primarily responsible for this behaviour?

A. Dialogue policy

B. Optical character recognition

C. Image segmentation

D. Speaker identification

Correct Answer: A. Dialogue policy

Question 2

Which evaluation measure is specifically designed to quantify substitutions, deletions, and insertions in speech transcription?

A. Intersection over Union

B. Word Error Rate

C. Mean IoU

D. Calibration error

Correct Answer: B. Word Error Rate

Question 3

A system identifies each separate vehicle in an image and assigns pixels to each vehicle instance. This is best described as:

A. Image classification

B. Semantic segmentation only

C. Instance segmentation

D. Language identification

Correct Answer: C. Instance segmentation

Question 4

Why can retrieval-augmented generation improve an enterprise question-answering system?

A. It guarantees that every generated statement is true

B. It provides relevant external evidence that can ground generation

C. It eliminates the need for document version control

D. It prevents all prompt-injection attacks

Correct Answer: B. It provides relevant external evidence that can ground generation

Question 5

In late multimodal fusion, a system generally:

A. Combines raw pixels and waveforms before encoding

B. Uses cross-attention between modalities at every layer

C. Combines decisions or representations produced by separate modality models

D. Discards all modality-specific processing

Correct Answer: C. Combines decisions or representations produced by separate modality models

Question 6

A medical transcription model has a low overall WER but frequently misrecognizes drug names. What is the most appropriate conclusion?

A. The model is automatically safe because its average WER is low

B. Task-critical vocabulary requires separate error analysis and validation

C. WER is irrelevant to speech recognition

D. The model should be evaluated only by response fluency

Correct Answer: B. Task-critical vocabulary requires separate error analysis and validation

Question 7

Which design choice best reduces the risk of an assistant performing an irreversible action incorrectly?

A. Allowing unrestricted tool access

B. Removing all uncertainty messages

C. Requiring authorization and user confirmation before execution

D. Optimizing only for shorter responses

Correct Answer: C. Requiring authorization and user confirmation before execution

Question 8

A vision model performs well in testing but fails after cameras are moved to a darker factory area. This is most directly an example of:

A. Domain shift

B. Tokenization

C. Dialogue-state tracking

D. Entity linking

Correct Answer: A. Domain shift

Question 9

Which practice most directly supports traceability in an information-extraction workflow?

A. Storing only the final extracted value

B. Recording source documents, confidence, and validation results

C. Removing human corrections from the system log

D. Using a single unversioned knowledge base

Correct Answer: B. Recording source documents, confidence, and validation results

Question 10

When text, an image, and a sensor reading provide conflicting evidence, a reliable multimodal system should primarily:

A. Assume all modalities have equal reliability

B. Choose the modality with the longest input

C. Apply task-specific evidence, uncertainty, and escalation policies

D. Ignore the conflict and generate the most fluent response

Correct Answer: C. Apply task-specific evidence, uncertainty, and escalation policies