Specialized and Multimodal Retrieval

Introduction

Information retrieval is often introduced through text documents, inverted indexes, and keyword queries. Real information environments are considerably broader. A digital library may contain articles, tables, maps, photographs, recordings, sensor streams, and video. A hospital may need to retrieve clinically similar images together with multilingual reports. An intelligence analyst may search for locations, people, events, and relationships across text, satellite imagery, and video. An e-commerce platform may allow a user to submit a photograph and retrieve visually similar products.

Specialized and multimodal retrieval adapts retrieval principles to the structure, semantics, and interaction requirements of particular data types. The central challenge is not merely indexing more formats. It is representing heterogeneous objects so that relevance can be estimated across modalities, languages, domains, and geographic or temporal constraints.

A useful abstraction is to represent an item as a multimodal object:

Here, each component may be absent, and each modality has its own representation, index, uncertainty, and relevance criteria. A retrieval system must therefore address five linked tasks: representation, indexing, query interpretation, ranking, and evaluation.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain how retrieval methods differ for structured, visual, auditory, video, geospatial, multilingual, and domain-specific data.
2. Select appropriate representations, indexes, similarity measures, and ranking strategies for specialized collections.
3. Explain early, late, and hybrid multimodal fusion.
4. Design a retrieval pipeline for a cross-modal application.
5. Evaluate specialized retrieval systems using effectiveness, efficiency, robustness, and fairness criteria.
6. Identify governance, privacy, copyright, and domain-safety risks.

1. Why Specialized Retrieval Requires Adaptation

A conventional text retrieval system assumes that documents can be represented by terms and that relevance is largely related to textual topical similarity. These assumptions become inadequate in several situations:

• A photograph has no reliable term sequence until it is captioned, tagged, or converted into visual features.
• A table has relationships between rows, columns, cells, and units that a bag-of-words representation may destroy.
• A video contains objects, speech, music, actions, scenes, and temporal events.
• A map query may depend on distance, containment, direction, route accessibility, or time.
• A multilingual collection may express the same concept through different scripts and linguistic structures.
• A legal, biomedical, or scientific collection uses terminology whose meaning depends strongly on domain context.

Consequently, “similarity” is not universal. For two images it may mean visual appearance, object identity, or scene function. For two locations it may mean physical distance, travel time, or shared administrative region. For two medical records it may mean comparable diagnoses, treatments, or patient trajectories.

A specialized retrieval design should begin by specifying:

1. Information need: What decision or task does the user have?
2. Object semantics: What properties make two objects relevantly similar?
3. Query modality: Is the query text, image, speech, coordinates, structured constraints, or a combination?
4. Evidence and risk: What errors are acceptable, and what evidence must be shown to the user?
5. Operational constraints: What are the latency, storage, privacy, and update requirements?

2. Retrieval over Structured Data

Structured data includes relational tables, knowledge graphs, XML or JSON records, spreadsheets, and semi-structured documents. Unlike ordinary text, structured data contains explicit schema, types, keys, relationships, and constraints.

2.1 Relational and semi-structured retrieval

A relational query specifies predicates over attributes and relationships. For example, a query for hospitals within 10 kilometres of a flood zone and possessing intensive-care beds combines spatial, numerical, and categorical conditions. Keyword search alone is insufficient because “10 kilometres” is a constraint rather than a topic term.

Structured retrieval commonly combines:

• Boolean predicates: equality, conjunction, disjunction, and negation.
• Range predicates: values between specified bounds.
• Aggregation: counts, averages, sums, and rankings.
• Faceted search: progressive filtering by attributes.
• Text search within fields: approximate matching over names, descriptions, or notes.
• Schema-aware expansion: mapping user phrases to field names and values.

A practical architecture uses a query planner. Exact filters are applied first when they substantially reduce the candidate set; text or semantic ranking is then applied to the remaining records. This is often more efficient and more interpretable than embedding every field into one vector.

2.2 Entity resolution and record linkage

The same entity can appear under different names, spellings, identifiers, or languages. Entity resolution links records that refer to the same real-world object. Evidence may include names, addresses, dates, identifiers, and relational context. False merges can be more damaging than missed links, especially in government, finance, and healthcare systems.

2.3 Knowledge-graph retrieval

A knowledge graph represents entities as nodes and typed relationships as edges. Queries may ask for paths, related entities, or explanations. Graph retrieval can use exact graph patterns, graph embeddings, or hybrid methods. Embeddings support approximate discovery, but graph constraints are necessary when relation type and direction matter. For example, “drugs that inhibit enzyme E” is not equivalent to “drugs associated with enzyme E.”

3. Image Retrieval

Image retrieval systems search collections using visual content, text associated with images, or both.

3.1 Content-based image retrieval

Content-based image retrieval represents an image through visual features such as colour distributions, texture, shape, local keypoints, or learned neural embeddings. Traditional features may remain useful for constrained industrial settings, while deep embeddings are effective for semantic similarity over diverse images.

Let an image encoder map image x to a d-dimensional vector:

The parameters θ are learned or transferred from another task. A common normalized similarity is cosine similarity:

The vectors must be non-zero. If vectors are normalized to unit length, cosine similarity equals their dot product. Cosine similarity measures orientation, not absolute magnitude, and is therefore appropriate when vector direction carries the semantic information.

Euclidean distance is another option:

Its interpretation depends on the embedding space and training objective. It should not be assumed equivalent to semantic similarity merely because the vectors are numeric.

3.2 Query-by-example and semantic image search

In query-by-example, the user supplies an image and the system retrieves nearby images in feature space. In semantic image search, a text query such as “red hiking backpack” is mapped into the same or a compatible space as images. Joint image–text models enable cross-modal retrieval, but performance depends on training data, domain shift, compositional understanding, and sensitivity to fine-grained attributes.

3.3 Relevance dimensions

Image retrieval may optimize different objectives:

• Appearance similarity: colour, layout, or texture.
• Object similarity: the same object or product type.
• Semantic similarity: related concepts or activities.
• Fine-grained similarity: model, species, disease pattern, or manufacturing defect.
• Provenance similarity: same source, photographer, or event.

A system must expose or infer which objective the user intends. A visually similar shoe may be a poor result if the user needs the same brand and model.

4. Audio and Speech Retrieval

Audio retrieval covers speech, music, environmental sound, and acoustic events. The first design decision is whether the information need concerns linguistic content, acoustic identity, or an event.

4.1 Speech retrieval

Automatic speech recognition (ASR) converts audio to text, after which conventional retrieval can be applied. However, ASR introduces errors caused by accents, noise, code-switching, overlapping speakers, and specialist vocabulary. A robust system stores time-aligned transcripts and permits retrieval of the relevant audio segment, not merely the whole recording.

Useful metadata includes speaker identity, turn boundaries, language, confidence, and timestamps. Speaker diarization separates “who spoke when,” but diarization errors can affect attribution and downstream search.

4.2 Acoustic retrieval

Music and environmental sound retrieval may use spectral descriptors, rhythm, timbre, or learned embeddings. Queries can be a hummed melody, a short audio clip, a textual description, or an event label such as “glass breaking.” Acoustic similarity is not necessarily semantic similarity: two recordings can sound alike while representing different events.

4.3 Audio indexing and evaluation

Long recordings are segmented into windows or semantically meaningful events. Indexing may use approximate nearest-neighbour search over embeddings, inverted indexes over transcripts, or both. Evaluation should measure timestamp accuracy when the system is expected to locate an event within a recording.

5. Video Retrieval

Video is a temporally ordered multimodal object. It contains frames, shots, scenes, audio, speech, subtitles, text overlays, detected entities, and actions.

5.1 Levels of video representation

A practical hierarchy is:

1. Frame level: objects, faces, colours, and text in individual frames.
2. Shot level: a continuous camera segment.
3. Scene level: a coherent setting or event.
4. Clip level: a user-defined temporal interval.
5. Video level: the complete program or recording.

Indexing only at the video level creates poor temporal precision. Indexing every frame creates excessive storage and redundant results. Shot boundary detection, key-frame selection, and temporal embeddings provide a compromise.

5.2 Multimodal video queries

A query such as “the interview where the minister discusses energy prices” requires speech retrieval, named-entity recognition, and temporal localization. “Find footage showing a cyclist falling near a bridge” requires visual action recognition and possibly geospatial or metadata constraints.

Ranking may aggregate evidence from transcripts, frames, audio, captions, and metadata. Results should display a representative thumbnail and a start time so that users can verify relevance efficiently.

6. Geospatial Retrieval

Geospatial retrieval incorporates coordinates, geometry, topology, distance, direction, scale, and time. Spatial relationships are often more important than textual descriptions.

6.1 Spatial predicates

Common predicates include:

• Distance: within a radius or nearest to a point.
• Containment: inside a city, region, parcel, or administrative boundary.
• Intersection: roads crossing a flood zone.
• Adjacency: neighbouring polygons or connected network segments.
• Direction: north of, downstream from, or near a coastline.
• Route accessibility: reachable under a travel-time or transport constraint.

For points with latitude and longitude, a spherical-distance approximation is the haversine formula:

Here, d is surface distance, R is the assumed Earth radius, φ is latitude in radians, and λ is longitude in radians. The formula assumes a spherical Earth and is suitable for many regional distance calculations, but high-precision surveying should use an appropriate ellipsoidal model. The output has the same distance unit as R.

6.2 Spatial indexing

R-trees index minimum bounding rectangles and support intersection and range queries. Geohashes partition space into hierarchical cells and are useful for approximate locality, caching, and prefix searches. Quadtrees recursively subdivide two-dimensional space. Network indexes are required when travel distance differs substantially from straight-line distance.

6.3 Geo-text retrieval

A query such as “quiet cafés near the university open after 9 p.m.” combines geographic filtering, temporal constraints, and textual or semantic ranking. A robust system applies hard constraints first, then ranks candidates using relevance, distance, popularity, freshness, and uncertainty. It should distinguish “near” as a user preference from a strict boundary.

7. Multilingual Retrieval

Multilingual information retrieval (MLIR) enables queries in one language to retrieve documents in another. It is essential for international news, public services, scientific communication, and global commerce.

7.1 Main approaches

1. Query translation: translate the query into the document language, then retrieve.
2. Document translation: translate documents into the query language, then retrieve.
3. Interlingual representation: map queries and documents into a shared semantic space.
4. Language-specific retrieval with fusion: search each language separately and combine results.

Query translation is computationally efficient for large collections but can fail on ambiguity and named entities. Document translation supports richer interaction but is expensive and may propagate systematic translation errors. Shared multilingual encoders support cross-language retrieval but can underrepresent low-resource languages.

7.2 Language identification and normalization

Language identification should operate at document, field, and sometimes sentence level. Collections may contain code-switching, transliteration, dialect variation, diacritics, and multiple scripts. Normalization must be language-aware; aggressive stemming or stop-word removal can harm languages with different morphology or writing systems.

7.3 Evaluation and fairness

Performance should be reported by language, not only as a global average. Translation quality and retrieval quality are distinct. A system may produce fluent translations while omitting culturally specific or politically sensitive meanings. Human evaluation by qualified speakers remains important, particularly for low-resource languages.

8. Domain-Specific Retrieval

General-purpose retrieval models often perform poorly in specialist corpora because terminology, document structure, evidence standards, and relevance criteria differ by domain.

8.1 Domain adaptation

Adaptation may involve:

• specialist tokenization and terminology dictionaries;
• ontologies, controlled vocabularies, and synonym maps;
• field-aware indexing, such as title, abstract, claims, or findings;
• domain-specific pretraining or fine-tuning;
• citation, authority, or provenance signals;
• expert relevance judgments;
• retrieval-augmented generation with source attribution.

In biomedical retrieval, abbreviations and named entities require disambiguation. In legal retrieval, jurisdiction, date, precedent status, and exact wording matter. In technical maintenance, part numbers and version identifiers may dominate semantic similarity.

8.2 Evidence and explainability

Specialized users often need evidence rather than merely a ranked list. A legal system should expose the relevant passage, citation, court, and date. A clinical system should show the source study, population, outcome, and publication status. Explanations should be faithful to the ranking process and should not imply certainty beyond the evidence.

9. Multimodal Fusion and Cross-Modal Retrieval

Multimodal retrieval combines evidence from two or more modalities. The system may accept text and return images, accept an image and return text, or search a collection using a mixture of constraints.

9.1 Early fusion

Early fusion combines modality representations before ranking. If image and text vectors are compatible, a joint vector may be constructed:

Here, [· ; ·] denotes concatenation, α and β are scaling weights, and the vectors must be appropriately normalized. Early fusion can learn interactions between modalities, but it requires aligned data and can be sensitive to missing or noisy modalities.

9.2 Late fusion

Late fusion ranks each modality separately and combines scores:

The index m denotes a modality, sₘ is its normalized score, and wₘ is its weight. Scores must be calibrated or normalized before combination because a cosine score, a probability, and a BM25 score are not directly comparable. Late fusion is robust to missing modalities and allows independent index updates.

9.3 Hybrid and learned fusion

Hybrid systems use structured filters, lexical retrieval, dense retrieval, and modality-specific rerankers. A common production pattern is:

1. Apply mandatory filters.
2. Retrieve candidates using lexical and dense indexes.
3. Fuse candidate lists.
4. Rerank with a cross-modal model.
5. Apply diversity, safety, freshness, and business rules.
6. Present evidence and provenance.

Learned fusion can estimate weights from judged relevance data. However, learned ranking may exploit spurious correlations, overrepresent popular content, or fail under distribution shift. Validation should include adversarial, out-of-domain, and missing-modality tests.

9.4 Metric learning and contrastive objectives

Cross-modal encoders are often trained so that matched pairs are close and mismatched pairs are far apart. For a query q and positive item d⁺, a contrastive loss over a batch B can be written as:

Here, τ > 0 is a temperature parameter, sim is a similarity function, and B contains candidate items including the positive. This formulation assumes that batch negatives are sufficiently informative and that labelled positives are reliable. False negatives, incomplete captions, and culturally biased data can distort learning.

10. Evaluation of Specialized Retrieval

Traditional metrics remain useful but must be adapted to the task.

• Precision@k: proportion of the top k results judged relevant.
• Recall@k: proportion of all relevant items retrieved in the top k, when the relevant set is known.
• Mean Average Precision: rewards early retrieval across multiple queries.
• Normalized Discounted Cumulative Gain (nDCG): supports graded relevance.
• Mean Reciprocal Rank: emphasizes the first relevant result.
• Intersection-over-Union: useful for spatial or temporal localization.
• Recall at a fixed latency: combines effectiveness and operational performance.
• Calibration and abstention quality: important when the system should indicate uncertainty.

For graded relevance, discounted cumulative gain at rank k is:

Here, relᵢ is the relevance grade at rank i. nDCG divides DCG by the ideal DCG for that query, making scores comparable across queries with different numbers of relevant items. The logarithmic discount assumes that users inspect higher-ranked results more closely.

Specialized evaluation should also test:

• performance by language, demographic group, geography, device, and modality;
• robustness to noise, missing fields, and adversarial content;
• temporal freshness and index update delay;
• duplicate and near-duplicate suppression;
• user effort, verification time, and task success;
• energy, storage, and serving cost.

11. Worked Design Example: Multimodal Disaster-Response Search

Background

The following is an illustrative case. A national emergency agency maintains satellite images, drone video, incident reports, weather feeds, road networks, and multilingual public messages. Analysts need to locate damaged bridges and identify accessible routes after a major storm.

Problem

A keyword-only system misses relevant evidence because bridge damage may be visible in imagery but absent from reports. A purely visual system cannot apply road-accessibility and administrative constraints. Reports occur in several languages, and drone videos contain the most recent evidence.

Analysis

The information need has four dimensions:

1. Object: bridges and road segments.
2. Evidence: images, video frames, text reports, and sensor observations.
3. Constraints: region, time window, damage type, and route status.
4. Decision: prioritize inspection and relief routes.

Relevance is therefore a combination of object identity, damage evidence, recency, geographic relation, and operational usefulness.

Approach

The agency builds a multimodal index with:

• OCR and multilingual ASR for reports and videos;
• image and video embeddings for visual similarity;
• a geospatial index for bridges, roads, flood polygons, and coordinates;
• entity resolution linking bridge names, identifiers, and locations;
• temporal metadata for acquisition time and report time;
• a knowledge graph linking incidents, infrastructure, evidence, and agencies.

For a query such as “bridges near River X damaged after 12 June and reachable from the eastern depot,” the system first applies time and geographic filters. It retrieves text passages containing bridge identifiers and damage terms, then searches image and video embeddings for visually similar damage. A route engine estimates travel accessibility. A late-fusion ranker combines evidence only after scores are normalized:

The weights are illustrative and should be learned or validated with expert judgments. Mandatory safety constraints, such as “evidence acquired after the specified date,” are filters rather than soft ranking weights.

Outcome

The result page presents ranked bridges, a map, the strongest supporting image or video frame, transcript excerpts, acquisition times, confidence indicators, and route status. Analysts can reject false matches and provide judgments for subsequent evaluation. The system does not automatically declare a bridge unsafe; it supports human assessment.

Learning points

1. Different modalities supply complementary evidence.
2. Hard constraints should not be hidden inside an arbitrary score.
3. Temporal and spatial metadata are part of relevance.
4. Evidence provenance is essential in high-consequence settings.
5. Evaluation must measure both retrieval quality and decision usefulness.

12. Engineering Architecture and Best Practices

A production system should separate modality-specific pipelines from a common retrieval and governance layer.

Ingestion and representation

Preserve original files, normalized derivatives, extraction confidence, timestamps, licensing information, and provenance. Generate multiple representations where necessary: sparse lexical features, dense vectors, structured fields, thumbnails, transcripts, and spatial geometries.

Indexing

Use specialized indexes rather than forcing all data into one store:

Approximate nearest-neighbour indexes improve latency at the cost of some recall. Their parameters should be tuned against exact-search samples, and index versions should be reproducible.

Ranking and interaction

Use progressive disclosure: filters and facets for explicit constraints, semantic search for exploratory discovery, and evidence previews for verification. Deduplicate results across sources and diversify by source, event, location, or viewpoint when repeated content would crowd out useful alternatives.

Monitoring

Monitor query latency, empty-result rate, click and completion signals, language-specific performance, drift in embedding distributions, stale metadata, and changes in source composition. User clicks are not pure relevance labels because position, presentation, and popularity influence them.

13. Ethical, Legal, and Governance Considerations

Specialized retrieval can expose sensitive information more effectively than ordinary search. Images may reveal faces, children, health conditions, or precise locations. Audio can contain biometric or private speech. Geospatial retrieval can enable surveillance. Domain systems can amplify discriminatory terminology or unequal access to evidence.

Important controls include:

• purpose limitation and data minimization;
• access control at document, field, frame, and coordinate level;
• encryption and retention policies;
• consent and lawful processing for personal data;
• copyright and licence tracking for images, recordings, and datasets;
• audit logs for queries and exports;
• redaction of sensitive entities or locations;
• provenance and source-quality indicators;
• human review for high-impact decisions;
• testing for demographic, linguistic, and geographic disparities.

A retrieval system should distinguish retrieval confidence from truth. A high score means that an item resembles the query under the model; it does not establish that the underlying claim is correct. In regulated domains, this distinction should be visible in the interface and documentation.

14. Contemporary Developments and Research Directions

Recent systems increasingly use multimodal foundation models, vector databases, retrieval-augmented generation, and agentic query planning. These technologies improve natural-language interaction but introduce new risks. A generative model may describe an image incorrectly, cite an irrelevant passage, or conceal uncertainty. Retrieval-augmented generation reduces unsupported generation only when retrieval is relevant, complete, and correctly grounded.

Important research directions include:

• better alignment for low-resource languages and modalities;
• retrieval from long video and continuous sensor streams;
• privacy-preserving and federated multimodal indexes;
• uncertainty-aware and abstaining retrieval;
• causal and counterfactual evaluation of ranking interventions;
• energy-efficient embedding and reranking;
• provenance-preserving multimodal generation;
• benchmarks that measure real user tasks rather than only static relevance labels.

The field is moving from “find similar items” toward “support evidence-based decisions across heterogeneous information.” That shift requires technical innovation together with careful evaluation and governance.

Chapter Summary

Specialized and multimodal retrieval adapts information retrieval to structured records, images, audio, video, geospatial objects, multilingual collections, and expert domains. Each modality has distinct representations, indexes, similarity measures, and failure modes. Structured retrieval relies on schema, predicates, aggregation, and graph relationships; image and audio retrieval often use learned embeddings; video requires temporal segmentation; geospatial retrieval uses topology, distance, and spatial indexes; multilingual retrieval requires translation or shared representations; and domain-specific retrieval depends on terminology, authority, provenance, and expert evidence.

Multimodal systems combine evidence through early, late, or hybrid fusion. Score normalization, hard constraints, missing-modality handling, and transparent evidence presentation are essential. Evaluation must extend beyond conventional ranking metrics to localization, calibration, latency, fairness, robustness, and user task success. In sensitive applications, privacy, copyright, provenance, access control, and human oversight are core design requirements rather than optional additions.

Key Terms

Multimodal retrieval: Retrieval that uses or returns more than one information modality.

Cross-modal retrieval: Retrieval in which the query and result use different modalities.

Content-based image retrieval: Image search based on visual content rather than only metadata.

Embedding: A numerical vector representation intended to preserve useful semantic relationships.

Cosine similarity: Similarity based on the angle between two non-zero vectors.

Approximate nearest-neighbour search: Efficient search for vectors close to a query without exhaustively comparing every vector.

Structured retrieval: Search over records with explicit fields, types, constraints, or relationships.

Entity resolution: Linking records that refer to the same real-world entity.

Knowledge graph: A graph of entities and typed relationships.

Geospatial retrieval: Retrieval using coordinates, geometry, topology, distance, or spatial relations.

Haversine formula: A spherical-Earth approximation for the distance between two latitude–longitude points.

Multilingual information retrieval: Retrieval across more than one language.

Domain adaptation: Modification of a retrieval model or pipeline for a specialist domain.

Early fusion: Combining modality representations before ranking.

Late fusion: Combining separately computed modality scores after retrieval.

Contrastive learning: Learning representations by bringing matched examples closer and separating mismatched examples.

Provenance: Information about an item’s source, history, time, and transformations.

nDCG: A graded-relevance ranking metric normalized by the ideal ranking.

Review Questions

Short Answer Questions

1. Why can a bag-of-words model be inadequate for retrieving structured records?
2. Distinguish appearance similarity from semantic similarity in image retrieval.
3. What problem does temporal segmentation solve in video retrieval?
4. Name two spatial predicates used in geospatial retrieval.
5. Why must modality-specific scores be normalized before late fusion?

Long Answer Questions

1. Compare query translation, document translation, and shared interlingual representations for multilingual retrieval.
2. Explain how structured filters and semantic ranking can be combined in a production retrieval system.
3. Discuss the representation, indexing, and evaluation challenges of audio retrieval.
4. Compare early, late, and hybrid multimodal fusion, including their advantages and limitations.
5. Design an evaluation framework for a domain-specific retrieval system used by expert professionals.

Analytical / Application Questions

1. Design a retrieval architecture for a museum collection containing multilingual descriptions, photographs, audio guides, and geospatial exhibition data. Justify the indexes and fusion strategy.
2. A disaster-response system returns visually similar bridge images but many are from before the flood. Explain how temporal constraints, metadata, and ranking should be redesigned.
3. A legal cross-language retrieval system performs well in English but poorly in a low-resource language. Propose a diagnosis and improvement plan covering data, models, evaluation, and governance.

Multiple Choice Questions with Answers

Question 1

Which retrieval method is most directly suited to a query requiring records inside a geographic boundary and satisfying a numeric attribute range?

A. A structured query with spatial and range predicates

B. A colour histogram comparison

C. Speaker diarization

D. Frame interpolation

Correct Answer: A. A structured query with spatial and range predicates

Question 2

Cosine similarity primarily measures what relationship between two non-zero embedding vectors?

A. Their physical storage size

B. The angle between their directions

C. The number of database fields

D. Their chronological order

Correct Answer: B. The angle between their directions

Question 3

What is a major reason to index video at shot or clip level rather than only at whole-video level?

A. To eliminate all transcription errors

B. To improve temporal localization of relevant content

C. To ensure every frame is unique

D. To remove the need for metadata

Correct Answer: B. To improve temporal localization of relevant content

Question 4

The haversine formula is generally used to estimate what quantity?

A. Audio frequency

B. Text-document length

C. Surface distance between latitude–longitude points

D. Image compression ratio

Correct Answer: C. Surface distance between latitude–longitude points

Question 5

Which approach translates the query into the language of the documents before retrieval?

A. Query translation

B. Document clustering

C. Late visual fusion

D. Speaker embedding

Correct Answer: A. Query translation

Question 6

Why are hard constraints often applied before multimodal ranking?

A. They guarantee that every result is semantically identical

B. They remove the need for an index

C. They enforce mandatory conditions that should not be outweighed by soft scores

D. They convert all modalities into images

Correct Answer: C. They enforce mandatory conditions that should not be outweighed by soft scores

Question 7

What is a defining characteristic of late fusion?

A. Representations are concatenated before encoding

B. Separate modality scores are combined after retrieval

C. Only structured fields are indexed

D. The system cannot handle missing modalities

Correct Answer: B. Separate modality scores are combined after retrieval

Question 8

What does nDCG accommodate more directly than binary precision?

A. Graded relevance judgments

B. Earth curvature

C. Audio sampling rate

D. Database normalization

Correct Answer: A. Graded relevance judgments

Question 9

In the disaster-response case, why is provenance important?

A. It makes all visual embeddings identical

B. It records the source and timing of evidence for verification

C. It automatically proves that a bridge is unsafe

D. It replaces geographic indexing

Correct Answer: B. It records the source and timing of evidence for verification

Question 10

Which governance measure is especially relevant when retrieval exposes precise locations of vulnerable people?

A. Removing all timestamps from public data

B. Access control and data minimization

C. Increasing embedding dimension without testing

D. Using only popularity ranking

Correct Answer: B. Access control and data minimization