Information Extraction, Search and Knowledge Systems

Introduction and Context

Text and social media analytics increasingly operates as an information-to-decision discipline. Customer comments, complaint records, call transcripts, reviews, emails, regulatory filings, and social posts contain references to people, products, locations, events, risks, intentions, and relationships. However, unstructured text is not directly suitable for reliable analysis. An organization must identify meaningful units, retrieve relevant evidence, connect facts, and present conclusions with appropriate uncertainty.

This chapter examines a complete information pipeline: information extraction, semantic search, knowledge graphs, summarization, and retrieval-augmented generation (RAG). These capabilities support customer intelligence, compliance monitoring, fraud investigation, service improvement, and enterprise decision support. The central design principle is that useful systems combine statistical language models with explicit data structures, domain rules, access controls, and human review.

Learning Objectives

By the end of this chapter, a learner should be able to:

1. Explain the roles of named-entity recognition, relation extraction, event extraction, coreference resolution, and summarization.
2. Design an extraction pipeline for noisy customer and social-media text.
3. Compare lexical, dense-vector, hybrid, and knowledge-graph retrieval.
4. Construct a knowledge graph with provenance, confidence, temporal validity, and entity resolution.
5. Explain the architecture and evaluation of retrieval-augmented systems.
6. Assess technical, managerial, ethical, privacy, and regulatory risks.
7. Select suitable metrics and governance controls for an enterprise deployment.

1. From Text to Structured Intelligence

Information extraction (IE) converts text into structured representations. A useful abstraction is:

document → mentions → entities, relations, events → indexed evidence → decisions

A mention is a span of text, such as “Acme Mobile” or “yesterday.” An entity is the real-world object to which one or more mentions refer. A relation connects entities, for example, customer–reported_fault–device. An event represents something occurring, such as a payment failure, product launch, or account closure. A fact should also record its source, time, extraction method, and confidence.

Text analytics is complicated by spelling variation, abbreviations, sarcasm, code-switching, emojis, domain terminology, negation, and incomplete grammar. “The battery is not overheating” contains the entity battery and a safety-related concept, but its meaning is the opposite of “the battery is overheating.” Consequently, extraction must be context-sensitive rather than based only on keyword matching.

1.1 The information-extraction pipeline

A production pipeline commonly includes:

1. Collection and consent: acquire authorized data and preserve source identifiers.
2. Normalization: detect language, standardize encoding, segment text, and handle URLs, emojis, and spelling variants.
3. Linguistic analysis: tokenize, tag parts of speech, parse dependencies, and identify sentence boundaries.
4. Extraction: detect entities, relations, events, sentiment, claims, and attributes.
5. Resolution: link mentions to canonical entities and resolve references such as “it” or “the previous model.”
6. Validation: apply business rules, confidence thresholds, and human review.
7. Storage and retrieval: write results to an index, graph, warehouse, or operational system.
8. Monitoring: measure quality, drift, latency, cost, and harmful errors.

The pipeline is usually iterative. A discovered alias may improve entity resolution; better resolution may improve relation extraction; improved extraction may reveal missing domain vocabulary.

2. Core Extraction Tasks

2.1 Named-entity recognition and linking

Named-entity recognition (NER) labels spans such as PERSON, ORGANIZATION, PRODUCT, LOCATION, DATE, MONEY, and REGULATION. Domain-specific schemas may add ACCOUNT, POLICY, DEVICE, CONTRACT, VULNERABILITY, or CLAIM. NER identifies what kind of thing a phrase denotes; entity linking maps it to a canonical record, such as product ID 4821.

A modern NER system may use a transformer encoder with a token-classification head, while a practical enterprise system often combines the model with dictionaries, regular expressions, and validation rules. For example, a policy number may be recognized reliably by a pattern, whereas a new product nickname may require contextual modeling.

Evaluation uses precision, recall, and F1. If TP denotes correct predictions, FP incorrect predictions, and FN missed instances, then:

These measures assume a defined annotation scheme and matching rule. Exact-span F1 is stricter than partial-span or relaxed matching. In compliance applications, recall may be prioritized for high-risk events, although excessive false positives can overwhelm investigators.

2.2 Relation extraction

Relation extraction identifies semantic links between entities. In “The regulator fined Delta Bank for misleading advertising,” a system may extract:

• subject: regulator
• relation: imposed_penalty_on
• object: Delta Bank
• reason: misleading advertising

Approaches include supervised classification over entity pairs, dependency-pattern rules, span-based neural models, and generative language models constrained by a schema. Relations should have direction, type, and scope. “Supplier acquired distributor” is not equivalent to “distributor acquired supplier.” Sentence-level extraction is insufficient when evidence is distributed across paragraphs; document-level models and coreference resolution may be required.

2.3 Event extraction

An event consists of a trigger, participants, attributes, and time. In “On 14 March, 230 customers reported duplicate charges,” reported is a possible trigger, the customers are participants, duplicate charges describe the event, and 14 March is its temporal attribute.

An event schema should specify required and optional roles. A service_outage event might contain service, start time, end time, affected region, severity, and evidence source. Schema design is a managerial decision: too little structure limits usefulness, whereas excessive structure increases annotation cost and inconsistency.

2.4 Negation, modality, and temporality

Extraction must distinguish asserted, negated, hypothetical, and alleged content. “The customer suspects fraud” is not equivalent to “fraud occurred.” Useful attributes include:

• Polarity: positive or negated.
• Modality: certain, possible, suspected, or recommended.
• Temporality: past, present, future, or time interval.
• Experiencer: who experienced or reported the condition.
• Source: who asserted the proposition.

These distinctions are vital in compliance and risk analytics. A social post quoting a news article should not automatically be treated as first-hand evidence.

2.5 Coreference and entity resolution

Coreference resolution links expressions referring to the same object: “The handset failed. It was returned.” Entity resolution links records or mentions across systems. A probabilistic matching model may combine name similarity, address, account history, and contextual evidence. The result should include a confidence score and a review policy, because incorrectly merging two customers can cause serious privacy and operational harm.

2.6 Summarization

Extractive summarization selects sentences or passages from a source. Abstractive summarization generates new wording. Extractive methods offer stronger traceability; abstractive methods are often more fluent but can introduce unsupported claims.

A decision-support summary should preserve the question, evidence, time period, uncertainty, and exceptions. Useful formats include an executive synthesis, a chronological incident summary, a claim-evidence table, and a customer-voice digest. Summaries should link each material assertion to source passages. Factuality, coverage, redundancy, readability, and usefulness should be evaluated separately; a fluent summary is not necessarily a faithful one.

3. Search and Retrieval Architecture

Search answers an information need by ranking candidate documents, passages, records, or graph facts. Traditional lexical retrieval uses exact or statistically weighted terms. A common term-frequency inverse-document-frequency weighting is:

Here, t is a term, d is a document, TF is term frequency, DF is the number of documents containing the term, and N is the total number of documents. Smoothing and length normalization are used in practical ranking functions such as BM25. Lexical retrieval performs well for exact product codes, legal phrases, and rare names, but may miss paraphrases.

Dense retrieval represents a query and document as vectors. Similarity is often measured using cosine similarity:

Here, q and d are embedding vectors, “·” is the dot product, and ||·|| is vector length. Cosine similarity is dimensionless and is appropriate when vector direction, rather than magnitude, represents semantic proximity. The embedding model and corpus domain determine whether semantically similar items are actually operationally relevant.

3.1 Lexical, dense, and hybrid retrieval

Hybrid search may combine normalized lexical and vector scores, or use one method to retrieve candidates and another to rerank them. Scores from different systems should not be added without calibration because they may have different scales.

3.2 Retrieval evaluation

Offline evaluation requires representative queries and relevance judgments. Precision at k is the proportion of the top k results that are relevant. Recall at k is the proportion of all relevant items retrieved in the top k. Mean reciprocal rank emphasizes the first relevant result:

Here, Q is the query set and rank_i is the rank of the first relevant result for query i. Ranking systems may also use nDCG when graded relevance is available. Online evaluation should include task completion, analyst time, reformulation rate, escalation quality, and harmful retrieval errors—not merely clicks.

4. Knowledge Graphs for Enterprise Intelligence

A knowledge graph represents entities and relationships as nodes and edges. A basic triple has the form:

(subject, predicate, object)

For example: (Customer_17, reported, BatteryOverheatingEvent_88). A production graph should extend this representation with provenance, extraction timestamp, source document, confidence, validity interval, and access classification. Property graphs and RDF/OWL-based graphs are alternative implementation families; the choice depends on interoperability, reasoning, query, and engineering requirements.

4.1 Ontology and schema design

An ontology defines classes, relations, constraints, and sometimes logical axioms. A customer-risk ontology might distinguish Customer, Account, Product, Complaint, Incident, Regulation, and Organization. It might constrain reported_by so that its object is a Customer, while regulated_by connects an Organization or Product to a Regulation.

Schema design should begin with competency questions:

• Which products are associated with recurring battery complaints?
• Which suppliers are linked to incidents in a regulated market?
• Which customers were affected during a specified outage?
• What evidence supports a compliance alert?

These questions prevent the graph from becoming an ungoverned collection of predicates.

4.2 Provenance, time, and uncertainty

A graph fact is not automatically truth. It is an assertion supported by evidence. Provenance allows an analyst to inspect the originating post, document, model version, and reviewer. Temporal validity distinguishes “is employed by” from “was employed by,” and event time from extraction time.

Confidence should be interpreted operationally, not as certainty. A threshold can route high-confidence facts to automation and low-confidence facts to review. Confidence values from different models should not be compared unless calibrated on suitable validation data.

4.3 Graph analytics

Graph queries can identify paths, neighborhoods, communities, and dependencies. For example, a compliance analyst can traverse from a regulation to controls, business processes, vendors, and incidents. Graph embeddings and link-prediction models can identify possible missing relationships, but predictions must remain hypotheses until supported by evidence.

5. Retrieval-Augmented Generation

RAG combines retrieval with language generation. At query time, the system retrieves relevant passages or graph facts, places them in a controlled context, and asks a language model to produce an answer. A typical architecture is:

Figure: A governed hybrid retrieval-augmented generation architecture.

RAG reduces reliance on a model’s static training memory and can provide citations, but it does not guarantee truth. Failure modes include poor chunking, missed evidence, irrelevant context, prompt injection in retrieved documents, stale indexes, and confident synthesis from contradictory sources. A robust system applies document-level permissions before retrieval, labels source authority, detects conflicts, and requires abstention when evidence is insufficient.

6. Worked Example: A Customer-Complaint Intelligence System

Assume an electronics company receives 100,000 monthly reviews, service tickets, and public posts. Managers want to detect product defects, identify affected regions, and provide evidence for quality investigations.

A complaint such as “My X12 overheated twice after the update; support replaced it last Friday” is processed as follows:

• Entities: Product = X12; Event = overheating; Organization = support.
• Events: overheating occurred twice; replacement occurred last Friday.
• Relations: customer reported overheating; support performed replacement.
• Attributes: frequency = 2; temporal reference = last Friday; polarity = asserted.
• Graph facts: (X12, associated_with, overheating_event) and (support, performed, replacement_event), each linked to the source.

The search layer supports the query “Find recurring overheating complaints after the software update.” Hybrid retrieval returns documents containing X12 and overheating, as well as semantically similar phrases such as “device became dangerously hot.” A graph query groups complaints by model, software version, region, and time. A RAG interface produces a cited summary and lists contradictory or low-confidence evidence separately.

Suppose an evaluation set contains 200 true overheating mentions. The extractor identifies 180 of them, of which 150 are correct. Then:

These figures apply to this test set and matching definition; they do not establish performance in every region or product line. The business may accept this model for triage but require human verification before a regulatory report.

Case outcome and learning points

In this illustrative case, the main value comes not from a single model but from connecting extraction, retrieval, graph structure, and workflow. Analysts can move from an aggregate trend to original evidence, inspect temporal patterns, and distinguish customer reports from confirmed engineering findings. Key lessons are to define the event schema early, retain provenance, measure errors by business consequence, and treat generated summaries as evidence-guided drafts rather than autonomous determinations.

7. Governance, Ethics, and Risk

Information systems built from social and customer data raise substantial governance issues.

Privacy and lawful use: Data collection should have a legitimate purpose, appropriate notice, retention limits, and access controls. Sensitive attributes should not be inferred or exposed without a justified basis. De-identification is not automatically irreversible, especially when graph connections enable re-identification.

Fairness: Dialect, language, disability-related expression, and cultural references can affect extraction accuracy. Audits should report performance across relevant language and demographic groups where lawful and ethical, while avoiding unnecessary collection of sensitive labels.

Security: Retrieved text can contain prompt-injection instructions or malicious content. Treat documents as untrusted data, separate instructions from evidence, restrict tools, and log model actions. Apply row-level and document-level authorization before indexing or retrieval.

Accountability: Every high-impact output should be traceable to sources, model versions, prompts or configurations, reviewers, and decision records. Automated alerts should support escalation rather than conceal uncertainty.

Regulatory compliance: Requirements vary by jurisdiction and sector, but common controls include purpose limitation, data minimization, explainability appropriate to risk, retention governance, auditability, and human oversight. Legal review is required for particular use cases.

8. Implementation and Best Practices

A mature implementation should:

1. Define business questions, risk tiers, entity schemas, and annotation guidelines before selecting a model.
2. Build a representative, versioned gold set with adjudicated labels and difficult examples.
3. Combine rules, statistical models, and human review according to error costs.
4. Use chunking that preserves headings, tables, sentence boundaries, and document identifiers.
5. Store source text, offsets, citations, timestamps, permissions, and model versions.
6. Calibrate thresholds and measure precision, recall, ranking quality, groundedness, latency, and cost.
7. Test multilingual, noisy, adversarial, stale, contradictory, and long-document inputs.
8. Establish feedback loops, drift detection, rollback procedures, and change control.
9. Separate discovery from decision automation; require approval for consequential actions.
10. Evaluate the complete workflow, including whether analysts make better and faster decisions.

Contemporary systems increasingly use multimodal extraction, domain-adapted language models, graph-enhanced retrieval, agentic search, and smaller task-specific models. These developments improve capability but also increase the need for evaluation, observability, and governance. A larger model cannot compensate for an incorrect ontology, missing permissions, weak evidence, or an undefined business decision.

Chapter Summary

Information extraction transforms text into entities, relations, events, claims, and attributes. Reliable extraction must handle negation, temporality, modality, coreference, entity resolution, and provenance. Lexical retrieval is strong for exact terms, dense retrieval captures semantic similarity, and hybrid retrieval often provides the most useful enterprise search. Knowledge graphs organize entities and relationships while preserving source, time, confidence, and access context. Retrieval-augmented generation combines search with language generation, but requires authorization, citations, conflict handling, and abstention controls. Successful deployments integrate models with schemas, evaluation data, human workflows, security, privacy, and continuous monitoring. The strategic objective is not merely to generate text, but to produce traceable evidence that improves customer intelligence, compliance, and decisions.

Key Terms

Information extraction: Conversion of unstructured text into structured facts.

Named-entity recognition: Identification and classification of entity mentions in text.

Entity linking: Mapping a mention to a canonical real-world entity.

Relation extraction: Identification of semantic links between entities.

Event extraction: Identification of events, triggers, participants, attributes, and time.

Coreference resolution: Linking expressions that refer to the same entity or event.

Negation detection: Identification of statements that deny or reverse a proposition.

Provenance: Information about the source and processing history of a fact.

Ontology: Formal representation of concepts, relations, and constraints in a domain.

Knowledge graph: A graph of entities and typed relationships supported by evidence.

Lexical retrieval: Search based primarily on terms and token-level matching.

Dense retrieval: Search using vector representations of queries and documents.

Hybrid retrieval: Combination of lexical and dense retrieval signals.

Cosine similarity: A measure of angular similarity between vectors.

Reranking: Reordering retrieved candidates using a more detailed relevance model.

Retrieval-augmented generation: Generation conditioned on retrieved external evidence.

Groundedness: Degree to which an answer is supported by retrieved evidence.

Precision: Proportion of retrieved or predicted items that are correct.

Recall: Proportion of relevant or true items that are retrieved or identified.

Entity resolution: Determination of whether records or mentions refer to the same entity.

Review Questions

Short Answer Questions

1. What is the difference between named-entity recognition and entity linking?
2. Why are negation and temporality important in compliance-oriented extraction?
3. State one strength and one limitation of dense retrieval.
4. What information should provenance attach to a knowledge-graph fact?
5. Why can retrieval-augmented generation produce an unsupported answer?

Long Answer Questions

1. Explain an end-to-end information-extraction pipeline for customer complaints, including normalization, extraction, validation, storage, and monitoring.
2. Compare lexical, dense, hybrid, and graph retrieval for enterprise search. Recommend a design for regulatory investigations and justify it.
3. Discuss ontology design using competency questions, event schemas, temporal validity, and entity resolution.
4. Explain precision, recall, and F1, including situations in which recall should be prioritized over precision.
5. Analyze the governance requirements for a RAG system used in customer-service or compliance decision support.

Analytical / Application Questions

1. An extractor finds 240 correct mentions, produces 60 false positives, and misses 40 true mentions. Calculate precision, recall, and F1, and recommend whether it should be used for automated regulatory reporting.
2. Design a knowledge-graph schema for investigating product-safety complaints. Specify at least six entity or event types, six relations, and the provenance fields required for each assertion.
3. A RAG system retrieves fluent but contradictory documents from different dates. Propose a retrieval, ranking, prompt, and human-review process that preserves temporal validity and prevents unsupported conclusions.

Multiple Choice Questions with Answers

Question 1

Which task maps a text mention to a canonical real-world record?

A. Named-entity recognition

B. Entity linking

C. Sentence segmentation

D. Extractive summarization

Correct Answer: B. Entity linking

Question 2

A system identifies 80 true positives and 20 false positives. What is its precision?

A. 60%

B. 75%

C. 80%

D. 90%

Correct Answer: C. 80%

Question 3

Which retrieval method is generally strongest for exact product codes and statutory phrases?

A. Lexical retrieval

B. Dense retrieval only

C. Random retrieval

D. Abstractive summarization

Correct Answer: A. Lexical retrieval

Question 4

What is the primary purpose of provenance in a knowledge graph?

A. To increase document length

B. To record the source and processing history of a fact

C. To remove all uncertainty

D. To replace entity resolution

Correct Answer: B. To record the source and processing history of a fact

Question 5

In the triple (Customer_17, reported, Incident_88), “reported” is the:

A. Subject

B. Object

C. Predicate

D. Document identifier

Correct Answer: C. Predicate

Question 6

Which issue is especially important when extracting safety complaints?

A. Ignoring negation

B. Removing all dates

C. Avoiding source citations

D. Treating every allegation as confirmed fact

Correct Answer: A. Ignoring negation

Question 7

What does cosine similarity compare?

A. The number of documents in a corpus

B. The angular relationship between vectors

C. The number of named entities in a sentence

D. The time required to train a model

Correct Answer: B. The angular relationship between vectors

Question 8

Which component most directly helps a RAG answer remain auditable?

A. Source citations linked to retrieved evidence

B. Removing document identifiers

C. Increasing temperature without evaluation

D. Discarding low-confidence passages silently

Correct Answer: A. Source citations linked to retrieved evidence

Question 9

If a system has recall of 75% on a defined test set, what does that mean?

A. 75% of its predictions are correct

B. It retrieved or identified 75% of the relevant true items

C. It produced answers 75% faster

D. 75% of documents were duplicated

Correct Answer: B. It retrieved or identified 75% of the relevant true items

Question 10

Which control best addresses prompt injection in retrieved documents?

A. Treating retrieved text as trusted instructions

B. Removing all access controls

C. Separating evidence from instructions and restricting tools

D. Automatically publishing every generated answer

Correct Answer: C. Separating evidence from instructions and restricting tools