11. Text, Web and Multimodal Mining

Introduction and Context

A large proportion of organisational knowledge is not stored in relational tables. It appears in reports, emails, contracts, customer reviews, news articles, web pages, social-media posts, images, audio recordings, videos, and interaction logs. These sources are rich in meaning but difficult to analyse because they are heterogeneous, noisy, unstructured, multilingual, rapidly changing, and often dependent on context. Text, web, and multimodal mining provide methods for converting such content into representations that support discovery, prediction, monitoring, and decision-making.

Text mining focuses on extracting patterns and knowledge from natural-language documents. Web mining extends these ideas to the content, structure, and usage of the World Wide Web. Multimodal mining integrates evidence from two or more modalities, such as text and images or speech and video. The central challenge is not merely to collect more data; it is to preserve meaning while transforming data into computationally usable forms.

This chapter connects document preprocessing and vector-space modelling with topic modelling, sentiment analysis, information extraction, embeddings, web-usage mining, language models, and multimodal analytics. It also considers evaluation, governance, privacy, bias, and operational risks. These issues are especially important at postgraduate level because effective mining requires both technical design and critical judgement about whether a discovered pattern is valid and useful.

Learning Objectives

After studying this chapter, the learner should be able to:

1. Explain the nature and challenges of text, web, and multimodal data.

2. Design a document-preprocessing pipeline for a specific analytical task.

3. Compare Boolean, TF–IDF, probabilistic, neural, and embedding-based representations.

4. Apply topic modelling, sentiment analysis, and information-extraction methods appropriately.

5. Describe web-content, web-structure, and web-usage mining workflows.

6. Explain how language models and multimodal models support contemporary analytics.

7. Evaluate models using task-appropriate metrics and investigate sources of error.

8. Address privacy, fairness, copyright, security, and governance concerns.

9. Select practical approaches for organisational and industry applications.

The Nature of Text, Web, and Multimodal Data

Text data may be classified as structured text, semi-structured text, or free text. A database field containing a product description is structured in location but unstructured in meaning. XML and HTML documents are semi-structured because tags indicate relationships, although the content within them may be textual. Customer narratives, legal documents, and discussion posts are largely free text. Text also contains linguistic ambiguity: the word “bank” may refer to a financial institution or a river bank, while “excellent” may be sincere, ironic, or context-dependent.

Web data has three complementary dimensions. Web-content mining analyses the information contained in pages, documents, images, and metadata. Web-structure mining analyses links and other relationships among pages, domains, or entities. Web-usage mining studies interaction traces such as clicks, queries, sessions, navigation paths, downloads, and dwell time. These dimensions can be combined: a search engine may use page content, hyperlink authority, and observed user behaviour to improve ranking.

Multimodal data combines modalities that describe the same object, event, or user from different perspectives. A product page may contain a title, description, photograph, customer review, and demonstration video. A medical case may include a clinical note and an image. A video contains visual frames, speech, subtitles, acoustic signals, and temporal events. Modalities may be complementary, redundant, or contradictory. Consequently, integration is not simply a matter of concatenating features.

Important terminology includes corpus, document, token, vocabulary, metadata, annotation, entity, relation, embedding, modality, session, clickstream, ontology, and provenance. A corpus is a collection of documents used for analysis; a token is a basic processing unit such as a word or subword; an embedding is a dense numerical representation intended to capture semantic or contextual relationships; and provenance records the origin and transformation history of data.

From Documents to Representations

Preprocessing as an analytical decision

Preprocessing is often described as cleaning, but it is better understood as the design of an information-preserving transformation. A pipeline may include document acquisition, format conversion, language identification, removal of boilerplate, sentence segmentation, tokenisation, normalisation, stop-word handling, stemming or lemmatisation, and extraction of metadata. Each operation should be justified by the intended task. Removing negation words may be acceptable for some information-retrieval tasks but harmful for sentiment analysis. Stemming can reduce sparsity but may merge words with different meanings.

A typical document pipeline is:

1. Acquire and validate data. Check file integrity, encoding, duplicates, missing fields, and source reliability.

2. Segment content. Separate title, body, headings, captions, comments, and navigation elements.

3. Normalise. Standardise case, Unicode forms, dates, measurements, spelling variants, and selected abbreviations.

4. Tokenise. Choose word, character, subword, or domain-specific tokenisation.

5. Handle linguistic variation. Use lemmatisation, stemming, phrase detection, or morphological analysis when justified.

6. Represent and index. Convert documents into sparse or dense vectors and retain document identifiers and provenance.

7. Validate. Inspect samples and compare token statistics before and after transformation.

Web pages require additional processing. Boilerplate removal distinguishes meaningful content from menus, advertisements, cookie notices, and repeated templates. Crawling must respect access controls, robots directives, rate limits, and terms of service. Dynamic pages may require browser automation, but such collection increases cost and introduces additional security concerns. Deduplication is essential because syndicated content and URL variants can produce misleading frequency estimates.

Sparse vector representations

The Boolean model represents a document by the presence or absence of terms. It is simple and useful for exact filtering, but it cannot express term frequency or semantic similarity. In the vector-space model, document and query vectors are compared using a similarity function, commonly cosine similarity:

cosine(x,y) = (x · y) / (||x|| ||y||).

Term frequency–inverse document frequency (TF–IDF) gives greater weight to terms frequent in a document but uncommon across the collection. A common formulation is:

TF–IDF(t,d) = TF(t,d) × log(N / DF(t)),

where TF(t,d) is the frequency of term t in document d, DF(t) is the number of documents containing t, and N is the number of documents. Weighting variants, logarithmic term frequency, and document-length normalisation can improve retrieval. However, TF–IDF treats words as largely independent features and may fail when synonyms use different vocabulary.

N-grams represent sequences of adjacent tokens, such as “credit risk” or “machine learning.” They capture local phrases and can improve classification, but their dimensionality grows rapidly. Character n-grams are useful for misspellings, morphologically rich languages, and noisy user-generated text.

Dense and contextual representations

Word embeddings map tokens to dense vectors in which geometric relationships reflect patterns of usage. Distributional methods such as Word2Vec and GloVe learn representations from co-occurrence information. They are compact and useful for similarity, clustering, and feature construction, but a single vector for a word cannot fully distinguish different senses.

Contextual language models generate representations dependent on surrounding words. Thus, “charge” can receive different representations in “charge the battery” and “charge the defendant.” Transformer architectures use attention to model relationships among tokens and support pretrained models that can be adapted through fine-tuning, prompting, or parameter-efficient methods. Sentence and document embeddings support semantic search, clustering, recommendation, and retrieval-augmented generation.

Embedding similarity is not equivalent to truth, causation, or relevance to a particular business objective. Vectors may encode social stereotypes, source-specific language, or accidental correlations. Domain adaptation, careful evaluation, and human review remain necessary.

Discovering Themes and Opinions

Topic modelling

Topic modelling is an unsupervised approach for discovering recurring semantic themes in a collection. In Latent Dirichlet Allocation (LDA), each document is treated as a mixture of topics, and each topic is represented as a probability distribution over words. The model assumes that documents are generated by selecting topic proportions and then selecting words associated with those topics. Topic proportions help compare documents, while high-probability words help interpret themes.

LDA is useful for exploratory analysis of large collections, such as policy reports or research abstracts. Its assumptions are not always suitable for short texts, highly contextual language, or collections with evolving terminology. The number of topics must be selected or explored, and topics require human interpretation. Coherence measures can assist comparison, but a mathematically coherent topic is not necessarily a meaningful organisational category. Alternative approaches include non-negative matrix factorisation, probabilistic clustering, dynamic topic models, and embedding-based topic discovery.

A sound workflow begins with a clearly defined corpus and preprocessing policy, explores several topic counts, examines representative documents rather than only top words, labels topics cautiously, and tests stability across samples or time periods. Topic models are generally descriptive; they should not be interpreted as proof that topics cause outcomes.

Sentiment and emotion analysis

Sentiment analysis estimates evaluative orientation, often as positive, negative, or neutral, while emotion analysis may identify categories such as anger, fear, joy, or sadness. Lexicon-based methods use dictionaries of sentiment-bearing words and rules for negation or intensification. Supervised classifiers learn from labelled examples. Transformer-based classifiers can model context and longer dependencies, but they require representative training data and careful calibration.

Sentiment is difficult because of sarcasm, ambiguity, domain-specific language, mixed opinions, implicit targets, and changing usage. “This update is sick” may be positive in one community and negative in another. Aspect-based sentiment analysis separates the target aspect from its evaluation: a review might praise a camera but criticise battery life. This distinction is more actionable than a single document-level score.

Evaluation should include precision, recall, F1-score, confusion matrices, and, where probabilities are used, calibration. A model trained on product reviews may perform poorly on political speech or clinical notes. Performance should therefore be reported across languages, demographic groups, time periods, and relevant content types.

Information extraction

Information extraction converts text into structured facts. Named-entity recognition identifies people, organisations, locations, dates, products, diseases, or other domain entities. Relation extraction identifies connections such as “company acquired company” or “drug treats condition.” Event extraction identifies an event, its participants, time, location, and attributes. Coreference resolution links expressions such as “the company” and “it” to the same entity.

Extraction may use rules, dictionaries, conditional models, neural sequence labelling, relation classifiers, or large language models. A practical architecture often combines methods: deterministic rules handle dates and identifiers, while learned models handle linguistic variation. Extracted results should retain confidence scores, source spans, document identifiers, and version information. Without provenance, analysts cannot audit or correct the resulting knowledge graph.

Web Mining in Practice

Web-content mining begins with source selection, crawling or API access, parsing, cleaning, indexing, and analysis. Search and recommendation systems commonly combine lexical retrieval with dense retrieval. A two-stage design first retrieves candidates efficiently and then applies a more expensive ranking model. Quality depends on freshness, duplicate control, spam detection, query interpretation, and evaluation against user intent.

Web-structure mining represents pages or entities as a graph. Link analysis methods such as PageRank estimate importance from the pattern of incoming links, although authority is not identical to factual reliability. Community detection can reveal clusters, while centrality measures identify influential or strategically positioned nodes. Graph analysis can support citation analysis, fraud investigation, supply-chain mapping, and discovery of related websites.

Web-usage mining treats interaction records as behavioural evidence. Logs commonly include user or session identifiers, timestamps, URLs, referrers, query terms, device information, and outcomes. A standard process is data cleaning, bot filtering, user identification, sessionisation, path construction, feature generation, modelling, and evaluation. Sessionisation may use authentication records or inactivity thresholds, but shared devices and privacy-preserving identifiers make reconstruction uncertain.

Usage mining supports personalisation, conversion analysis, search improvement, anomaly detection, capacity planning, and user-experience research. Metrics such as click-through rate, abandonment rate, dwell time, conversion rate, and path frequency must be interpreted carefully. A high click-through rate may reflect an attractive title, but not satisfaction; a long dwell time may indicate engagement or confusion. Product and service decisions should combine behavioural metrics with qualitative evidence.

Multimodal Mining and Fusion Strategies

Multimodal analytics requires alignment. A caption may describe an image, a transcript may correspond to a video segment, and a sensor stream may record the same event as a text report. Three broad fusion strategies are common:

• Early fusion: features from several modalities are combined before modelling. This can capture cross-modal interactions but requires compatible timing, scale, and missing-data treatment.
• Late fusion: each modality is modelled separately and predictions are combined. It is modular and robust when modalities are sometimes absent, but may miss fine-grained interactions.
• Intermediate or joint fusion: modality-specific encoders feed a shared representation, often through attention or cross-modal transformers. This can model relationships effectively but demands substantial data and computational resources.

For image–text retrieval, a system may embed images and captions in a shared space. For video understanding, visual frames, speech transcripts, and acoustic features can be aligned over time. In document intelligence, optical character recognition converts scanned pages into text while layout models preserve tables, headings, and spatial relationships. In every case, missing or low-quality modalities should be handled explicitly rather than silently treated as evidence of absence.

Language Models and Contemporary Analytics

Large language models extend text-mining capabilities through contextual representation, generation, summarisation, question answering, classification, extraction, and conversational interfaces. They can support analysts by proposing labels, extracting candidate entities, generating search queries, and summarising long collections. Retrieval-augmented generation combines a language model with an external index so that answers can be grounded in retrieved documents. This can improve traceability, provided the retrieval set is relevant and citations are verified.

Language models also introduce risks. They may hallucinate facts, reproduce biases, expose memorised information, follow malicious instructions embedded in documents, or produce inconsistent outputs. Prompting is not a substitute for evaluation. Production systems need input filtering, access control, retrieval validation, output checks, logging, model-version management, and escalation to human experts for high-impact decisions.

Multimodal foundation models can process combinations of text, images, audio, and video. Applications include visual question answering, document understanding, accessibility tools, industrial inspection, media indexing, and educational content analysis. Their performance can degrade on uncommon objects, poor image quality, culturally specific content, or adversarial inputs. Organisations should distinguish demonstration value from validated operational performance.

An Illustrative Case: Mining Customer Experience Across Channels

Background

Consider an illustrative case involving a national telecommunications provider. The organisation receives call-centre transcripts, online reviews, support emails, network-outage posts, mobile-app logs, and screenshots uploaded by customers. Management wants to identify recurring service problems and prioritise improvements.

Situation and problem

Existing reporting counts complaints by category, but categories are manually assigned and updated slowly. A single incident may appear as a call, a social-media post, and an app interaction. Executives need to know which issues are increasing, which customer segments are affected, and whether operational changes improve sentiment.

Analysis and approach

The analytics team first establishes governance rules for consent, retention, redaction, access, and deletion. Personally identifying information is masked before modelling. Text is language-identified, normalised, and deduplicated. Call recordings are transcribed, with confidence scores retained. Images are processed for optical text recognition when customers upload screenshots, while irrelevant visual content is discarded.

A hybrid extraction pipeline identifies products, locations, outage events, and dates. Aspect-based sentiment assigns sentiment to network reliability, billing, customer service, and application usability. Embeddings support semantic clustering so that differently worded complaints about intermittent connectivity can be grouped. A topic model provides exploratory evidence about emerging themes. App clickstream data is sessionised and joined only through privacy-approved pseudonymous identifiers.

The team builds a dashboard showing issue volume, sentiment by aspect, geographic distribution, resolution time, and changes before and after interventions. Retrieval-augmented summaries provide links to representative source records rather than unsupported generated explanations. Analysts sample false positives and false negatives each month, and operations staff validate whether extracted issues correspond to actual incidents.

Outcome

The organisation identifies that a large group of “billing complaints” actually concerns confusing notifications following a plan change. A redesigned message reduces repeat contacts, while a separate network issue is escalated to engineering. The dashboard also reveals that negative sentiment is concentrated in a language community underrepresented in the original labelled data; additional annotation improves performance for that group.

Learning points

The case demonstrates that useful mining depends on integration rather than a single algorithm. Preprocessing, entity resolution, aspect-level analysis, multimodal alignment, privacy controls, and human validation are all necessary. It also shows why correlation in usage data should not automatically be interpreted as customer dissatisfaction or as evidence that a particular intervention caused improvement.

Evaluation, Risks, and Governance

Evaluation must correspond to the intended decision. Information retrieval uses precision, recall, mean average precision, or ranking-based measures. Classification uses precision, recall, F1-score, ROC-AUC where appropriate, and calibration. Extraction requires entity- and relation-level matching, with partial-match policies stated clearly. Topic models require coherence, stability, interpretability, and usefulness to stakeholders. Clustering and embeddings should be assessed through both intrinsic measures and downstream performance.

Common mistakes include deleting meaningful domain terms as stop words, training on duplicated documents, leaking future information into features, using random splits for time-dependent data, treating noisy labels as ground truth, and reporting only average accuracy. Analysts should inspect errors, establish baselines, test robustness to paraphrase and missing modalities, and document data and model versions.

Text and web mining may process personal opinions, browsing histories, health information, political expression, or workplace communications. Lawful collection does not automatically make every use ethical. Purpose limitation, data minimisation, informed notice, retention controls, access restriction, and secure deletion are central safeguards. Sensitive attributes may be inferred from seemingly harmless text or images; such inference should be strictly governed.

Fairness requires examining differential error rates and representational gaps. Translation systems, sentiment classifiers, and speech recognisers can perform unevenly across languages and accents. Copyright, licensing, terms of service, and database rights may limit collection and reuse. Security threats include prompt injection, poisoned training data, malicious web content, re-identification, and model extraction. High-impact uses should include human oversight, appeal mechanisms, audit trails, and documented accountability.

Emerging Directions and Best Practices

Current development is moving toward smaller specialised language models, multimodal foundation models, retrieval-based systems, synthetic data, agentic workflows, graph-enhanced retrieval, and privacy-preserving learning. These developments may lower cost or improve domain performance, but they do not eliminate the need for data quality and governance. Continual learning can address changing language and topics, yet it may also cause model drift and make historical comparisons difficult.

A professional implementation should:

1. Define the decision, user, harm conditions, and success measure before choosing a model.

2. Maintain source provenance, consent or lawful-use records, and transformation logs.

3. Begin with transparent baselines such as TF–IDF, rules, or logistic regression.

4. Use representative annotation guidelines and measure inter-annotator agreement.

5. Compare modalities separately before evaluating fusion.

6. Monitor drift, calibration, subgroup performance, latency, cost, and security.

7. Preserve evidence spans, retrieved sources, and uncertainty for auditability.

8. Keep human review in workflows where errors can cause material harm.

9. Reassess models when vocabulary, user behaviour, policy, or data sources change.

Chapter Summary

Text, web, and multimodal mining transform heterogeneous content and behaviour into representations for discovery and decision-making. Document preprocessing must be task-sensitive because operations such as stop-word removal, stemming, and boilerplate filtering can alter meaning. Sparse representations such as TF–IDF remain useful and interpretable, while embeddings and contextual language models capture richer semantic relationships. Topic modelling supports exploratory theme discovery; sentiment analysis estimates opinions and emotions; information extraction produces entities, relations, and events.

Web mining includes content, structure, and usage mining. Multimodal systems combine evidence through early, late, or joint fusion, with alignment and missing-data treatment as central concerns. Language models and retrieval-augmented systems enable powerful contemporary applications but require grounding, evaluation, security, and governance. Successful professional practice combines algorithms with provenance, privacy protection, fairness analysis, human validation, and continuous monitoring.

Key Terms

Corpus: A collection of documents or multimodal records analysed as a unit.

Tokenisation: The process of dividing text into tokens such as words, subwords, or characters.

Lemmatisation: Reduction of a word to its dictionary base form using linguistic analysis.

TF–IDF: A weighting scheme that emphasises terms frequent in a document but uncommon in the corpus.

Embedding: A dense numerical representation intended to encode similarity or meaning.

Contextual representation: A representation whose value changes according to surrounding content.

Topic modelling: Unsupervised discovery of recurring themes represented through distributions over words or embeddings.

Sentiment analysis: Computational estimation of evaluative orientation in text.

Aspect-based sentiment analysis: Identification of sentiment associated with specific targets or aspects.

Named-entity recognition: Detection and classification of entities such as people, organisations, and locations.

Relation extraction: Identification of semantic relationships between entities.

Web-content mining: Analysis of information contained in web pages and digital documents.

Web-structure mining: Analysis of links and relationships among web entities.

Web-usage mining: Analysis of user interaction and navigation records.

Sessionisation: Grouping interaction events into an estimated user session.

Early fusion: Combining features from multiple modalities before modelling.

Late fusion: Combining outputs or predictions from separate modality-specific models.

Retrieval-augmented generation: Generation grounded in documents retrieved from an external information source.

Provenance: A record of data origin, transformations, and processing history.

Model drift: Decline in model validity caused by changes in data, language, behaviour, or relationships.

Review Questions

Short Answer Questions

1. Why can preprocessing choices affect the validity of a text-mining result?

2. Distinguish web-content mining from web-usage mining.

3. What is the purpose of inverse document frequency in TF–IDF?

4. Why is aspect-based sentiment analysis more actionable than document-level sentiment in many applications?

5. What information should be retained with an extracted entity for auditability?

Long Answer Questions

1. Explain a complete document-preprocessing workflow and discuss how its design changes for information retrieval, sentiment analysis, and information extraction.

2. Compare Boolean, TF–IDF, static embedding, and contextual language-model representations.

3. Explain LDA topic modelling, including its assumptions, interpretation challenges, and suitable evaluation practices.

4. Describe the stages of a web-usage-mining project from raw logs to organisational decision-making.

5. Discuss multimodal fusion strategies and the technical and governance issues involved in deploying multimodal foundation models.

Analytical / Application Questions

1. A bank receives short multilingual customer messages containing code-switching, spelling errors, and sarcasm. Design a sentiment-analysis pipeline and justify preprocessing, representation, evaluation, and human-review choices.

2. An online retailer observes a rise in page clicks but a fall in completed purchases. Explain how content, structure, and usage mining could be combined to investigate the change without overinterpreting any single metric.

3. A hospital proposes using a language model to summarise clinical notes and images for physicians. Analyse the potential benefits, failure modes, privacy requirements, evaluation design, and safeguards needed before deployment.

22. Multiple Choice Questions with Answers

Question 1

Which preprocessing decision is most likely to harm sentiment analysis if applied without modification?

A. Removing duplicate documents

B. Removing all negation words as stop words

C. Detecting the language of each document

D. Recording document provenance

Correct Answer: B. Removing all negation words as stop words

Question 2

What does the inverse document frequency component of TF–IDF primarily accomplish?

A. It increases the weight of terms appearing in every document

B. It gives greater weight to terms rare across the collection

C. It converts sparse vectors into image features

D. It identifies hyperlinks between web pages

Correct Answer: B. It gives greater weight to terms rare across the collection

Question 3

A topic model produces a mathematically coherent topic whose top words are difficult for analysts to interpret. What is the most appropriate response?

A. Assume the topic is automatically valid

B. Delete every document containing the topic

C. Inspect representative documents and assess practical interpretability

D. Replace all words with Boolean values

Correct Answer: C. Inspect representative documents and assess practical interpretability

Question 4

Which task is an example of information extraction rather than sentiment analysis?

A. Estimating whether a review is positive

B. Identifying that a company acquired another company

C. Measuring the emotional intensity of a message

D. Ranking documents by cosine similarity

Correct Answer: B. Identifying that a company acquired another company

Question 5

A system analyses hyperlinks among websites to identify influential pages. This is primarily an example of:

A. Web-content mining

B. Web-structure mining

C. Web-usage mining

D. Optical character recognition

Correct Answer: B. Web-structure mining

Question 6

Which statement best describes late fusion in multimodal mining?

A. All raw modality features are concatenated before training

B. Each modality is modelled separately and their outputs are combined

C. Only the most complete modality is retained

D. Modalities are converted into stop-word lists

Correct Answer: B. Each modality is modelled separately and their outputs are combined

Question 7

Why can a high dwell time on a web page be misleading as a measure of satisfaction?

A. Dwell time cannot be recorded digitally

B. Long duration may indicate confusion rather than engagement

C. Dwell time is identical to conversion rate

D. It is independent of user behaviour

Correct Answer: B. Long duration may indicate confusion rather than engagement

Question 8

Which practice most directly helps ground a language-model answer in organisational evidence?

A. Increasing temperature without evaluation

B. Using retrieval-augmented generation with verified source documents

C. Removing document identifiers from the index

D. Allowing unrestricted access to all web pages

Correct Answer: B. Using retrieval-augmented generation with verified source documents

Question 9

In the illustrative telecommunications case, why were sentiment and topic analysis combined with aspect extraction?

A. To eliminate the need for data governance

B. To associate opinions with specific service problems

C. To ensure every complaint became positive

D. To replace all operational records

Correct Answer: B. To associate opinions with specific service problems

Question 10

Which evaluation practice is especially important when a model may perform differently across languages or demographic groups?

A. Reporting only overall accuracy

B. Testing subgroup performance and error rates

C. Discarding all minority-language records

D. Using only training-set results

Correct Answer: B. Testing subgroup performance and error rates