Text Analytics, AI and Generative AI

Introduction and Context

Business information is increasingly expressed in language: customer reviews, call-centre transcripts, emails, contracts, social-media posts, survey responses, incident reports and internal knowledge bases. Traditional analytics systems were designed mainly for structured data stored in rows and columns. Text analytics extends analytical capability to unstructured and semi-structured language data by converting text into representations that can be searched, classified, compared and summarized.

Natural-language processing (NLP) provides the computational methods for working with human language. Sentiment analysis identifies evaluative attitudes, such as satisfaction or dissatisfaction. Embeddings represent language as numerical vectors that preserve aspects of semantic similarity. Large language models (LLMs) generate and transform text using patterns learned from very large datasets. Generative AI systems built on LLMs can answer questions, draft documents, extract information and support decision-making, but they also introduce risks including hallucination, bias, confidentiality loss and excessive automation.

For business analysts, the central issue is not whether a model is impressive in a demonstration. It is whether the system produces sufficiently accurate, explainable, secure and economically valuable outputs for a defined business process. A suitable enterprise design therefore combines data preparation, model selection, evaluation, retrieval, human oversight and governance.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the main stages of a text-analytics pipeline.
2. Distinguish tokenization, normalization, classification, sentiment analysis, information extraction and topic discovery.
3. Describe embeddings and similarity search, including their business uses and limitations.
4. Evaluate prompt design and select appropriate prompting strategies.
5. Explain retrieval-augmented generation (RAG) and its role in grounding model responses.
6. Assess hallucination, bias, privacy, security and other limitations of generative AI.
7. Design a human-in-the-loop workflow for an enterprise use case.
8. Recommend suitable applications and evaluation measures for business settings.

1. The Text-Analytics Pipeline

A text-analytics project normally consists of the following stages:

1. Business formulation: Define the decision to be improved, the users, the expected benefit and the consequences of error.
2. Data acquisition: Collect relevant documents, messages, transcripts or records, while checking consent, licensing and privacy requirements.
3. Preparation: Remove duplicates, detect language, correct encoding problems, segment documents and preserve useful metadata.
4. Representation: Convert text into features such as tokens, counts, weighted terms or embeddings.
5. Analysis or generation: Apply a classifier, extractor, search system, clustering method or language model.
6. Evaluation: Compare outputs with a labelled or expert-reviewed reference set and assess business impact.
7. Deployment and monitoring: Integrate the system into a workflow and monitor drift, errors, security and user behaviour.

The stages are iterative rather than strictly linear. For example, error analysis may reveal that the original labels were inconsistent, that a document type was missing, or that the business question was too vague.

Important language-processing tasks

Language is context dependent. The statement “That was sick” may express approval in one community and criticism in another. “The product is cheap” may be positive when discussing price but negative when discussing quality. Therefore, a model should be evaluated on the language, domain and decision context in which it will operate.

2. Representing Text for Analysis

Tokenization and normalization

Tokenization divides text into units such as words, subwords or characters. Modern language models often use subword tokenization, which handles unfamiliar words by combining smaller pieces. Normalization may include lowercasing, punctuation handling, spelling correction and removal of irrelevant markup. These operations must be applied carefully: deleting punctuation can change meaning, and lowercasing may remove information from product codes or named entities.

Stop-word removal and stemming were common in earlier information-retrieval systems. Stop words are frequent terms such as “the” or “and”; stemming reduces words to approximate roots. Such methods can reduce dimensionality, but they may remove distinctions useful for sentiment, negation or legal interpretation. A contemporary pipeline should justify each preprocessing decision rather than applying a fixed recipe.

Term frequency and TF–IDF

A simple document representation counts terms. Term frequency–inverse document frequency (TF–IDF) gives greater weight to terms that are frequent in a document but relatively uncommon across the collection. One common formulation is:

Here, t is a term, d is a document, TF(t,d) is the term frequency in document d, N is the number of documents, and DF(t) is the number of documents containing t. The logarithm reduces the effect of very common terms. If a term occurs in every document, its inverse-document component is zero under this formulation. Variants use normalized term frequency and smoothing, such as log((N+1)/(DF(t)+1)). TF–IDF is useful for interpretable search and baseline classification, but it does not understand synonyms or word order.

Embeddings

An embedding maps a text unit to a vector in a numerical space. Similar meanings may be placed near one another, although the quality of that relationship depends on the training data, model and context. A common similarity measure is cosine similarity:

In this expression, x and y are embedding vectors, x · y is their dot product, and ||x|| and ||y|| are their Euclidean lengths. The result is defined for non-zero vectors and lies between −1 and 1. In many semantic-search systems, larger cosine similarity indicates greater directional similarity, but it is not a probability and should not automatically be interpreted as factual similarity.

Embeddings support:

• semantic search for relevant documents;
• clustering of customer comments;
• duplicate or near-duplicate detection;
• recommendation and content matching;
• retrieval of examples for a language model; and
• feature construction for downstream predictive models.

Embeddings can encode unwanted associations from their training data. They may also perform poorly on specialized vocabulary, low-resource languages, short ambiguous queries or highly structured information. A similarity score should therefore be validated against human judgements and business outcomes.

3. Sentiment and Opinion Analysis

Sentiment analysis estimates the emotional or evaluative orientation of text. A basic system may assign positive, negative or neutral labels. More advanced systems estimate emotion categories, aspect-level sentiment or the target of an opinion.

Document-level sentiment assigns one label to a whole review. Aspect-based sentiment analysis identifies an aspect and then evaluates it. For example, “The battery lasts all day, but the screen is dim” contains positive sentiment about battery life and negative sentiment about the screen. Aspect-level analysis is more useful for product improvement because an overall score can conceal opposing experiences.

Approaches include:

• Lexicon-based methods: Use dictionaries of positive and negative terms. They are transparent but often fail with negation, sarcasm and domain-specific meanings.
• Supervised machine learning: Learn from labelled examples. Performance depends on label quality and representativeness.
• Transformer-based models: Use contextual representations and can capture more complex language patterns, but require careful validation and may be difficult to explain.
• LLM-based classification: Ask a generative model to classify or explain sentiment. This can be flexible but may be less stable and more costly than a specialized classifier.

Evaluation should use a held-out test set and measures appropriate to the task. For binary classification, precision and recall are:

Here, TP is the number of true positives, FP false positives and FN false negatives. Precision answers, “Of the items predicted positive, how many were positive?” Recall answers, “Of the truly positive items, how many were found?” These measures assume clearly defined positive labels and a labelled evaluation sample. If false positives and false negatives have different costs, a threshold should be selected using the business cost of error rather than accuracy alone.

A confusion matrix, class-specific recall and calibration are often more informative than overall accuracy, especially when dissatisfaction is rare. Analysts should also test performance across languages, customer groups, products and time periods.

4. Large Language Models and Generative AI

An LLM is a model trained to estimate likely sequences of tokens from context. Transformer architectures use attention mechanisms to relate tokens to one another across a context window. During generation, the model repeatedly selects or samples a next token, conditioned on the preceding prompt and generated text. The model does not retrieve truth automatically; it generates a statistically plausible continuation based on learned patterns and current context.

Generative AI refers more broadly to systems that create text, images, audio, code or other content. In business, common language-model tasks include drafting, transformation, extraction, summarization, classification, conversational search and coding assistance.

A useful distinction is between:

• Closed-book generation: The model responds from its learned parameters and the prompt.
• Grounded generation: The model receives relevant, current source material and is instructed to base its response on that material.
• Tool-using generation: The model calls software such as a calculator, database, search engine or transaction system.

The last two designs can reduce some errors, but neither guarantees correctness. A model may misread retrieved text, use an inappropriate source or produce an unsupported conclusion.

Prompt design

A prompt is an instruction and contextual input provided to a model. Effective prompts typically specify:

1. the role or task;
2. the relevant context;
3. the expected output format;
4. constraints and decision rules;
5. examples, when useful; and
6. an instruction to identify uncertainty or missing information.

For example, instead of asking “Summarize this complaint,” an analyst might specify: “Summarize the complaint in three bullet points. Identify the product, the requested remedy and any safety issue. Do not infer facts not stated in the text. If a field is absent, write ‘not stated.’”

Zero-shot prompting provides instructions without examples. Few-shot prompting provides examples of desired input-output behaviour. Structured prompting requests a table or a fixed schema. Chain-of-thought prompting has been used to encourage intermediate reasoning, but organizations should not assume that a model’s verbalized reasoning is a faithful account of its internal process. For auditability, it is safer to request concise justifications, citations, extracted evidence and confidence or uncertainty indicators rather than relying on hidden reasoning narratives.

Prompt quality is not a substitute for evaluation. Small wording changes can alter outputs, and a prompt that works on one model or version may fail after an update. Production prompts should be version-controlled and tested with representative cases.

5. Retrieval-Augmented Generation

Retrieval-augmented generation combines information retrieval with text generation. A typical RAG pipeline is:

1. collect and authorize source documents;
2. divide documents into meaningful chunks;
3. create an embedding for each chunk and store it in an index;
4. convert a user query into an embedding;
5. retrieve relevant chunks using semantic, keyword or hybrid search;
6. place selected evidence in the model context;
7. instruct the model to answer only from the evidence, cite sources and acknowledge gaps;
8. evaluate both retrieval and generation.

Chunk size matters. Very small chunks may lose context; very large chunks may exceed the context budget and reduce retrieval precision. Metadata such as document title, department, date, access level and version should be retained. Access controls must be enforced during retrieval, not merely at the user interface.

RAG has two distinct quality questions:

• Retrieval quality: Did the system find the evidence needed to answer?
• Generation quality: Did the model accurately use that evidence?

A fluent answer cannot compensate for missing or unauthorized evidence. Retrieval can be evaluated with measures such as recall at k, which asks whether relevant evidence appears among the top k results. Generation can be evaluated for factual consistency, completeness, citation correctness and helpfulness.

RAG is especially suitable for policies, product documentation, procedures and frequently changing internal knowledge. It is less suitable when the answer requires a transaction, a precise calculation or a legally binding interpretation unless the system is connected to authoritative tools and reviewed by an accountable professional.

6. Hallucination, Bias and Model Limitations

A hallucination is an output that is unsupported, false or misleading, even if it sounds confident. Causes include incomplete context, ambiguous prompts, imperfect training data, next-token generation, conflicting sources and pressure to provide an answer when the evidence is insufficient.

Risk-reduction practices include:

• grounding responses in authoritative sources;
• requiring citations or quoted evidence;
• allowing “insufficient information” as an answer;
• using calculators, databases or deterministic rules for exact tasks;
• applying output validation and schema checks;
• separating extraction from interpretation;
• sampling outputs for expert review; and
• monitoring errors after deployment.

Other limitations include:

• Knowledge cutoff and change: Learned information may be outdated.
• Context limits: Important information may be omitted or diluted in long inputs.
• Sensitivity: Outputs can vary with wording, order and formatting.
• Bias: Training data can reproduce stereotypes or unequal treatment.
• Confidentiality risk: Sensitive data may be exposed through prompts, logs or vendors.
• Automation bias: Users may accept fluent answers without sufficient checking.
• Reproducibility: Sampling and model updates can change results.
• Copyright and provenance: Generated text may resemble protected or confidential material.
• Adversarial input: Prompt injection can cause a system to disregard instructions or disclose data.

A model’s confidence-like language is not reliable evidence of correctness. Confidence should be estimated through validated procedures, such as agreement across repeated runs, classifier calibration or evidence-coverage checks, and should be communicated carefully.

7. Human Oversight and Enterprise Governance

Human oversight should be designed according to risk, not added as a vague final instruction. A low-risk system that drafts internal meeting summaries may require user review before distribution. A system affecting credit, employment, healthcare, insurance or legal rights requires stronger controls, documented accountability, fairness testing and often human decision authority.

A practical governance framework addresses:

Organizations should maintain model cards or system documentation describing purpose, data, limitations, evaluation results, known failure modes and intended users. A data-protection impact assessment may be appropriate when personal data or high-impact decisions are involved. Applicable law varies by jurisdiction, so legal review is required; technical controls do not replace regulatory compliance.

Human review is meaningful only when reviewers have sufficient time, expertise, authority and access to evidence. If a reviewer merely clicks “approve” on every output, the process creates the appearance rather than the substance of oversight.

8. Selecting Business Use Cases

A use case should be assessed across value, feasibility and risk. High-value, relatively lower-risk applications often include internal search, document summarization with source links, ticket classification, meeting-action extraction and drafting that is always reviewed. Higher-risk applications include autonomous customer eligibility decisions, employee screening, medical advice, legal conclusions and automated termination or denial decisions.

A simple prioritization matrix can consider:

• frequency and cost of the current task;
• potential time or quality improvement;
• availability of authoritative data;
• tolerance for error;
• reversibility of mistakes;
• privacy and security sensitivity;
• degree of required human judgement; and
• measurable evaluation criteria.

The correct question is not “Where can we add a chatbot?” but “Which language-intensive activity can be improved without transferring unacceptable risk to customers, employees or the organization?”

Applied Case: Illustrative Customer-Complaint Intelligence System

Background

A fictional online retailer receives 30,000 customer messages each month through email and web forms. Managers want faster identification of delivery failures, defective products and possible safety concerns.

Problem

The existing keyword rules produce many false positives and do not distinguish product dissatisfaction from delivery dissatisfaction. Managers also need evidence for each reported trend.

Analysis

The analytics team defines four tasks: classify the complaint type, extract product and order identifiers, estimate aspect-level sentiment, and summarize weekly themes. A stratified sample of messages is labelled by trained reviewers. The sample includes multiple products, languages and channels. Reviewers document disagreement rather than forcing uncertain cases into a label.

A TF–IDF classifier is built as an interpretable baseline. An embedding model supports semantic retrieval of similar complaints. A domain-tested language model extracts structured fields and produces summaries using retrieved examples and current policy documents. Safety-related messages are routed to a specialist rather than resolved automatically.

Approach and solution

The production workflow uses the following controls:

1. Personal data is minimized before general analysis, while authorized staff retain access to necessary order details.
2. Classification outputs include probabilities or review flags, but these are validated rather than treated as truth.
3. RAG supplies current product and safety procedures, with document titles and dates attached to each answer.
4. The model must return “not stated” when an identifier or fact is absent.
5. Messages involving safety, legal threats or vulnerable customers require human review.
6. A sample of accepted outputs is audited weekly, with error rates tracked by complaint type and language.

Outcome

In this illustrative case, the retailer does not evaluate success only by the apparent fluency of summaries. It measures routing recall for safety complaints, precision of defect classification, extraction accuracy, review time and the rate of unsupported statements. Managers use aggregated themes to prioritize product and logistics improvements, while individual customer decisions remain subject to established service procedures.

Learning points

The case demonstrates that text analytics, embeddings and generative AI perform different roles. A specialized classifier may be preferable for stable routing; embeddings improve discovery; and an LLM is useful for flexible extraction and summarization. Retrieval, source citations and human escalation reduce—but do not eliminate—risk. Most importantly, business process design and evaluation determine whether the technology creates value.

Contemporary Developments and Best Practices

Multimodal models increasingly combine text with images, audio and tables. Smaller domain-specific models may offer lower cost, latency and data exposure than general-purpose models. Hybrid retrieval combines keyword matching with embeddings, which is valuable when exact identifiers and semantic concepts both matter. Tool-using systems can call databases and calculators, but permissions and transaction controls must be explicit.

Recommended practice is to:

1. begin with a narrow, measurable business problem;
2. establish a simple non-generative baseline;
3. create representative, high-quality evaluation data;
4. separate retrieval, reasoning, generation and action permissions;
5. use authoritative sources and preserve provenance;
6. test edge cases, adversarial prompts and distribution shifts;
7. protect personal, confidential and proprietary information;
8. require appropriate human review;
9. monitor quality, cost, latency, drift and incidents; and
10. reassess the system whenever the model, data or business process changes.

Chapter Summary

Text analytics converts language into information that supports classification, extraction, search, discovery and decision-making. Traditional representations such as TF–IDF are interpretable and useful baselines, while embeddings support semantic similarity and retrieval. Sentiment analysis may operate at document or aspect level and must be evaluated with measures such as precision and recall.

Large language models generate text from learned patterns and prompts; they do not guarantee factual accuracy. Prompt design should specify tasks, context, constraints and output formats, but prompts require systematic testing. Retrieval-augmented generation grounds responses in selected documents, making retrieval quality, source authorization and citation accuracy central concerns.

Enterprise adoption requires attention to hallucination, bias, privacy, security, copyright, changing knowledge and automation bias. Human oversight must be matched to risk and supported by evidence, authority and accountability. The strongest business applications combine suitable models with reliable data, clear evaluation, controlled workflows and continuous governance.

Key Terms

Natural-language processing: Computational analysis and generation of human language.

Tokenization: Division of text into words, subwords, characters or other processing units.

TF–IDF: A term-weighting method that emphasizes terms frequent in a document but uncommon in a collection.

Embedding: A numerical vector representation intended to capture characteristics or meaning of text.

Cosine similarity: A measure of the angle-based similarity between two non-zero vectors.

Sentiment analysis: Estimation of evaluative attitude or polarity in text.

Aspect-based sentiment analysis: Sentiment analysis linked to particular products, features or topics.

Named-entity recognition: Identification and categorization of entities such as people, firms and locations.

Large language model: A language model trained on very large text collections to predict and generate token sequences.

Generative AI: Artificial intelligence that produces new content such as text, images, audio or code.

Prompt: Instructions and contextual input supplied to a generative model.

Zero-shot prompting: Task instruction without worked examples.

Few-shot prompting: Task instruction accompanied by examples of desired behaviour.

Retrieval-augmented generation: Generation supported by information retrieved from an external knowledge source.

Hallucination: A false, unsupported or misleading model output.

Prompt injection: An attempt to manipulate a model through input so that it violates intended instructions or controls.

Human-in-the-loop: A system design in which people review, approve, correct or oversee automated outputs.

Model drift: Deterioration or change in system performance as data, language or operating conditions change.

Provenance: Information about the origin, version and supporting evidence of data or an output.

Review Questions

Short Answer Questions

1. What are the principal stages of a text-analytics pipeline?
2. How does aspect-based sentiment analysis differ from document-level sentiment analysis?
3. What does cosine similarity measure in an embedding system?
4. Define hallucination in the context of generative AI.
5. Why should a business establish a non-generative baseline?

Long Answer Questions

1. Explain how TF–IDF and embeddings represent text, and compare their strengths and limitations.
2. Discuss prompt-design principles for an enterprise document-summarization system.
3. Explain the retrieval and generation stages of a RAG system and describe how each should be evaluated.
4. Analyze the main privacy, security, fairness and accountability issues in deploying an LLM.
5. Explain why meaningful human oversight requires more than placing an approval step after model generation.

Analytical / Application Questions

1. A bank wants to use an LLM to summarize loan applications and recommend approvals. Evaluate the use case, identify major risks and propose a safer system design.
2. A retailer’s sentiment model has high overall accuracy but misses most negative comments in a minority language. Diagnose the problem and recommend an evaluation and improvement plan.
3. Design a RAG workflow for an organization’s internal policy assistant. Specify document controls, retrieval steps, prompt constraints, human escalation rules and performance measures.

Multiple Choice Questions with Answers

Question 1

Which representation is designed to give greater weight to a term that is frequent in one document but uncommon across a collection?

A. TF–IDF

B. One-hot customer ID encoding

C. A confusion matrix

D. A transaction log

Correct Answer: A. TF–IDF

Question 2

What does cosine similarity primarily compare between two embedding vectors?

A. Their document publication dates

B. The angle between their directions

C. The number of labels in each document

D. The legal status of their sources

Correct Answer: B. The angle between their directions

Question 3

Which measure answers: of the items predicted positive, how many were actually positive?

A. Recall

B. Specificity

C. Precision

D. Coverage

Correct Answer: C. Precision

Question 4

Which feature is most characteristic of retrieval-augmented generation?

A. It prevents all model errors

B. It supplies retrieved external evidence to the generator

C. It removes the need for document authorization

D. It works only with numerical data

Correct Answer: B. It supplies retrieved external evidence to the generator

Question 5

Why is aspect-based sentiment analysis useful for product management?

A. It ignores product features

B. It identifies sentiment associated with particular aspects

C. It guarantees unbiased labels

D. It eliminates the need for evaluation

Correct Answer: B. It identifies sentiment associated with particular aspects

Question 6

Which prompt is most appropriately constrained for extracting complaint information?

A. Write something useful about this complaint.

B. Decide what the customer really wants, even if it is not stated.

C. Extract product, requested remedy and safety issue; write ‘not stated’ when absent.

D. Make the complaint sound more positive.

Correct Answer: C. Extract product, requested remedy and safety issue; write ‘not stated’ when absent.

Question 7

Which statement about an LLM’s fluent answer is correct?

A. Fluency guarantees factual accuracy.

B. Fluency is evidence that the sources were authorized.

C. Fluency may occur even when the answer is unsupported or false.

D. Fluency eliminates the need for human review.

Correct Answer: C. Fluency may occur even when the answer is unsupported or false.

Question 8

What is a central purpose of chunking documents in a RAG system?

A. To make every document identical in length

B. To create retrievable units while attempting to preserve useful context

C. To remove all metadata

D. To guarantee that every answer is correct

Correct Answer: B. To create retrievable units while attempting to preserve useful context

Question 9

Which use case generally requires the strongest safeguards?

A. Drafting an internal meeting agenda

B. Suggesting headings for a public report

C. Summarizing a non-sensitive article

D. Making an automated decision affecting a person’s eligibility for credit

Correct Answer: D. Making an automated decision affecting a person’s eligibility for credit

Question 10

Why should a business use a simple baseline before deploying a generative system?

A. To establish a comparison for value and performance

B. To ensure the model never changes

C. To avoid collecting evaluation data

D. To replace governance requirements

Correct Answer: A. To establish a comparison for value and performance