NLP and Generative AI Applications

Introduction

Natural Language Processing (NLP) enables computers to analyse, interpret and generate human language. Modern NLP combines linguistic knowledge, statistical learning, neural networks and large-scale pre-trained models. Generative Artificial Intelligence (GenAI) extends this capability by producing new text, dialogue, summaries, translations, code and recommendations from learned patterns and user instructions.

Applications such as sentiment analysis, named-entity recognition, question answering and summarisation were traditionally developed as separate systems. Transformer architectures and foundation models now allow many of these tasks to be implemented through fine-tuning, prompting or retrieval-augmented generation. However, a successful application is not defined only by fluent output. It must also be accurate, useful, secure, explainable, fair and appropriate for its operational context.

This chapter examines the design and evaluation of practical NLP and GenAI solutions. It focuses on the complete path from problem definition and data preparation to model selection, deployment, monitoring and governance.

Learning Objectives

After studying this chapter, students should be able to:

1. Distinguish major NLP and generative-AI application tasks.
2. Design pipelines for classification, sentiment analysis and named-entity recognition.
3. Explain the operation of question-answering, summarisation and translation systems.
4. Compare prompting, fine-tuning and retrieval-augmented generation.
5. Design conversational, content-generation and decision-support applications.
6. Select suitable evaluation metrics and interpret their limitations.
7. Identify risks involving hallucination, bias, privacy, security and accountability.
8. Propose a responsible implementation for a real organisational problem.

1. An Application-Centred NLP Framework

An NLP application transforms language inputs into predictions, retrieved evidence, generated outputs or actions. A useful abstraction is:

Here, the input may be a document, question or conversation; context may include retrieved documents or user history; the task specification defines the desired behaviour; the model performs inference; and policy imposes safety, privacy and business constraints. The same foundation model can therefore behave very differently depending on its context and controls.

A production pipeline commonly contains the following stages:

1. Problem definition: Specify the user, decision, input, output, acceptable error and consequences of failure.
2. Data preparation: Collect representative data, remove or protect sensitive information, label examples and divide data into training, validation and test sets.
3. Representation: Convert text into tokens, embeddings or structured linguistic features.
4. Modelling: Select a classifier, sequence tagger, retrieval system, language model or hybrid architecture.
5. Evaluation: Measure task quality, robustness, fairness, latency, cost and safety.
6. Deployment: Integrate the model with software, databases, identity controls and user interfaces.
7. Monitoring: Detect drift, abuse, performance degradation and changes in user needs.

The most important design principle is to match the system’s level of autonomy to the risk of the task. A model may draft a low-risk email without approval, while a system supporting medical, legal, financial or employment decisions should provide evidence, uncertainty information and human review.

2. Text Classification and Sentiment Analysis

2.1 Classification

Text classification assigns one or more labels to a text. In single-label classification, exactly one class is selected, such as spam or not spam. In multi-label classification, several labels may apply, such as billing, technical fault and urgent.

A classifier estimates a conditional probability such as P(y|x), where x is the text and y is a class. In a neural classifier, the text encoder produces a representation h, and a classification layer produces class scores. For K classes, the softmax function is commonly used:

Here, z_k is the score for class k. The scores are converted into probabilities that sum to one. This formula is appropriate for mutually exclusive classes; multi-label systems generally use an independent sigmoid output for each label.

The standard training objective for a labelled single-label dataset is cross-entropy loss:

In this expression, N is the number of examples, y_ik is 1 when example i belongs to class k and 0 otherwise, and p_ik is the predicted probability. The loss penalises confident incorrect predictions more strongly than uncertain predictions. It assumes that labels are sufficiently reliable and that the training distribution is relevant to the deployment distribution.

2.2 Sentiment Analysis

Sentiment analysis identifies affective orientation in text. A basic system predicts positive, negative or neutral sentiment. More advanced systems identify emotion categories, sentiment intensity, aspects and targets. For example, in the sentence “The camera is excellent, but the battery is disappointing,” overall sentiment is mixed; aspect-based analysis links excellent to the camera and disappointing to the battery.

A practical workflow includes:

• defining the sentiment scale and annotation guidelines;
• distinguishing opinion from factual description;
• handling negation, irony, slang, code-switching and domain vocabulary;
• evaluating separately by language, demographic group, product and time period;
• inspecting errors rather than relying only on an aggregate score.

Sentiment is not a direct measurement of a person’s psychological state. It is an interpretation of linguistic evidence, and confidence should decrease when text is short, ambiguous or sarcastic.

2.3 Evaluation of Classifiers

A confusion matrix records true positives, true negatives, false positives and false negatives. Important metrics include:

Precision measures the proportion of predicted positives that are correct. Recall measures the proportion of actual positives that are detected. F1 is their harmonic mean and is useful when both are important. These metrics assume a clearly defined positive class. Accuracy can be misleading when classes are imbalanced. A fraud detector, for example, may achieve high accuracy by predicting “not fraud” for every transaction.

Thresholds should be selected according to the cost of errors. If missing a safety complaint is more harmful than sending an unnecessary alert, the system should prioritise recall, possibly at the expense of precision.

3. Named-Entity Recognition and Information Extraction

Named-Entity Recognition (NER) identifies spans of text and assigns types such as PERSON, ORGANISATION, LOCATION, DATE, PRODUCT or MEDICAL CONDITION. In “Dr Amina Khan joined Orion Systems in Nairobi,” a system may identify “Amina Khan” as PERSON, “Orion Systems” as ORGANISATION and “Nairobi” as LOCATION.

NER is commonly formulated as sequence labelling. Each token receives a tag using the BIO scheme:

Modern systems use transformer encoders, sometimes followed by a conditional random field or constrained decoding layer. Domain adaptation is often essential because general-purpose models may not recognise specialised entities such as chemical compounds, legal clauses or engineering components.

Evaluation should use entity-level precision, recall and F1 rather than only token accuracy. A prediction is normally correct when both the entity span and its type match the gold annotation. A system that identifies only part of “University of Cape Town” has not fully extracted the entity under strict evaluation.

NER has direct applications in document search, customer analytics, redaction, compliance monitoring and knowledge-graph construction. It also creates privacy risks: extracting names, addresses and health conditions can increase the exposure of personal information. Access control, encryption, retention limits and purpose limitation are therefore necessary.

4. Question Answering and Retrieval-Augmented Generation

Question-answering (QA) systems produce answers to user questions. They may be:

• extractive, selecting a span from a source document;
• generative, composing an answer in natural language;
• closed-domain, restricted to a particular knowledge base;
• open-domain, searching broad external collections.

A reliable enterprise architecture often uses retrieval-augmented generation (RAG). The retriever searches relevant documents, and a generator uses the retrieved passages to formulate an answer. A simplified pipeline is:

Figure: A retrieval-augmented generation pipeline.

RAG reduces unsupported generation when the knowledge base is authoritative and retrieval is effective, but it does not guarantee truth. Errors may arise from incomplete indexing, poor chunking, outdated documents, ambiguous queries or an answer that contradicts the evidence. Production systems should display citations, record retrieved passages and provide an abstention response when evidence is insufficient.

QA evaluation may include exact match and token-level F1 for extractive tasks, but generative answers require assessment of correctness, relevance, completeness, groundedness and readability. Human review remains important for high-impact domains.

5. Summarisation and Translation

5.1 Summarisation

Summarisation compresses a source into a shorter representation while preserving important meaning. Extractive summarisation selects existing sentences; abstractive summarisation generates new wording. Abstractive methods are more flexible but may introduce unsupported claims, altered quantities or omitted qualifications.

A professional summarisation specification should state the audience, desired length, coverage requirements, acceptable style and treatment of uncertainty. For legal, medical or technical material, the system should preserve modality: “may,” “must,” and “is proven” are not interchangeable.

Automatic metrics such as ROUGE compare overlap with reference summaries. They are useful for benchmarking but do not fully measure factuality or usefulness. A summary can have high word overlap and still reverse the meaning of a sentence. Evaluation should therefore include factual consistency checks and expert review.

5.2 Machine Translation

Machine translation maps a source-language sequence to a target-language sequence. Neural translation models use an encoder to represent the source and a decoder to generate the target. Important concerns include terminology consistency, gender and politeness, word order, idioms, low-resource languages and culturally sensitive expressions.

BLEU measures modified n-gram precision with a brevity penalty, while chrF evaluates character n-gram similarity. These metrics support comparison but should not be treated as complete measures of translation quality. Human evaluation should examine adequacy, fluency, terminology and preservation of meaning. For safety-critical translation, a qualified human must review the output.

6. Conversational AI

A conversational system may combine speech recognition, dialogue management, retrieval, language generation, tool execution and speech synthesis. Its quality depends on the whole interaction rather than a single response.

A robust design includes:

• conversation state: user goal, prior turns, unresolved references and permissions;
• grounding: trusted documents, databases or tools;
• intent and entity handling: structured interpretation where deterministic action is needed;
• fallbacks: clarification, transfer to an agent or safe refusal;
• tool controls: validation of parameters and confirmation before consequential actions;
• logging: privacy-preserving records for auditing and improvement.

A language model should not be granted unrestricted authority to send payments, delete records or alter access rights. Tool use should follow least privilege, schema validation, allow-lists and transaction confirmation. Prompt injection is a major threat: malicious text in a document or user message may attempt to override system instructions or cause unauthorised tool use.

Conversation evaluation includes task completion, turns per task, escalation rate, response latency, user satisfaction and safety incidents. A high completion rate is not sufficient if the system completes tasks incorrectly.

7. Content Generation and Decision Support

Generative systems can draft reports, product descriptions, learning materials, software documentation, test data and creative content. A useful generation specification identifies:

1. purpose and audience;
2. factual sources and prohibited claims;
3. tone, format and length;
4. required citations or evidence;
5. review and approval responsibilities.

Prompting can use role instructions, constraints, examples and structured output schemas. Prompt engineering is not a substitute for data governance or evaluation. For repeated tasks, fine-tuning may improve consistency, terminology and style, but it requires representative data and careful control of memorisation and bias.

Decision-support systems should inform rather than obscure human judgement. They may rank cases, identify relevant evidence, estimate risk or propose alternatives. A decision-support output should distinguish facts, assumptions, predictions and recommendations. It should also expose the basis for the recommendation and allow an authorised person to challenge or override it.

A useful conceptual separation is:

• descriptive: What has happened?
• diagnostic: Why might it have happened?
• predictive: What may happen next?
• prescriptive: What action is recommended?

Generative language can make a recommendation sound more certain than the underlying evidence. Interfaces should therefore display confidence cautiously, cite sources and avoid presenting probability as certainty.

8. Worked Application: University Student-Service Assistant

The following is an illustrative case.

Background

A university receives thousands of questions about registration, fees, examinations, accommodation and technical support. Students currently search multiple web pages or wait for email responses.

Problem

The university wants a conversational assistant that answers routine questions, classifies unresolved cases and transfers sensitive matters to staff. It must not invent policy or expose another student’s information.

Analysis

The system has several tasks:

• intent classification for routing;
• NER for programme names, dates and student-reference numbers;
• retrieval from approved policy documents;
• grounded answer generation;
• sentiment and urgency detection for escalation;
• structured ticket creation.

The risk analysis identifies privacy, outdated policy, incorrect fee information, prompt injection and unequal performance across languages.

Approach and Solution

The university creates a versioned document repository. Each policy passage includes an owner, effective date and review date. Documents are chunked by semantic section, embedded into a vector index and also indexed with keywords. Hybrid retrieval improves performance for both natural-language questions and exact policy terms.

The assistant first classifies intent. For high-risk categories such as appeals, disability accommodations and financial hardship, it provides general information and transfers the case to an authorised adviser. For routine questions, it retrieves relevant passages and generates an answer containing citations and the policy date. If retrieval confidence is low or sources conflict, the assistant asks a clarifying question or abstains.

Student-reference numbers are masked in logs. Staff review a sample of conversations each week. A dashboard reports grounded-answer rate, escalation rate, unresolved queries, latency, source freshness and error reports. Policy owners receive alerts before documents expire.

Outcome

In this illustrative case, routine questions are answered more quickly, while complex cases are routed with better context. The most important improvement is not merely automation; it is controlled access to current policy and transparent escalation.

Learning Points

1. A foundation model is only one component of an application.
2. Retrieval quality and document governance strongly affect answer quality.
3. Abstention is a necessary capability, not a failure.
4. High-risk cases require human involvement.
5. Monitoring must cover safety, fairness and operational performance.

9. Governance, Ethics and Professional Practice

NLP systems process language that may contain personal, confidential or copyrighted material. Organisations should establish a lawful purpose, minimise collection, define retention periods and restrict access. Sensitive data should not be placed in external model services without an appropriate contractual and technical assessment.

Bias can enter through data, annotation, model pre-training, retrieval sources or deployment decisions. Evaluation should be stratified across relevant languages, dialects, demographic groups and document types. Fairness is context-dependent; identical error rates may not be feasible or appropriate in every application, so the organisation must document its chosen criteria.

Important controls include:

• data provenance and versioning;
• documented model and prompt changes;
• red-team testing for harmful outputs and prompt injection;
• human review for high-impact decisions;
• access control and audit logs;
• incident response and rollback procedures;
• disclosure when users interact with an AI system;
• copyright and licensing review for training and generated material.

Generated content can contain hallucinations, which are plausible but unsupported statements. Mitigation includes retrieval, constrained decoding, tool verification, post-generation checks, calibrated abstention and human review. No single technique eliminates hallucination in all contexts.

10. Contemporary Developments and Best Practices

Current systems increasingly combine language models with multimodal inputs, structured tools, vector databases, knowledge graphs and smaller specialised models. Efficient deployment may use quantisation, batching, caching and model routing. Smaller models can be preferable where latency, privacy or cost matters.

Agentic systems can plan and execute multiple steps, but increased autonomy also increases failure modes. A practical progression is to begin with read-only retrieval, then introduce narrowly scoped tools, explicit approval and extensive testing. Autonomous action should be justified by demonstrated reliability and low consequence of failure.

Best practice is to:

1. define success before selecting a model;
2. establish a strong baseline, including non-generative methods;
3. use representative and legally usable data;
4. separate development, validation and final testing;
5. evaluate both average and worst-group performance;
6. ground factual answers in authoritative sources;
7. design clear refusal and escalation paths;
8. protect personal and confidential information;
9. monitor drift, cost, latency and safety incidents;
10. maintain human accountability for consequential outcomes.

Chapter Summary

NLP and Generative AI applications include classification, sentiment analysis, NER, question answering, summarisation, translation, conversational AI, content generation and decision support. Classification predicts labels, while NER extracts typed spans and sentiment analysis interprets opinion or emotion. Precision, recall and F1 provide complementary measures, especially under class imbalance.

Question-answering systems may extract, retrieve or generate answers. Retrieval-augmented generation improves grounding when supported by authoritative, current documents, but it still requires citation, abstention and evaluation. Summarisation and translation must be assessed for meaning preservation, not only lexical similarity. Conversational systems require state management, secure tool use, escalation and monitoring.

Generative systems are valuable for drafting and decision support, but fluent language does not guarantee truth. Effective professional practice combines suitable models with data governance, privacy protection, fairness testing, prompt-injection defence, human oversight and continuous monitoring. The central design principle is to align system autonomy with the consequences of error.

Key Terms

Natural Language Processing: Computational analysis and generation of human language.

Generative AI: AI that creates new content from learned patterns and instructions.

Text classification: Assigning one or more categories to a text.

Sentiment analysis: Estimating opinion, polarity, emotion or intensity in language.

Named-Entity Recognition: Identifying and typing spans such as people, places and organisations.

BIO tagging: A sequence-labelling scheme using beginning, inside and outside tags.

Precision: The proportion of predicted positive cases that are correct.

Recall: The proportion of actual positive cases that are detected.

F1 score: The harmonic mean of precision and recall.

Question answering: Producing an answer to a natural-language question.

Retrieval-Augmented Generation: Generating an answer using retrieved external evidence.

Grounding: Linking an output to reliable evidence or data.

Hallucination: An unsupported or fabricated model-generated claim.

Abstention: Declining to answer when evidence or confidence is inadequate.

Summarisation: Producing a shorter representation of source material.

Machine translation: Automatically converting content from one language to another.

Prompt injection: An attempt to manipulate system behaviour through malicious instructions in input content.

Fine-tuning: Training a pre-trained model on task- or domain-specific examples.

Decision support: Information or recommendations that assist, but do not necessarily replace, human judgement.

Model drift: A decline in performance caused by changes in data, behaviour or environment.

Review Questions

Short Answer Questions

1. What is the difference between single-label and multi-label classification?
2. Why can accuracy be misleading for an imbalanced classifier?
3. What does a BIO tagging scheme represent?
4. State two advantages and two limitations of retrieval-augmented generation.
5. Why is human review important in high-impact NLP applications?

Long Answer Questions

1. Explain the complete lifecycle of an NLP application from problem definition to monitoring.
2. Compare extractive and abstractive summarisation, including their principal risks.
3. Describe how a conversational AI system can securely use external tools.
4. Explain the role of precision, recall and F1 in evaluating a safety-complaint classifier.
5. Discuss ethical, privacy and governance requirements for a university AI assistant.

Analytical / Application Questions

1. A customer-service classifier detects 90 of 100 genuine urgent cases and incorrectly labels 30 routine cases as urgent. Calculate its recall and precision, and recommend whether further threshold analysis is needed.
2. Design a RAG architecture for an organisation whose policies change monthly. Specify document governance, retrieval, citation, abstention and monitoring controls.
3. A generative system produces fluent loan recommendations but cannot provide evidence for them. Analyse the risks and propose a safer decision-support design.

Multiple Choice Questions with Answers

Question 1

Which metric measures the proportion of predicted positive cases that are actually positive?

A. Recall

B. Precision

C. Accuracy

D. Specificity

Correct Answer: B. Precision

Question 2

In BIO tagging, what does B-ORG normally indicate?

A. A token outside every entity

B. A continuation of a person entity

C. The beginning of an organisation entity

D. The end of an organisation entity

Correct Answer: C. The beginning of an organisation entity

Question 3

What is the principal purpose of retrieval-augmented generation?

A. To remove the need for source documents

B. To ground generated responses in retrieved evidence

C. To guarantee that every generated answer is correct

D. To replace evaluation with prompting

Correct Answer: B. To ground generated responses in retrieved evidence

Question 4

Which situation is most appropriate for prioritising recall?

A. Missing a genuine safety complaint has serious consequences

B. Every false positive is legally prohibited

C. The classes are perfectly balanced and errors have equal cost

D. The system is generating creative poetry

Correct Answer: A. Missing a genuine safety complaint has serious consequences

Question 5

Which is an example of aspect-based sentiment analysis?

A. Labelling every review only as positive or negative

B. Detecting the language of a review

C. Linking positive sentiment to a camera and negative sentiment to its battery

D. Counting the number of words in a review

Correct Answer: C. Linking positive sentiment to a camera and negative sentiment to its battery

Question 6

What is a major limitation of ROUGE for summarisation evaluation?

A. It cannot compare any text sequences

B. It measures factual truth and safety perfectly

C. It may reward word overlap without detecting factual distortion

D. It is applicable only to speech recognition

Correct Answer: C. It may reward word overlap without detecting factual distortion

Question 7

Which control best limits the risk of unauthorised conversational-agent actions?

A. Giving the model unrestricted administrator access

B. Using least privilege, parameter validation and confirmation

C. Removing all logs from the system

D. Allowing documents to override system policies

Correct Answer: B. Using least privilege, parameter validation and confirmation

Question 8

What is hallucination in a generative-AI system?

A. A deliberately cited answer from an approved database

B. A model output that is unsupported or fabricated

C. A temporary network delay

D. A correctly classified named entity

Correct Answer: B. A model output that is unsupported or fabricated

Question 9

Which task is most directly associated with identifying PERSON and LOCATION spans?

A. Named-entity recognition

B. Machine translation

C. Text compression

D. Speech synthesis

Correct Answer: A. Named-entity recognition

Question 10

Why should an NLP system support abstention?

A. To ensure it always produces a confident answer

B. To avoid answering when evidence or confidence is inadequate

C. To eliminate the need for monitoring

D. To increase unsupported content generation

Correct Answer: B. To avoid answering when evidence or confidence is inadequate