Transformers and Pretrained Language Models

Introduction and Context

Transformers are the dominant architecture behind modern natural language processing (NLP), large language models, machine translation systems, search assistants, speech applications, and many generative artificial intelligence tools. Their importance arises from a central design decision: instead of processing a sequence strictly from left to right, a transformer can relate each token to other relevant tokens through self-attention.

Earlier recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) processed tokens sequentially. This made them useful for language modelling but created difficulties with long-range dependencies and parallel computation. Transformers, introduced by Vaswani and colleagues in 2017, replaced recurrence with attention and feed-forward transformations. This permitted efficient training on large datasets and provided a flexible foundation for pretrained language models.

A pretrained language model learns general linguistic and world patterns from large collections of text. It can then be adapted to specific tasks through fine-tuning, prompting, instruction tuning, retrieval augmentation, or parameter-efficient methods. Understanding the transformer architecture is therefore essential for designing, evaluating, and responsibly deploying generative AI systems.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the purpose and operation of self-attention and multi-head attention.
2. Describe positional representations and why order information is required.
3. Distinguish encoder-only, decoder-only, and encoder–decoder transformer architectures.
4. Explain common pretraining objectives and their relationship to downstream tasks.
5. Compare tokenization methods and calculate basic sequence dimensions.
6. Describe transfer learning, fine-tuning, prompting, and parameter-efficient adaptation.
7. Identify major transformer model families and select an appropriate family for an application.
8. Evaluate practical, computational, ethical, and governance limitations of pretrained language models.

1. From Sequential Models to Attention

Language is sequential, but the meaning of a token often depends on distant tokens. In the sentence “The bank approved the loan because it had sufficient capital,” the interpretation of “it” depends on the broader sentence. An effective language model must represent both local syntax and long-range relationships.

An RNN updates a hidden state sequentially. In simplified form, the hidden state at position t depends on the current input and the preceding state. Although gated architectures reduce information loss, sequential computation limits training parallelism. A transformer instead represents all positions in a sequence and computes relationships among them simultaneously, subject to a finite context window and computational budget.

The basic transformer layer contains:

• an attention sublayer;
• a position-wise feed-forward network;
• residual connections;
• layer normalization; and
• a positional representation or positional signal.

A stack of such layers gradually transforms token representations from surface-level patterns into contextual representations.

2. Tokenization: Converting Text into Model Inputs

Neural models operate on numerical vectors rather than raw text. Tokenization converts text into a sequence of tokens, and a vocabulary maps each token to an integer identifier. A token may be a word, subword, character, punctuation mark, or byte-level unit.

2.1 Word, character, and subword tokenization

Word tokenization is intuitive but produces very large vocabularies and fails on unseen words. Character tokenization handles arbitrary spellings but usually creates long sequences. Subword tokenization balances these limitations. Common methods include:

• Byte-pair encoding (BPE): repeatedly merges frequent adjacent symbol pairs.
• WordPiece: selects subword units using a likelihood-oriented vocabulary-building procedure.
• Unigram language modelling: begins with a large candidate vocabulary and removes units that contribute least to corpus likelihood.
• Byte-level tokenization: represents text through byte-derived units, improving coverage across writing systems and unusual strings.

For example, a word such as “unhappiness” might be represented as “un”, “happi”, and “ness”, depending on the vocabulary. Tokenization is not linguistically neutral: it affects sequence length, multilingual performance, spelling robustness, cost, and the model’s ability to represent names or technical terms.

A typical input is augmented with special tokens. An encoder model may use a classification token and separator tokens, while a decoder model commonly uses beginning-of-sequence and end-of-sequence markers. Padding tokens allow examples of different lengths to form a batch; an attention mask prevents padding from influencing computation.

2.2 Vocabulary and sequence length

If a tokenized input has length n and the model embedding dimension is d, its initial representation can be written as a matrix X ∈ ℝⁿˣᵈ. Each row represents one token embedding. The sequence length is measured in tokens, not characters or words. A long document may therefore exceed a model’s context window even when its word count appears moderate.

3. Self-Attention

Self-attention allows each position to assign different importance to other positions in the same sequence. Three learned transformations produce queries, keys, and values:

Here, X is the input representation matrix; Q, K, and V are the query, key, and value matrices; and W^Q, W^K, and W^V are learned projection matrices. If the model dimension is d and the query/key dimension is dₖ, then Q and K have shape n × dₖ, while V has shape n × dᵥ.

The attention weights are computed by scaled dot products:

The transpose Kᵀ produces an n × n score matrix. Dividing by √dₖ controls the magnitude of dot products. Without scaling, large dimensions can produce very large scores, causing the softmax function to become excessively concentrated and making optimization less stable. The softmax operation is applied row by row, so each query position obtains a probability distribution over key positions.

For a query vector q and key vectors kⱼ, the unnormalized score is:

The corresponding attention weight is:

The output is a weighted sum of value vectors. The weights are dimensionless, and the output has the same value dimension as V. Attention is therefore a content-dependent mixture rather than a fixed convolution.

Worked example: a small attention calculation

Suppose a query has scores [1, 2, 0] after scaling. The softmax weights are approximately [0.245, 0.665, 0.090], because exp(1), exp(2), and exp(0) are normalized by their sum. The second token receives the greatest weight. If the corresponding one-dimensional values are [10, 20, 5], the output is approximately:

This example illustrates weighting, not the full training process. In an actual model, the scores, values, and projections are learned and multidimensional.

3.1 Multi-head attention

One attention operation may emphasize syntactic agreement, another may track a named entity, and another may connect a pronoun with its antecedent. Multi-head attention gives the model several representation subspaces:

Here, h is the number of heads and W^O is a learned output projection. Usually, the model dimension is divided across heads so that the total projection size remains manageable. Multi-head attention does not guarantee that every head has a clear human-interpretable function; apparent specialization may be partial, distributed, or unstable across layers.

3.2 Attention masks

A mask modifies attention scores before softmax. In an encoder, padding positions are masked. In an autoregressive decoder, a causal mask prevents position t from attending to future positions t + 1 onward. A masked score is often replaced by a very negative number so its softmax weight is effectively zero.

Causal masking is essential for next-token prediction. Without it, a training example could reveal the target token to the model, producing information leakage and an invalid training objective.

4. Positional Representations

Self-attention alone is permutation-equivariant: if token representations are rearranged and the same rearrangement is applied to the inputs, attention cannot inherently determine the original order. Language, however, depends strongly on order. “Dog bites person” does not mean the same as “Person bites dog.”

Positional information is added or integrated into token representations in several ways.

4.1 Sinusoidal positional encoding

The original transformer used deterministic sinusoidal functions. For position p and feature index i:

Here, d is the representation dimension, p is the token position, and i indexes feature pairs. The arguments are dimensionless; position is treated as an index rather than a physical unit. Sinusoidal encodings provide multiple frequencies and can, in principle, support extrapolation beyond the positions observed during training, although practical extrapolation is not guaranteed.

4.2 Learned and relative positions

Learned absolute position embeddings assign a trainable vector to each position up to a specified maximum. Relative position methods represent the distance or direction between tokens, often making it easier to model patterns such as “the previous word” or “three positions later.” Rotary position representations rotate query and key components as a function of position, allowing relative-position information to appear in their dot products. Each method involves trade-offs in implementation, context extension, memory, and extrapolation.

5. The Transformer Block

A standard transformer block combines attention with a feed-forward transformation. A simplified post-normalization form is:

The residual connection adds the sublayer input to its output, supporting gradient flow and preserving information. Layer normalization normalizes features within each token representation. The feed-forward network is applied independently to each position:

W₁ and W₂ are learned matrices, b₁ and b₂ are bias vectors, and σ is a nonlinear activation such as GELU. Unlike attention, the feed-forward network does not directly mix information between positions; attention performs that cross-position interaction.

Modern implementations may use pre-normalization, RMS-based normalization, gated feed-forward units, optimized attention kernels, and other engineering refinements. These variations preserve the broad conceptual structure while improving stability or efficiency.

6. Encoder, Decoder, and Encoder–Decoder Architectures

6.1 Encoder-only models

An encoder uses bidirectional self-attention: a token can attend to tokens on both sides, subject to padding masks. Encoder outputs are contextual representations suitable for classification, named entity recognition, semantic similarity, retrieval, and extractive question answering.

BERT is the best-known example. Its original pretraining included masked language modelling, in which selected tokens are hidden and predicted from their context. Encoder-only models are generally not designed to generate unrestricted text token by token, because their bidirectional training does not enforce causal generation.

6.2 Decoder-only models

A decoder-only transformer uses causal self-attention. At position t, it can attend only to positions at or before t. It is trained commonly through next-token prediction and is naturally suited to text generation, dialogue, code completion, and instruction following.

If a token sequence is x₁, x₂, …, xₙ, an autoregressive model factorizes the sequence probability as:

The model predicts the next token from the preceding context. During training, all positions can be computed in parallel using a causal mask; during generation, tokens are usually produced sequentially.

6.3 Encoder–decoder models

An encoder–decoder transformer has an encoder that reads the source sequence and a decoder that generates a target sequence. The decoder contains causal self-attention and cross-attention to encoder outputs. Cross-attention uses decoder queries and encoder-derived keys and values. This architecture is especially appropriate for translation, summarization, and structured sequence transformation.

7. Pretraining Objectives

A pretraining objective defines what the model learns before task-specific adaptation.

7.1 Causal language modelling

The objective maximizes the likelihood of each token given previous tokens. The loss for a sequence is commonly the average negative log-likelihood:

Here, n is the number of prediction positions, x₍₁:ₜ₋₁₎ is the preceding token context, and θ denotes model parameters. The logarithm is usually natural logarithm, so the loss is measured in nats per token. If logarithm base 2 is used, the unit is bits per token.

7.2 Masked language modelling

Masked language modelling replaces or hides selected input tokens and trains the model to reconstruct them using both left and right context. It is useful for learning bidirectional representations, though the corruption process differs from ordinary downstream text.

7.3 Sequence-to-sequence denoising

Encoder–decoder models may corrupt spans, shuffle or delete text, and train the decoder to reconstruct the original sequence. Span corruption encourages the model to generate missing passages rather than merely classify individual masked tokens.

7.4 Other objectives

Additional objectives include contrastive learning, sentence-order or entailment-related tasks, code completion, preference optimization, and multimodal alignment. The usefulness of an objective depends on data quality, task distribution, model architecture, and evaluation design. A low pretraining loss does not automatically imply factual accuracy, fairness, reasoning ability, or safe deployment.

8. Transfer Learning and Adaptation

Transfer learning uses general representations learned from broad data for a particular task or domain. Common adaptation strategies include:

• Feature extraction: freeze the pretrained model and train a task-specific prediction layer.
• Full fine-tuning: update most or all parameters using labelled task data.
• Instruction tuning: train on examples expressed as instructions and desired responses.
• Prompting: specify the task in the input without changing model weights.
• Few-shot prompting: provide a small number of demonstrations in context.
• Parameter-efficient fine-tuning: update a small parameter set, such as adapter modules or low-rank matrices.
• Retrieval-augmented generation (RAG): retrieve external documents and provide them as context during generation.

Low-rank adaptation represents a weight update approximately as:

where B and A are lower-rank matrices. If W has dimensions m × n and the adaptation rank is r, the trainable parameters are approximately r(m + n) rather than mn, provided r is much smaller than both dimensions. This reduces memory and storage requirements, but does not remove the need for careful data selection and evaluation.

Adaptation may cause catastrophic forgetting, in which performance on earlier capabilities declines. It can also amplify biases or encode confidential training examples. Versioning datasets, prompts, checkpoints, and evaluation results is therefore a professional requirement.

9. Major Transformer Model Families

Examples of influential families include BERT-style encoders, GPT-style decoder-only models, and T5-style text-to-text encoder–decoder models. Model names do not by themselves determine capability; training data, scale, tokenizer, optimization, alignment, context length, and evaluation all matter.

10. Computational Considerations

For a standard self-attention layer, the attention score matrix has n² entries. Thus, attention time and memory generally grow quadratically with sequence length, approximately O(n²d) for the score-related operations when representation dimension is d. Feed-forward computation is approximately O(nd²) for common dense layers. These are asymptotic descriptions, not exact hardware runtime formulas.

Long-context systems use techniques such as sparse or local attention, grouped-query attention, key–value caching, sliding windows, recurrent memory, compressed representations, and optimized kernels. During autoregressive inference, key–value caching stores prior keys and values so they do not need to be recomputed for every new token. This improves generation speed but increases memory use with context length.

The practical cost of a model includes training energy, accelerator time, data preparation, inference latency, memory, and monitoring. Smaller specialized models may be preferable to very large general models when the task is narrow and reliability or cost is important.

11. Illustrative Case Study: A University Policy Assistant

Background

A university wishes to deploy an assistant that answers questions about admissions, assessment rules, fees, and academic integrity. Policies change frequently and contain exceptions.

Problem

A general decoder-only language model can produce fluent responses but may invent a deadline or rely on obsolete information. Direct fine-tuning on a small policy corpus may also memorize outdated rules without providing reliable source attribution.

Analysis

The problem is primarily knowledge access and controlled generation, not merely language fluency. Policy documents are authoritative, versioned, and institution-specific. A suitable design must distinguish retrieved evidence from model-generated wording.

Approach and solution

The university implements a RAG pipeline. Documents are approved, dated, chunked, embedded, indexed, and retrieved using the user’s query. Retrieved passages are inserted into a constrained prompt instructing the model to answer only from the evidence, cite document titles and dates, and state uncertainty when evidence is insufficient. An encoder-style model may support semantic retrieval, while a decoder-only or encoder–decoder model generates the answer. A human escalation path handles disputed or high-risk cases.

Evaluation includes retrieval recall, citation correctness, answer faithfulness, refusal quality, latency, and performance across demographic and linguistic groups. Access controls prevent confidential student records from entering general prompts.

Outcome

The system is more auditable and easier to update than a model that stores all policy knowledge in its parameters. However, retrieval does not guarantee correctness: an incorrect index, incomplete passage, ambiguous policy, or misleading prompt can still produce a poor answer.

Learning points

1. Architecture should follow the task: retrieval and controlled generation may be more appropriate than indiscriminate fine-tuning.
2. Current, authoritative evidence is essential for changing institutional information.
3. Evaluation must test factual grounding, not only linguistic quality.
4. Human oversight, privacy protection, and clear responsibility remain necessary.

12. Challenges, Ethics, and Governance

Bias and representation

Pretraining data reflect social inequalities, stereotypes, and uneven language coverage. A model may produce different quality or harmful associations across languages, dialects, identities, or domains. Testing should use representative and adversarial evaluation sets, and mitigation should be assessed for both benefit and unintended effects.

Hallucination and calibration

A fluent output may be unsupported or false. Confidence implied by prose is not a reliable probability estimate. High-risk applications should use retrieval, structured validation, abstention, citations, and human review.

Privacy and data governance

Training and prompting may expose personal, confidential, or copyrighted material. Organizations should define lawful data sources, retention rules, access controls, deletion procedures, and incident response. Sensitive information should not be entered into a model merely because the interface permits it.

Security

Prompt injection can cause a system to ignore intended instructions, especially when retrieved documents contain adversarial text. Data poisoning, model extraction, membership inference, and insecure tool use are additional risks. Systems should isolate instructions from untrusted content, validate tool arguments, limit privileges, and log important actions.

Environmental and economic impact

Large-scale training and inference consume energy and specialized hardware. Model selection should consider energy, latency, total cost of ownership, and whether a smaller model can meet the requirement. Responsible design values sufficient performance rather than maximum scale.

13. Best Practices for Professional Use

1. Define the task, risk level, users, and success criteria before selecting a model.
2. Inspect tokenization and context limits using representative data.
3. Use a model family suited to the task rather than assuming that generation is always required.
4. Separate training, validation, and test data; avoid leakage between them.
5. Evaluate factuality, robustness, fairness, latency, cost, and safety.
6. Version the model, tokenizer, prompts, datasets, retrieval index, and software environment.
7. Use retrieval or tools for changing facts and require evidence where appropriate.
8. Apply least-privilege access, privacy controls, monitoring, and escalation procedures.
9. Treat benchmark scores as evidence, not as a complete safety or quality guarantee.
10. Re-evaluate after changes to data, model weights, prompts, or external knowledge sources.

Chapter Summary

Transformers replace recurrence with attention-based interaction among token representations. Self-attention uses queries, keys, and values, with scaled dot products and softmax weights determining how information is mixed. Multi-head attention permits several learned relationship patterns, while masks control padding and causal visibility. Because attention alone does not encode order, positional representations are required.

Encoder-only models provide bidirectional contextual representations; decoder-only models generate autoregressively under causal masking; and encoder–decoder models transform one sequence into another through self-attention and cross-attention. Pretraining objectives include causal language modelling, masked language modelling, and sequence-to-sequence denoising. Transfer learning adapts these general models through fine-tuning, prompting, parameter-efficient methods, or retrieval augmentation.

Tokenization affects vocabulary coverage, sequence length, cost, and multilingual performance. Transformer deployment must address quadratic attention costs, memory, latency, hallucination, bias, privacy, security, and governance. Effective professional practice combines architectural understanding with evidence-based evaluation and responsible system design.

Key Terms

Self-attention: An operation in which each token representation attends to other token representations in the same sequence.

Query: A projected representation used to seek relevant information.

Key: A projected representation compared with a query to calculate relevance.

Value: A projected representation that is combined according to attention weights.

Multi-head attention: Several attention operations performed in different learned subspaces.

Causal mask: A mask that prevents a position from attending to future positions.

Positional encoding: Information supplied to a model to represent token order.

Tokenization: Conversion of text into model-processing units.

Subword token: A token representing part of a word or a frequently occurring character sequence.

Context window: The maximum sequence context that a model can process in one interaction.

Encoder: A transformer component that builds contextual representations of an input sequence.

Decoder: A transformer component designed to generate or conditionally produce a sequence.

Cross-attention: Attention from decoder states to encoder outputs.

Masked language modelling: Predicting hidden or corrupted tokens using surrounding context.

Causal language modelling: Predicting each token from preceding tokens.

Transfer learning: Applying knowledge learned from one data distribution or task to another.

Fine-tuning: Updating pretrained model parameters on task-specific data.

Parameter-efficient fine-tuning: Adaptation that updates only a small subset or compact transformation of parameters.

Retrieval-augmented generation: Generation supported by externally retrieved evidence.

Hallucination: An unsupported, fabricated, or incorrect model output presented as if meaningful or factual.

Review Questions

Short Answer Questions

1. Why is scaling by √dₖ used in the self-attention score calculation?
2. What problem does positional encoding solve?
3. Distinguish bidirectional attention from causal attention.
4. State one suitable application for each of encoder-only and decoder-only models.
5. Why can tokenization affect the cost of model inference?

Long Answer Questions

1. Explain the complete flow of multi-head self-attention, including the roles of queries, keys, values, masking, concatenation, and projection.
2. Compare encoder-only, decoder-only, and encoder–decoder transformers in terms of architecture, objectives, and applications.
3. Explain causal language modelling and masked language modelling, including one advantage and one limitation of each.
4. Discuss how positional representations have developed from sinusoidal encodings to relative and rotary approaches.
5. Describe a responsible transfer-learning strategy for adapting a pretrained model to a specialized organizational domain.

Analytical / Application Questions

1. A university wants a system that summarizes current policy documents and cites the relevant sections. Which transformer architecture and adaptation strategy would you recommend, and why?
2. A model performs well on English news but poorly on multilingual customer messages. Analyze how tokenizer design, training data, context length, and evaluation may contribute to the problem.
3. Design an evaluation and governance plan for a decoder-only model used to answer health-information questions. Identify risks, metrics, safeguards, and conditions requiring human escalation.

Multiple Choice Questions with Answers

Question 1

Which expression correctly represents scaled dot-product attention?

A. softmax(QKᵀ / √dₖ)V

B. softmax(Q + K + V)

C. QKᵀV / dₖ²

D. softmax(QVᵀ / √dᵥ)K

Correct Answer: A. softmax(QKᵀ / √dₖ)V

Question 2

What is the primary purpose of a causal mask in a decoder-only transformer?

A. To remove punctuation tokens

B. To prevent a position from attending to future tokens

C. To make all attention weights equal

D. To increase the vocabulary size

Correct Answer: B. To prevent a position from attending to future tokens

Question 3

Which model architecture is most naturally suited to unrestricted next-token text generation?

A. Encoder-only

B. Decoder-only

C. Convolution-only

D. Embedding-only

Correct Answer: B. Decoder-only

Question 4

If a sequence has 12 tokens, how many entries are in its full self-attention score matrix before masking?

A. 24

B. 48

C. 144

D. 1,728

Correct Answer: C. 144

Question 5

Why are positional representations needed in a transformer?

A. To convert probabilities into labels

B. To provide information about token order

C. To remove the need for embeddings

D. To guarantee factual outputs

Correct Answer: B. To provide information about token order

Question 6

Which objective predicts a hidden token using information on both sides of it?

A. Masked language modelling

B. Causal language modelling

C. Greedy decoding

D. Beam pruning

Correct Answer: A. Masked language modelling

Question 7

What is cross-attention in an encoder–decoder transformer?

A. Attention from decoder states to encoder outputs

B. Attention only among padding tokens

C. Attention that ignores the source sequence

D. Attention between two unrelated batches

Correct Answer: A. Attention from decoder states to encoder outputs

Question 8

Which method updates a small trainable transformation rather than all original model weights?

A. Parameter-efficient fine-tuning

B. Random token deletion

C. Causal masking

D. Vocabulary sorting

Correct Answer: A. Parameter-efficient fine-tuning

Question 9

What is a major practical limitation of standard full self-attention for long sequences?

A. Its score matrix generally grows quadratically with sequence length

B. It cannot represent numerical tokens

C. It always requires character tokenization

D. It prevents parallel training

Correct Answer: A. Its score matrix generally grows quadratically with sequence length

Question 10

Why is retrieval-augmented generation useful for frequently changing institutional policies?

A. It permanently removes the need for evaluation

B. It supplies current external evidence at generation time

C. It guarantees that every retrieved document is correct

D. It eliminates all privacy risks

Correct Answer: B. It supplies current external evidence at generation time