3. Transformer Architecture and LLM Design

Introduction and Context

Large language models (LLMs) are built from a family of neural architectures known as Transformers. Introduced in 2017, the Transformer replaced recurrent processing with attention-based computation. This change was consequential: it allowed models to process tokens in parallel during training, represent long-range relationships more directly, and scale effectively with data, parameters, and compute.

A Transformer is not merely a stack of attention layers. It is an engineered system involving token representation, positional information, normalization, residual pathways, feed-forward transformations, masking, optimization, memory management, and infrastructure. Decisions about these components affect accuracy, latency, training stability, cost, interpretability, and safety. For postgraduate students, understanding the architecture therefore requires both mathematical comprehension and design judgment.

This chapter examines the mechanisms underlying modern Transformer-based LLMs. It compares encoder-only, encoder-decoder, and decoder-only designs; explains multi-head self-attention, positional encoding, normalization, residual connections, and mixture-of-experts models; and interprets scaling laws and computational trade-offs. The discussion also considers practical deployment, professional decision-making, and emerging architectural directions.

Learning Objectives

After studying this chapter, students should be able to:

1. Explain self-attention mathematically and intuitively.

2. Distinguish encoder-only, encoder-decoder, and decoder-only Transformer architectures.

3. Analyze the roles of positional encoding, normalization, and residual connections.

4. Evaluate mixture-of-experts architectures and their routing trade-offs.

5. Interpret scaling laws in relation to parameters, data, and compute.

6. Estimate the computational and memory implications of attention and inference.

7. Select an appropriate architecture for a language or multimodal application.

8. Identify design risks involving efficiency, robustness, governance, and deployment.

From Sequential Processing to Attention

Earlier language models commonly used recurrent neural networks, including LSTMs and GRUs. These models processed tokens sequentially, maintaining a hidden state that summarized preceding context. Although recurrence supported variable-length inputs, it limited parallelism and made it difficult to preserve information over very long sequences.

The Transformer treats a sequence as a collection of token representations and allows each token to interact with other tokens through attention. If a sentence contains the words “The bank approved the loan,” the representation of “bank” may attend to “approved” and “loan,” helping the model infer a financial meaning. Attention is content-dependent: the model learns which positions are relevant for a particular token and task.

A Transformer layer usually contains two principal sublayers: a multi-head attention mechanism and a position-wise feed-forward network. Each sublayer is surrounded by residual connections and normalization. Repeating this structure many times produces a deep network capable of progressively transforming token representations.

Self-Attention: Mechanism and Interpretation

Let the input sequence be represented by a matrix X with one row per token. Self-attention constructs three projections:

Q = XW_Q, K = XW_K, V = XW_V

where Q, K, and V denote queries, keys, and values. The query expresses what a token is seeking; the key describes what each token offers for matching; and the value contains information that will be aggregated. The attention output is:

Attention(Q,K,V) = softmax(QK^T / √d_k)V

The product QK^T produces pairwise compatibility scores. Division by √d_k prevents dot products from becoming excessively large as the key dimension increases. Softmax converts scores into normalized weights. The output for each token is therefore a weighted combination of value vectors from the sequence.

Self-attention differs from fixed-window methods because every token can, in principle, access every other token in the same sequence. It also differs from recurrence because all positions can be processed simultaneously during training. However, full attention has a time and memory complexity of approximately O(n²d), where n is sequence length and d is the representation dimension. The quadratic term becomes expensive for long contexts.

Multi-Head Attention

Rather than using one attention operation, Transformers commonly use several attention heads. Each head has separate projections and can learn a different relationship, such as syntactic agreement, coreference, positional proximity, or semantic association. The head outputs are concatenated and projected back into the model dimension.

Multi-head attention does not guarantee that each head has a simple human-interpretable function. Nevertheless, multiple subspaces increase representational flexibility. The number of heads, their dimension, and the total model dimension are architectural hyperparameters that influence quality and efficiency.

Causal Attention and Masking

In autoregressive language modeling, a token must not use information from future tokens when predicting the next token. A causal mask assigns a very negative value to attention scores directed toward future positions. After softmax, those positions receive effectively zero probability. Causal masking preserves the factorization:

P(x_1,...,x_n) = ∏ᵢ P(xᵢ | x_1,...,xᵢ₋₁)

During training, all positions can still be computed in parallel because the mask controls information flow. During generation, tokens are produced sequentially, which creates an inference bottleneck.

Position: Encoding Order into Representations

Attention alone is permutation-equivariant: if token order is changed and no positional signal is supplied, the model cannot inherently distinguish the resulting arrangement. Positional information is therefore essential.

Original Transformers used sinusoidal positional encodings. For position p and dimension i, sinusoidal functions with different frequencies were added to token embeddings. Learned absolute positional embeddings use a trainable vector for each position. Both approaches provide an explicit position representation, but absolute schemes can be less flexible when a model is used beyond its training context length.

Relative positional methods represent the distance or directional relationship between tokens. Rotary positional embeddings (RoPE), widely used in contemporary LLMs, rotate query and key representations according to position. This causes attention scores to depend on relative position while retaining useful absolute-position structure. Other approaches add learned biases based on token distance or modify attention scores directly.

Position encoding creates a practical design issue: extending context length is not simply a matter of accepting more tokens. The model’s training distribution, positional method, attention memory, and optimization behavior must all support the longer context. Long-context systems may use efficient attention, sliding windows, retrieval, or hierarchical memory in addition to positional modifications.

The Transformer Block: Residual Paths, Normalization, and Transformation

After attention, each token representation passes through a feed-forward network, often called an MLP. A typical form is:

FFN(x) = W₂ σ(W₁x + b₁) + b₂

The hidden dimension of the MLP is usually larger than the model dimension. Modern variants frequently use gated activations such as SwiGLU, in which one projection gates another. The MLP supplies token-wise nonlinear transformation, while attention enables communication between tokens.

Residual connections add a sublayer’s output to its input. In simplified form, the output is x + F(x). This creates a direct pathway through the network, improving gradient propagation and making optimization of deep models more stable. Residual streams also allow successive layers to refine rather than completely replace representations.

Normalization controls the scale of activations. Layer normalization normalizes features within each token representation. In a pre-normalization block, normalization occurs before attention and the feed-forward network; in a post-normalization block, it follows the residual addition. Pre-normalization is often easier to optimize in very deep networks, although exact behavior depends on the architecture and training regime. Root Mean Square Normalization (RMSNorm) omits mean subtraction and can reduce computation while maintaining effective scale control.

A simplified pre-normalized block can be written as:

x' = x + Attention(Norm(x))

y = x' + FFN(Norm(x'))

Small details—normalization placement, initialization, activation choice, residual scaling, and attention-logit scaling—can substantially affect training stability. Architecture design must therefore be evaluated as a system rather than as a collection of independent components.

Three Architectural Families

Encoder-Only Models

An encoder reads the entire input using bidirectional self-attention. Each token can attend to tokens on both sides. Encoder-only models are well suited to representation learning, classification, retrieval, token labeling, and semantic similarity. Masked language modeling, in which selected tokens are hidden and reconstructed, is a common pretraining objective.

These models are not naturally optimized for unrestricted text generation because their representations are produced from a complete input rather than a left-to-right generation process. They are attractive when the output is a label, vector, span, or structured prediction.

Encoder-Decoder Models

An encoder-decoder model first transforms an input sequence into contextual representations. A decoder then generates an output autoregressively. Decoder layers contain causal self-attention and cross-attention, allowing generated tokens to attend to encoder outputs.

This separation is especially useful for translation, summarization, instruction transformation, and other sequence-to-sequence tasks. The encoder can analyze the entire source while the decoder produces the target under causal constraints. The cost is architectural complexity and the need to maintain both source representations and decoder states.

Decoder-Only Models

Decoder-only models use causal self-attention throughout. They predict the next token and can perform generation, completion, dialogue, code synthesis, and in-context learning. Their uniform objective and scalable architecture have made them central to contemporary LLM development.

Their limitations include sequential generation, sensitivity to prompt formulation, and the fact that a single next-token objective does not automatically guarantee factuality, instruction following, or safe behavior. Post-training, retrieval, tool use, and evaluation are required to address these concerns.

Mixture-of-Experts and Conditional Computation

A dense Transformer activates the same parameters for every token. A mixture-of-experts (MoE) layer instead contains multiple expert feed-forward networks and a router that assigns each token to a small number of experts. If the model has many total parameters but activates only a subset for each token, it can increase parameter capacity without proportionally increasing the computation per token.

A router commonly computes scores over experts and selects the top one or top few. A load-balancing objective discourages the router from sending too many tokens to a small number of experts. Capacity limits may cause tokens to be dropped, rerouted, or processed by shared pathways when an expert is overloaded.

MoE models introduce communication costs in distributed training because tokens may need to move between devices. They can also suffer from routing instability, imbalanced expert utilization, and complicated deployment. Thus, “more parameters” does not necessarily mean better practical performance. Effective capacity, active parameters, data quality, routing quality, and infrastructure all matter.

Scaling Laws and Model Design

Scaling laws describe empirical relationships between model loss and quantities such as parameter count, training data, and compute. A common conceptual form is that loss decreases as these quantities increase, usually with diminishing returns. Scaling is therefore not an argument for unlimited model growth; it is a framework for allocating finite resources.

A model with many parameters but insufficient data may be undertrained. Conversely, abundant data paired with too few parameters may limit representational capacity. Compute-optimal training considers model size and the number of training tokens jointly. In practice, the optimal allocation also depends on data quality, sequence length, optimizer behavior, hardware utilization, and the intended inference workload.

Scaling laws predict average loss, not every capability or failure mode. Improvements in benchmark performance may be uneven, and emergent behavior claims require careful measurement. A model may become more fluent while remaining vulnerable to hallucination, bias, prompt injection, or distribution shift.

Computational Trade-offs in Training and Inference

Training efficiency benefits from parallel matrix operations, distributed data and model parallelism, mixed-precision arithmetic, activation checkpointing, and optimized attention kernels. Memory is consumed by parameters, gradients, optimizer states, activations, and attention-related intermediate values. Reducing one category may not solve the dominant bottleneck.

At inference time, autoregressive decoding repeatedly computes the next-token distribution. Key-value (KV) caching stores past keys and values so they need not be recomputed for every new token. This reduces computation but increases memory proportional to layers, sequence length, batch size, and head dimensions. Quantization can reduce memory and improve throughput, but aggressive quantization may reduce accuracy or destabilize certain workloads.

Architecture selection should reflect the application. A retrieval encoder may be preferable to a large generative model for semantic search. An encoder-decoder system may be efficient for controlled transformation. A decoder-only model may provide the broadest generative interface but incur high serving costs. Long-context attention, batching, latency requirements, privacy constraints, and hardware availability must be included in the decision.

Applied Scenario: Designing an Enterprise Knowledge Assistant

Background

Consider an illustrative organization, Northstar Manufacturing, that wants an assistant for maintenance engineers. The system must answer questions using equipment manuals, maintenance records, and safety procedures. Incorrect advice could cause downtime or physical risk.

Situation and Analysis

A general decoder-only LLM can produce fluent answers, but its internal training data may be outdated or unrelated to Northstar’s equipment. Fine-tuning alone may not reliably store every document or preserve document updates. Long prompts containing all manuals would create high latency and attention cost.

The design team compares three options. An encoder-only model can retrieve relevant passages efficiently but cannot independently generate a detailed answer. An encoder-decoder model can transform retrieved evidence into a response, while a decoder-only model can combine retrieval, reasoning, and tool calls in one interface. The team also evaluates context length, citation requirements, data privacy, and inference budget.

Approach and Solution

The selected architecture uses an encoder-based retrieval component to create embeddings and rank passages, followed by a decoder-only generation model. A reranker improves evidence selection. The generator receives the user question, retrieved passages, equipment identifier, and safety constraints. It is instructed to cite evidence, distinguish documented facts from uncertainty, and refuse unsupported maintenance procedures.

The serving system uses KV caching, quantization tested against a quality threshold, and request batching. A smaller model handles query classification and retrieval, while the larger model is reserved for complex responses. Human reviewers evaluate factual grounding, unsafe recommendations, latency, and escalation behavior.

Outcome

The design reduces unnecessary generation and makes document updates possible without retraining the generator. It does not eliminate hallucination: retrieval errors, ambiguous manuals, and model misinterpretation remain possible. Therefore, the system includes confidence indicators, source citations, audit logs, access controls, and a human escalation path.

Learning Points

The case demonstrates that architecture is an organizational decision as well as a technical one. The most capable general model is not automatically the best system. Retrieval, model specialization, safety controls, evaluation, and cost must be considered together.

Challenges, Risks, and Common Design Errors

A frequent error is treating parameter count as a complete measure of capability. Data composition, training quality, architecture, post-training, and evaluation affect performance. Another error is assuming that a longer context always improves reasoning. Excessive context can increase cost, dilute relevant evidence, and create attention competition.

Attention may identify statistical associations rather than reliable causal relationships. Models can generate plausible but false content, reproduce biases in data, leak memorized information, or follow malicious instructions embedded in retrieved documents. MoE systems add routing and distributed-systems failure modes. Quantization and pruning can produce uneven degradation across languages, domains, or safety behaviors.

Evaluation should therefore include task accuracy, calibration, robustness, fairness, privacy, latency, throughput, energy use, and cost. Tests should cover adversarial prompts, out-of-distribution inputs, long contexts, multilingual cases, and failure recovery. Monitoring after deployment is essential because user behavior and data distributions change.

Governance and Professional Responsibility

LLM architecture influences governance. A model that processes sensitive documents requires data minimization, access control, encryption, retention policies, and auditable logs. Organizations should document training sources where possible, licensing constraints, model versions, evaluation results, and known limitations.

High-impact applications require human oversight and clearly defined accountability. System designers should not describe a model as “understanding” or “verified” unless the claim is operationally supported. Safety policies must be implemented through system design, access controls, monitoring, and evaluation rather than relying exclusively on a prompt. Professional practice requires communicating uncertainty and preserving the ability to investigate harmful outputs.

Emerging Directions

Current research explores sparse and linear-attention alternatives, state-space sequence models, retrieval-integrated architectures, multimodal Transformers, adaptive computation, speculative decoding, and architectures that separate short-term context from persistent memory. Context extension methods seek to support very long documents without incurring full quadratic attention costs.

Another direction is system-level specialization. Smaller models, expert routing, tool-using agents, and cascades may outperform a single enormous model on cost-sensitive tasks. The central research question is increasingly not only how to enlarge a model, but how to allocate computation dynamically, ground outputs in reliable information, and measure capability under realistic conditions.

Best Practices for Transformer and LLM Design

1. Define the task, output constraints, latency target, privacy requirements, and failure cost before selecting a model.

2. Match architecture to the task: encoder for representation, encoder-decoder for transformation, and decoder-only for flexible generation.

3. Treat positional encoding, normalization, residual design, and masking as stability-critical components.

4. Estimate training and serving memory, including optimizer states, activations, KV cache, and communication overhead.

5. Use retrieval, tools, or specialized components when current or verifiable knowledge is required.

6. Evaluate quality together with robustness, fairness, safety, latency, energy, and total cost.

7. Validate quantization, sparsity, and MoE routing on the actual target workload.

8. Maintain monitoring, version control, documentation, and human escalation for consequential applications.

Chapter Summary

Transformers model token relationships through self-attention rather than recurrence. Queries, keys, and values determine how information is exchanged, while causal masks support autoregressive generation. Because attention does not inherently encode order, positional methods are required. Feed-forward networks provide nonlinear token-wise transformation, and residual connections and normalization support the training of deep networks.

Encoder-only, encoder-decoder, and decoder-only architectures serve different purposes. Mixture-of-experts models increase conditional capacity but introduce routing and communication challenges. Scaling laws explain how model size, data, and compute interact, but they do not guarantee factuality, safety, or real-world utility. Training and inference require careful management of quadratic attention, KV-cache memory, parallelism, quantization, and hardware constraints.

Effective LLM design is therefore a multi-objective engineering and governance problem. Architectural capability must be balanced with evidence grounding, reliability, cost, privacy, safety, and organizational accountability.

Key Terms

Self-attention: An operation in which each token aggregates information from other tokens using content-dependent weights.

Query: A vector representing the information a token seeks during attention.

Key: A vector used to determine how relevant a token is to a query.

Value: The information vector aggregated according to attention weights.

Multi-head attention: Several attention operations performed in different learned representation subspaces.

Causal mask: A constraint preventing a token from attending to future positions.

Positional encoding: Information supplied to represent token order or relative distance.

Rotary positional embedding: A positional method that rotates query and key vectors according to position.

Residual connection: A pathway that adds a sublayer output to its input.

Layer normalization: Feature normalization applied within each token representation.

RMSNorm: A normalization method based on root mean square scale without mean subtraction.

Feed-forward network: A token-wise nonlinear transformation within a Transformer block.

Encoder-only model: A bidirectional Transformer primarily used for representation and understanding tasks.

Encoder-decoder model: A Transformer with an input encoder and autoregressive output decoder.

Decoder-only model: A causal Transformer designed to predict subsequent tokens.

Mixture-of-experts: A conditional architecture that routes tokens to selected expert subnetworks.

Scaling law: An empirical relationship linking performance to parameters, data, or computation.

KV cache: Stored past key and value states used to accelerate autoregressive decoding.

Quantization: Representing model values with lower numerical precision to reduce cost or memory.

Cross-attention: Attention in which decoder queries attend to representations produced by an encoder.

Review Questions

Short Answer Questions

1. Why is the dot product in attention divided by the square root of the key dimension?

2. What problem does causal masking solve?

3. Explain the purpose of a residual connection in a Transformer block.

4. How does an encoder-only model differ from a decoder-only model?

5. What is the function of a KV cache during inference?

Long Answer Questions

1. Derive and explain the scaled dot-product attention mechanism, including the roles of queries, keys, values, and softmax.

2. Compare encoder-only, encoder-decoder, and decoder-only architectures for classification, translation, and open-ended generation.

3. Discuss how positional encoding, normalization, and residual connections contribute to Transformer performance and stability.

4. Explain mixture-of-experts architecture and evaluate its benefits and operational risks.

5. Analyze how parameter count, training data, compute, context length, and inference cost interact under scaling laws.

Analytical / Application Questions

1. You must build a low-latency semantic search service over millions of documents. Which Transformer architecture would you initially select, and what additional components would you require?

2. A decoder-only model produces fluent but unsupported answers from a changing technical knowledge base. Diagnose the architectural and system-level causes and propose a safer design.

3. An MoE model achieves strong benchmark results but has high inter-device communication and uneven expert utilization. Recommend an evaluation and optimization plan.

22. Multiple Choice Questions with Answers

Question 1

Why is the factor √d_k used in scaled dot-product attention?

A. To eliminate the need for positional encoding

B. To prevent attention logits from becoming excessively large

C. To force every attention head to select one token

D. To make causal masking unnecessary

Correct Answer: B. To prevent attention logits from becoming excessively large

Question 2

Which architecture is most naturally suited to bidirectional sentence representation for classification?

A. Encoder-only

B. Decoder-only

C. Decoder-only with causal masking

D. Mixture-of-experts decoder without attention

Correct Answer: A. Encoder-only

Question 3

What does a causal attention mask accomplish during autoregressive training?

A. It removes repeated tokens from the input

B. It allows every token to attend equally to all positions

C. It prevents a position from using future-token information

D. It converts an encoder into a retrieval model

Correct Answer: C. It prevents a position from using future-token information

Question 4

A system must translate a complete source document into a target document. Which feature of an encoder-decoder Transformer is especially useful?

A. The decoder can use cross-attention to encoder representations

B. The encoder can only view the first source token

C. The decoder ignores previously generated tokens

D. The model requires no positional information

Correct Answer: A. The decoder can use cross-attention to encoder representations

Question 5

What is the principal purpose of a residual connection?

A. To reduce the vocabulary size

B. To provide a direct pathway that supports gradient flow

C. To replace all nonlinear transformations

D. To prevent tokens from attending to one another

Correct Answer: B. To provide a direct pathway that supports gradient flow

Question 6

Which statement best describes a mixture-of-experts Transformer?

A. Every token activates every expert equally

B. Tokens are routed to a subset of expert subnetworks

C. It removes the need for a router

D. It guarantees lower total communication cost

Correct Answer: B. Tokens are routed to a subset of expert subnetworks

Question 7

Why can extending context length increase inference cost substantially?

A. Full attention involves interactions that grow approximately quadratically with sequence length

B. The vocabulary necessarily doubles

C. Layer normalization becomes impossible

D. Causal masking requires a separate model for every position

Correct Answer: A. Full attention involves interactions that grow approximately quadratically with sequence length

Question 8

What is the main inference benefit of KV caching?

A. It prevents all hallucinations

B. It allows future tokens to be viewed during decoding

C. It avoids recomputing past keys and values at every generation step

D. It replaces the model's feed-forward layers

Correct Answer: C. It avoids recomputing past keys and values at every generation step

Question 9

Which design is most appropriate for a knowledge assistant whose documents change frequently and whose answers must cite evidence?

A. Use only a larger pretrained model with no document access

B. Use retrieval with a generator and source-grounding controls

C. Remove all safety and access controls to improve fluency

D. Use only positional embeddings without attention

Correct Answer: B. Use retrieval with a generator and source-grounding controls

Question 10

What is a central limitation of interpreting scaling laws?

A. They show that more data always reduces every type of risk

B. They predict average performance trends but do not guarantee factuality or safety

C. They apply only to encoder-only models

D. They eliminate the need for evaluation after deployment

Correct Answer: B. They predict average performance trends but do not guarantee factuality or safety