Protocol Models, Encapsulation and Communication Processes

Introduction

Computer networks are built from many hardware and software components produced by different vendors. A laptop, switch, router, wireless access point and web server must nevertheless exchange information predictably. Protocols provide the rules that make this possible. A protocol specifies such matters as message format, addressing, timing, error handling and the actions taken when communication succeeds or fails.

Protocol models organize these rules into layers. The two most important models for network technicians are the seven-layer Open Systems Interconnection (OSI) reference model and the four- or five-layer TCP/IP model. These models are not physical devices and are not exact descriptions of every implementation. They are conceptual frameworks used to design networks, interpret packet captures, troubleshoot faults and explain how applications communicate across interconnected systems.

This chapter follows a message from an application on one host to an application on another. It explains encapsulation, protocol data units, framing, segmentation, packet forwarding, ports and the difference between connection-oriented and connectionless communication.

Learning Objectives

By the end of this chapter, you should be able to:

• describe the purpose and layers of the OSI and TCP/IP models;
• map common protocols and devices to appropriate layers;
• explain encapsulation and decapsulation;
• distinguish data, segments, datagrams, packets and frames;
• trace segmentation, framing and router-based forwarding;
• explain ports, sockets and protocol headers;
• compare connection-oriented and connectionless communication;
• interpret a simplified packet header and identify likely troubleshooting locations.

1. Why Layered Protocol Models Are Used

A network could be designed as one large set of rules, but such a design would be difficult to develop and maintain. Layering divides communication into related functions. Each layer provides services to the layer above and uses services from the layer below.

Layering offers several advantages:

1. Modularity: A change in one layer need not redesign every other layer. Ethernet can be replaced by Wi-Fi while an application continues to use HTTP.
2. Interoperability: Equipment from different manufacturers can communicate when it follows common standards.
3. Troubleshooting: A technician can ask whether the failure is physical, local-network, routing, transport or application related.
4. Encapsulation: Each layer can add control information without requiring the application to understand lower-level details.
5. Education and design: Complex communication processes become easier to describe and implement.

Layering is an abstraction. Real protocols sometimes cross layer boundaries. Network Address Translation, firewalls, tunnelling and content-aware load balancers are examples of technologies that inspect or modify information associated with several layers.

2. The OSI Reference Model

The OSI model, standardized by ISO, contains seven layers. They are commonly listed from lowest to highest as follows:

The upper three layers are often implemented together in modern TCP/IP software. The OSI model remains valuable because it separates communication concerns clearly.

Layer 1: Physical

The physical layer transmits raw bits as electrical, optical or radio signals. It defines characteristics such as voltage or light levels, modulation, timing, connector types, channel frequencies and data rates. A damaged cable, incorrect transceiver or radio interference is primarily a physical-layer problem.

A repeater regenerates or retransmits signals but does not make forwarding decisions based on IP addresses. Physical transmission is local to a link; it does not by itself understand networks or applications.

Layer 2: Data Link

The data-link layer organizes bits into frames and provides communication over a directly connected network segment. Ethernet and IEEE 802.11 Wi-Fi use link-layer addresses, commonly called MAC addresses. A switch examines frame information and forwards frames through selected ports.

A frame normally contains a header, a payload and a trailer. The header may contain source and destination MAC addresses; the trailer commonly contains a frame-check sequence used to detect corruption. Detection is not the same as correction: a corrupted Ethernet frame is normally discarded, and recovery, if required, is performed by a higher layer.

Layer 3: Network

The network layer provides logical addressing and forwarding between different networks. IP addresses identify interfaces in an internetworking scheme. A router examines a destination IP address, consults its routing table and sends the packet toward the next hop.

A router does not normally preserve the original frame as it forwards traffic. The incoming frame is removed, the IP packet is examined, and a new frame appropriate for the next link is created. The source and destination IP addresses generally remain the same from end to end, although fields such as time to live change and address translation may modify addresses.

Layer 4: Transport

The transport layer provides process-to-process communication. It uses port numbers to identify applications or services on a host. TCP provides reliable, ordered, connection-oriented byte-stream delivery. UDP provides a connectionless datagram service with low protocol overhead but no general guarantee of delivery, order or duplicate suppression.

Transport protocols may divide application data into smaller units. TCP calls its transport unit a segment. UDP commonly calls its unit a datagram.

Layers 5–7

The session layer manages dialogue and logical sessions; the presentation layer handles representation, translation, compression and encryption; and the application layer provides services such as web browsing, name resolution and file transfer. In practice, these functions may be combined. For example, a web application can use HTTP at the application layer, TLS for confidentiality and authentication, TCP for transport, IP for internetworking and Ethernet or Wi-Fi for local delivery.

3. The TCP/IP Model

The TCP/IP model reflects the protocols used on the Internet. A common four-layer representation is:

Some textbooks use a five-layer Internet model by separating the physical and data-link layers. Both descriptions are useful if their terminology is stated clearly.

The OSI model is a general reference model, whereas TCP/IP is a practical protocol suite. It is therefore more accurate to say that TCP/IP protocols are mapped to OSI layers approximately, not that every TCP/IP component fits perfectly into exactly one OSI layer.

4. Protocol Data Units and Headers

A protocol data unit (PDU) is the unit of data defined at a particular layer. Common names are:

A header is control information placed before a payload. A trailer is control information placed after it. Header fields may identify addresses, ports, sequence numbers, lengths, flags, protocol types, checksums or time limits.

The term “packet” is sometimes used informally for any network data unit. In precise discussion, however, packet usually refers to a network-layer PDU, while frame refers to a data-link PDU.

5. Encapsulation and Decapsulation

Encapsulation is the process in which each lower layer treats the PDU from the layer above as its payload and adds its own control information. Decapsulation is the reverse process at the destination.

Suppose a user requests a web page:

1. The browser creates application data, such as an HTTP request.
2. TCP adds a header containing source and destination ports, sequence information and control flags. The result is a TCP segment.
3. IP adds a header containing source and destination IP addresses. The result is an IP packet.
4. Ethernet or Wi-Fi adds a link-layer header and usually a trailer. The result is a frame.
5. The physical layer transmits the frame as signals.

At the receiving host, the process occurs in reverse. The link layer verifies and removes the frame information, IP processes the packet, TCP reassembles and delivers the byte stream, and the application receives the request or response.

A simplified representation is:

Application data → [TCP header | data] → [IP header | TCP segment] → [Frame header | IP packet | trailer] → signals

Encapsulation permits independent responsibilities. The web application need not calculate a Wi-Fi frame check sequence, and a switch need not understand the meaning of an HTTP request.

Encapsulation at an Intermediate Router

At a router, decapsulation is normally only partial. The router receives a frame, checks the link-layer information, extracts the IP packet, makes a forwarding decision and encapsulates that same packet in a new outgoing frame. Thus, the link-layer addresses change at every routed hop, while the end-to-end IP addresses normally remain unchanged.

For example, a host may send a frame with:

• source MAC: Host A;
• destination MAC: Router interface R1;
• source IP: Host A’s IP address;
• destination IP: Server B’s IP address.

On the next link, the router may send:

• source MAC: Router interface R2;
• destination MAC: next-hop router or Server B;
• source IP: Host A’s IP address;
• destination IP: Server B’s IP address.

The changed MAC addresses reflect hop-by-hop delivery. The IP addresses identify the logical endpoints, subject to mechanisms such as NAT.

6. Segmentation, MTU and Fragmentation

Applications may produce data larger than a particular link can carry in one frame. The transport layer can segment the application data into manageable units. The maximum transmission unit (MTU) is the largest network-layer packet that a link can carry without requiring link-specific fragmentation.

TCP segmentation is influenced by the maximum segment size (MSS), which is commonly related to MTU by:

This relationship assumes a TCP/IP path in which the headers have the stated sizes and no additional options or encapsulation overhead is present. For example, with a 1500-byte MTU, a 20-byte IPv4 header and a 20-byte TCP header:

The result is a payload size, not the total TCP segment size. TCP options, IPv6 headers, tunnels and virtual private networks can change the practical value.

IPv4 may fragment a packet when it is too large for a next link, although modern networks commonly rely on Path MTU Discovery and avoid fragmentation. IPv6 routers do not fragment packets in transit; the sending host must use an appropriate packet size, or fragmentation is handled by the source using extension headers. Fragmentation increases overhead and complicates reassembly, so avoiding it is generally preferred.

7. Forwarding and Routing

A switch and a router make different decisions. A switch usually forwards frames within a local broadcast domain using MAC addresses. A router forwards packets between networks using IP addresses and a routing table.

A simplified router process is:

1. Receive a frame on an interface.
2. Check the frame’s integrity and destination link address.
3. Remove the incoming frame header and trailer.
4. Inspect the destination IP address.
5. Decrease the IPv4 time-to-live (TTL), or update the corresponding IPv6 hop limit.
6. Select the best matching route.
7. Resolve the next-hop link address if necessary.
8. Construct a new frame for the outgoing interface.
9. Transmit the frame.

The “best matching route” normally means the route with the longest matching network prefix. A default route is used when no more specific route exists. If no usable route exists, the router may discard the packet and generate an ICMP error, depending on the situation.

8. Ports, Sockets and Multiplexing

An IP address identifies a host interface or logical endpoint at the network layer. A port identifies a transport-layer service or process. Ports allow several applications to use the same IP address simultaneously.

A socket endpoint is commonly described by an IP address, a transport protocol and a port number. A TCP conversation is identified by a four-tuple:

source IP, source port, destination IP, destination port

The protocol is also relevant when distinguishing TCP from UDP. A client often uses a temporary ephemeral source port, while a server listens on a well-known or registered port. For example, HTTPS commonly uses TCP port 443, DNS commonly uses UDP port 53 for ordinary queries, and SSH commonly uses TCP port 22. Port numbers identify conventions, not proof of application identity; services can be configured to use different ports.

Multiplexing occurs when a transport layer accepts data from several applications and labels it with ports. Demultiplexing occurs when the receiving host uses destination port information to deliver data to the correct process.

9. Connection-Oriented and Connectionless Communication

Connection-Oriented Communication

TCP is connection-oriented. Before application data is normally exchanged, endpoints establish state through a handshake. The familiar TCP three-way handshake uses SYN, SYN-ACK and ACK control messages. TCP then provides sequence numbering, acknowledgements, retransmission, flow control and congestion-control mechanisms.

Connection-oriented service is suitable when complete, ordered data is important, such as web transactions, remote login and file transfer. It does not mean that the physical path is permanently reserved. TCP maintains logical state at the endpoints while routers continue to forward individual packets.

Connectionless Communication

UDP is connectionless. It sends independent datagrams without establishing a general-purpose session at the transport layer. UDP has a small header and does not itself guarantee delivery, order, retransmission or congestion control. Applications may add their own reliability or may accept occasional loss.

UDP is useful for DNS queries, real-time audio and video, online games, telemetry and protocols that implement reliability above UDP. Low delay can be more important than perfect delivery in real-time communication.

Connection-oriented does not always mean reliable in every protocol, and connectionless does not mean unusable or careless. These terms describe the service model and state maintained by the protocol.

10. Interpreting Simplified Headers

A technician can often locate a fault by reading headers in a packet capture. Important fields include:

• Ethernet: source and destination MAC addresses, EtherType and frame-check information;
• IP: source and destination addresses, version, header length, total length, TTL or hop limit, protocol or next-header field, and fragmentation fields;
• TCP: source and destination ports, sequence and acknowledgement numbers, flags, receive window and checksum;
• UDP: source and destination ports, length and checksum;
• Application protocols: request methods, response codes, names, commands or data formats.

For example, a packet with a correct destination IP but an unexpected destination port may indicate that the service is not listening, a firewall is filtering traffic, or the client used the wrong service configuration. Repeated TCP SYN packets without SYN-ACK responses suggest a problem involving the destination host, routing, filtering or service availability. A successful handshake followed by application errors points higher in the stack.

Header interpretation must consider encryption. TLS protects application content, so a packet capture may show addresses, ports and transport behaviour while hiding the HTTP request contents.

11. Worked Communication Trace

Consider a client at 192.0.2.10 using TCP source port 51500 to reach a web server at 198.51.100.20 on TCP port 443.

The client creates an HTTPS request. TCP identifies the conversation as:

192.0.2.10:51500 → 198.51.100.20:443

TCP places the request into one or more segments. IP adds the source and destination addresses. The client’s local Ethernet frame may use the client MAC address as source and the default gateway’s MAC address as destination, because the server is on another network.

The first router removes the Ethernet frame, examines the IP destination, reduces the TTL, selects an outgoing interface and creates a new frame. On the next link, both MAC addresses may be different. The IP destination remains 198.51.100.20 unless NAT is applied. At the server, the final frame is accepted, TCP uses port 443 to select the HTTPS service, and the data is passed upward.

The reply reverses the endpoint roles: the server uses source port 443 and destination port 51500. The client’s transport layer uses sequence and acknowledgement information to place the response correctly in its byte stream.

12. Applied Case: Diagnosing an Unreachable Internal Web Service

Background. A college network contains student laptops, access switches, a router and an internal learning server. Students report that the server’s IP address responds to ping, but the learning website does not open.

Problem. The technician must determine whether the failure is physical, link-layer, network-layer, transport-layer or application-layer.

Analysis. Link lights are active and the switch shows the laptop’s MAC address, making a basic physical or local switching failure less likely. The laptop can ping the server, so IP addressing and a route exist. A packet capture shows TCP SYN packets from an ephemeral client port to server port 443, but no SYN-ACK returns. A scan performed under authorization shows that the server is listening on port 80 but not port 443.

Approach and solution. The technician confirms the intended service configuration, enables HTTPS on the approved server, installs a valid certificate and verifies that the firewall permits TCP 443 from the student network. A new capture shows SYN, SYN-ACK and ACK, followed by encrypted application traffic.

Outcome. The website becomes reachable. The investigation succeeded because the technician separated layers instead of replacing network equipment immediately.

Learning points. Ping tests reachability using ICMP; it does not prove that a TCP service is listening. A functioning network layer can coexist with a failed transport service. Packet captures reveal the progression from attempted connection to established session, while authorization and change control are essential when testing or modifying systems.

13. Limitations, Security and Professional Practice

Models simplify reality. A firewall may operate at several layers, a VPN adds headers and encryption, and NAT modifies addressing between network regions. Therefore, technicians should use models as reasoning tools rather than rigid classifications.

Security controls also depend on protocol information. Firewalls filter addresses, ports, protocols and sometimes application content. Encryption protects confidentiality and integrity but can reduce visibility for troubleshooting and security monitoring. Administrators must balance inspection with privacy, legal requirements and data-minimization principles.

Network testing must be authorized. Port scanning, packet capture and interception can expose credentials or personal information and may violate policy or law. Professionals should obtain permission, capture only necessary traffic, protect logs, document changes and use least-privilege access.

Contemporary protocols increasingly use encryption, multiplexing and user-space transports. HTTP/3 uses QUIC, which runs over UDP while providing reliable streams, encryption and connection-management features above UDP. This illustrates an important principle: protocol functions do not always correspond neatly to traditional layers. The layered models remain useful, but modern implementations may combine, bypass or extend their functions.

Best Practices for Protocol Analysis

• Start with the physical link and progress upward through the layers.
• Record source and destination addresses, ports and protocol numbers.
• Distinguish local MAC addresses from end-to-end IP addresses.
• Check whether the service is listening on the expected port.
• Use packet captures to compare both directions of a conversation.
• Account for MTU, tunnelling, NAT, firewalls and encryption.
• Test with authorized tools and protect captured data.
• Document evidence before changing configuration.

Chapter Summary

Protocol models divide network communication into manageable functions. The OSI model has seven layers, while the TCP/IP model groups those functions into a practical Internet-oriented suite. Data is encapsulated as it moves downward: application data becomes a transport segment or datagram, an IP packet and a link-layer frame. At the destination, decapsulation reverses the process.

Switches primarily forward frames using MAC addresses, whereas routers forward IP packets between networks and create a new frame for each link. Transport ports identify application processes. TCP is connection-oriented and provides ordered, reliable byte-stream service; UDP is connectionless and provides lightweight datagrams without built-in delivery guarantees. Headers, flags, addresses and ports allow technicians to trace communication and isolate faults. Modern technologies such as QUIC demonstrate that models are analytical frameworks rather than inflexible descriptions of implementation.

Key Terms

Protocol: A defined set of communication rules.

Protocol model: A layered framework for organizing network functions.

OSI model: A seven-layer reference model standardized by ISO.

TCP/IP model: A practical model based on the Internet protocol suite.

Encapsulation: Adding layer-specific control information around a higher-layer PDU.

Decapsulation: Removing protocol information as data moves upward at the destination.

Header: Control information placed before a payload.

Trailer: Control information placed after a payload.

PDU: Protocol data unit defined at a particular layer.

Frame: A data-link-layer PDU.

Packet: A network-layer PDU, commonly an IP packet.

Segment: A TCP transport-layer PDU.

Datagram: An independent data unit, commonly associated with UDP.

MTU: Maximum transmission unit carried by a link without link-specific fragmentation.

Port: A transport-layer identifier for an application service.

Socket: An endpoint identified using addressing and transport information.

TCP: Reliable, ordered, connection-oriented transport protocol.

UDP: Connectionless transport protocol with low overhead.

Routing: Selecting a path or next hop for an IP packet.

Multiplexing: Combining traffic from multiple applications for transport.

Review Questions

Short Answer Questions

1. What is the main purpose of a layered protocol model?
2. State two differences between a switch and a router.
3. Define encapsulation and decapsulation.
4. What information does a transport-layer port identify?
5. Name two services provided by TCP that UDP does not provide by itself.

Long Answer Questions

1. Explain the seven OSI layers and give one example of a protocol or technology associated with each.
2. Trace an HTTPS request from a client application to a remote server, identifying the PDU at each major layer.
3. Compare the OSI and TCP/IP models, including their purposes and limitations.
4. Explain how a router forwards a packet and why link-layer addresses usually change at every routed hop.
5. Discuss the practical differences between TCP and UDP and recommend suitable uses for each.

Analytical / Application Questions

1. A host can ping a server but cannot connect to TCP port 443. Identify at least three possible causes and describe the evidence you would collect to distinguish them.
2. A packet capture shows that a TCP SYN leaves a client, but no SYN-ACK returns. Analyze possible faults at the client, network, firewall and server layers.
3. A 1500-byte MTU path uses 20-byte IPv4 and 20-byte TCP headers with no options. Calculate the TCP payload size and explain how TCP options or tunnelling could change the result.

Multiple Choice Questions with Answers

Question 1

Which OSI layer is primarily responsible for logical addressing and forwarding between networks?

A. Physical

B. Data Link

C. Network

D. Transport

Correct Answer: C. Network

Question 2

What is the usual name for a TCP protocol data unit?

A. Frame

B. Segment

C. Symbol

D. Message digest

Correct Answer: B. Segment

Question 3

Which address normally changes at each routed link?

A. The destination application port

B. The end-to-end destination IP address

C. The link-layer MAC address

D. The TCP sequence number only

Correct Answer: C. The link-layer MAC address

Question 4

What is the main purpose of a transport-layer port number?

A. To identify a physical cable

B. To identify an application process or service

C. To replace an IP address

D. To calculate a frame check sequence

Correct Answer: B. To identify an application process or service

Question 5

Which property is provided by TCP but not by UDP itself?

A. IP addressing

B. Radio modulation

C. Ordered reliable delivery

D. Ethernet framing

Correct Answer: C. Ordered reliable delivery

Question 6

A link has an MTU of 1500 bytes, with a 20-byte IPv4 header and a 20-byte TCP header. What TCP payload size fits without additional options?

A. 1400 bytes

B. 1460 bytes

C. 1480 bytes

D. 1500 bytes

Correct Answer: B. 1460 bytes

Question 7

Which device primarily uses a MAC address table to forward local network frames?

A. Switch

B. Router

C. DNS resolver

D. Web server

Correct Answer: A. Switch

Question 8

What occurs during decapsulation at a receiving host?

A. Each layer adds a new physical cable

B. Headers and trailers are processed and removed as data moves upward

C. All IP addresses are replaced by port numbers

D. The application data is converted into radio waves

Correct Answer: B. Headers and trailers are processed and removed as data moves upward

Question 9

Which statement best describes UDP?

A. It requires a three-way handshake for every exchange

B. It provides an ordered byte stream with retransmission

C. It sends independent datagrams without built-in general delivery guarantees

D. It operates only inside Ethernet switches

Correct Answer: C. It sends independent datagrams without built-in general delivery guarantees

Question 10

A successful ping does not prove that a web service is available because ping normally tests:

A. TCP port 443 only

B. ICMP reachability rather than the web service port

C. The server’s application certificate

D. The Ethernet cable’s maximum speed

Correct Answer: B. ICMP reachability rather than the web service port