Secure Internet Protocols and Services

Introduction

Internet services are delivered through protocols: agreed rules that define how systems identify one another, exchange messages, authenticate users, and recover from errors. A protocol may provide excellent functionality while offering little security. For example, traditional DNS provides name resolution but does not normally authenticate the origin of a response; HTTP provides web access but sends application data in clear text; Telnet permits remote administration but exposes credentials and commands to interception.

Secure protocol design therefore requires more than replacing one port number with another. It involves confidentiality, integrity, authentication, availability, accountability, secure configuration, certificate or key management, and careful control of trust relationships. This chapter examines the principal Internet and network services used in organisations and compares their insecure and secure alternatives.

Learning Objectives

By the end of this chapter, a learner should be able to:

• explain the security purpose and limitations of DNS, DHCP, email, web, file-transfer, remote-access, and management protocols;
• distinguish insecure protocols from TLS, HTTPS, SSH, IPsec, DNSSEC, SNMPv3, and secure email alternatives;
• identify threats such as interception, spoofing, downgrade, replay, poisoning, credential theft, and unauthorised administration;
• select appropriate protective controls for a small or medium-sized organisation;
• diagnose protocol-level weaknesses using packet captures, logs, configuration review, and controlled testing;
• discuss operational, ethical, governance, and regulatory responsibilities associated with secure services.

1. Security Properties and Protocol Threats

A secure service should be assessed against several properties:

Common protocol-level attacks include passive eavesdropping, active modification, impersonation, replay, downgrade attacks, credential stuffing, cache poisoning, and denial of service. Encryption alone does not solve every problem. A connection may be encrypted but still reach a fraudulent server, expose metadata, accept weak algorithms, or permit an attacker who has stolen a valid credential.

A useful professional question is: What is protected, from whom, and at which point in the communication path? For example, TLS may protect a browser-to-web-server connection, but it does not automatically protect data stored on the server, traffic between internal services, or a message after it has been decrypted by the recipient.

2. DNS: Naming, Resolution, and Trust

The Domain Name System translates names such as portal.example.org into IP addresses and other records. DNS uses a distributed hierarchy: root servers delegate to top-level domains, which delegate to authoritative servers. Clients normally query a recursive resolver, which obtains answers on their behalf and caches them for a period controlled by the record’s time to live (TTL).

Traditional DNS commonly uses UDP port 53 and may use TCP port 53 for large responses, zone transfers, or reliability requirements. Classic DNS does not inherently authenticate responses. An attacker able to interfere with traffic may inject a false answer, redirecting users to a malicious address. Cache poisoning is especially serious because a forged response can affect many clients until the cache entry expires.

DNS security controls

• DNSSEC adds origin authentication and integrity to DNS data. A zone signs records with digital signatures, and a validating resolver follows a chain of trust from a configured trust anchor. DNSSEC does not encrypt DNS queries and does not hide the queried domain.
• Secure zone transfers should use controlled primary and secondary servers, access restrictions, and authenticated transfer mechanisms such as TSIG where appropriate.
• Split-horizon DNS provides different answers for internal and external users, but must be designed carefully to avoid information leakage.
• Resolver hardening includes restricting recursion to authorised networks, patching software, minimising version disclosure, and applying response-rate controls.
• Encrypted DNS transport, such as DNS over TLS or DNS over HTTPS, protects the connection between a client and a resolver from local observation. It does not replace DNSSEC, because encrypted transport and data-origin authentication solve different problems.

DNSSEC validation failure should generally produce a failure rather than silently accepting an unsigned or invalid answer, although deployment must account for legacy systems and operational errors. Administrators should monitor expired signatures, broken delegation, incorrect DS records, and excessive query anomalies.

3. DHCP: Automatic Configuration and Local Trust

The Dynamic Host Configuration Protocol supplies parameters such as an IP address, subnet mask, default gateway, DNS resolver, and lease duration. DHCP normally uses UDP ports 67 and 68. A client initially broadcasts because it does not yet know its network address or DHCP server.

The protocol assumes a trusted local network. A rogue DHCP server can offer a malicious gateway or DNS resolver, enabling traffic interception or redirection. A DHCP starvation attack consumes the available address pool, while spoofed DHCP messages can disrupt clients. DHCP itself does not provide strong end-to-end authentication for ordinary enterprise deployments.

Important controls include DHCP snooping on managed switches, trusted uplink configuration, rate limiting, port security, segmentation, monitoring for unauthorised servers, and carefully controlled reservations. Dynamic ARP Inspection and IP Source Guard can use DHCP snooping information to reduce related local-network attacks. Administrators must recognise that these controls are switch features, not properties of DHCP itself.

4. Email Security

Email involves several protocols and trust boundaries. SMTP transfers messages between mail servers and from clients to submission services. POP3 and IMAP allow clients to retrieve messages. Historically, these protocols could expose credentials and message contents in clear text.

Modern deployments use authenticated submission, commonly with TLS, and secure retrieval through IMAPS or POP3S, or through explicit TLS negotiation. TLS protects a connection while it is active; it does not guarantee that every server in a multi-hop delivery path used encryption. SMTP transport encryption is therefore not equivalent to end-to-end message encryption.

Secure email mechanisms

• SPF publishes authorised sending hosts for a domain. It helps receivers detect unauthorised sources but does not, by itself, sign a message or prevent all spoofing.
• DKIM adds a cryptographic signature to selected message headers and content. A recipient retrieves the public key from DNS and verifies whether the signed content was altered and whether an authorised domain signed it.
• DMARC allows a domain owner to state how receivers should evaluate alignment between the visible From domain and SPF or DKIM results. Policies may request monitoring, quarantine, or rejection.
• S/MIME and OpenPGP can provide message-level encryption and digital signatures. They require reliable key distribution, identity verification, lifecycle management, and recovery procedures.

Email security is difficult because users trust visible names, forwarding can break authentication alignment, mailing lists may modify content, and social engineering often bypasses technical controls. Organisations should combine authentication standards with attachment scanning, URL analysis, phishing-resistant authentication, user reporting, and clear incident procedures.

5. Web Services: HTTP, HTTPS, and TLS

HTTP is an application protocol for requesting and transferring web resources. Plain HTTP sends requests, cookies, credentials, and responses without cryptographic protection. An attacker on the path may read or modify them, inject scripts, or redirect users.

HTTPS is HTTP carried over TLS. TLS provides an authenticated, encrypted, and integrity-protected channel when correctly configured. During a typical TLS handshake, the client and server negotiate protocol parameters, the server presents a certificate, the client validates the certificate chain and hostname, and both derive session keys. Modern TLS versions use authenticated encryption such as AES-GCM or ChaCha20-Poly1305.

TLS does not make a malicious website trustworthy merely because the connection has a padlock. A valid certificate proves control of a name under the certificate authority’s rules, not that the organisation is honest or that the content is safe. TLS also cannot prevent application vulnerabilities such as SQL injection, cross-site scripting, insecure direct object references, or weak authorisation.

Web-service best practice includes:

• redirecting HTTP to HTTPS while avoiding insecure mixed content;
• using certificates with correct names, suitable validity periods, and automated renewal;
• disabling obsolete protocol versions and weak cipher suites;
• enabling HSTS after careful testing;
• protecting cookies with Secure, HttpOnly, and appropriate SameSite attributes;
• applying server-side authorisation and input validation;
• using security headers, logging, patch management, and vulnerability testing;
• separating public, application, and database tiers.

A secure web architecture may use TLS termination at a reverse proxy, but traffic from the proxy to an internal application should also be protected where the network is not fully trusted.

6. File Transfer Services

FTP uses separate control and data connections and traditionally sends credentials and files in clear text. It also creates firewall and passive-mode configuration challenges. Plain TFTP is simpler and commonly used for limited boot or device-transfer functions, but it has no meaningful authentication and should never be exposed unnecessarily.

Secure alternatives include:

• FTPS, which adds TLS to FTP. It can provide confidentiality and authentication but retains FTP’s multiple-channel complexity and requires careful certificate and passive-port management.
• SFTP, which is a file-transfer subsystem over SSH, not “secure FTP.” It normally uses one encrypted SSH connection and supports strong authentication and access controls.
• HTTPS-based transfer, useful for browser or API-driven workflows, provided authentication, authorisation, upload validation, and storage controls are sound.

A secure file-transfer design uses separate accounts or service identities, least privilege, chroot or equivalent isolation where appropriate, malware scanning, quotas, audit logs, retention rules, and controlled key rotation. File names and contents should be treated as untrusted input.

7. Remote Access: Telnet, RDP, VPN, and SSH

Telnet transmits commands and credentials without adequate protection and should be removed from production administration. SSH provides encrypted transport, server authentication, user authentication, integrity, and secure forwarding. Administrators should prefer key-based or certificate-based authentication, protect private keys with passphrases, restrict permitted algorithms, disable unnecessary features, and use multi-factor authentication where supported.

Remote desktop protocols can be useful but are high-value targets. They should not be directly exposed to the public Internet. Use network-level authentication, strong authentication, patching, session controls, logging, and access through a controlled VPN or zero-trust access gateway.

IPsec operates at the network layer and can protect host-to-host or network-to-network traffic. Its principal components are commonly deployed through Internet Key Exchange and authenticated encryption. Tunnel mode encapsulates an entire original IP packet, while transport mode protects the payload of an IP packet. IPsec VPNs are effective for site-to-site connectivity, but configuration errors, excessive trust after connection, weak pre-shared keys, and unpatched gateways remain serious risks.

A VPN is a protected path, not a complete security policy. Once a device is admitted, segmentation, identity-aware authorisation, endpoint security, and monitoring are still required.

8. Network Management: SNMP and Secure Administration

SNMP allows management systems to read counters, receive alerts, and sometimes change device settings. SNMPv1 and SNMPv2c use community strings that function like weak shared secrets and are commonly transmitted without adequate protection. They should not be used for sensitive management across untrusted networks.

SNMPv3 supports security models including authentication and privacy. In practical terms, configure authenticated and encrypted communication, restrict management sources, use views to limit accessible objects, separate monitoring from configuration authority, and rotate credentials. SNMP write access should be disabled unless operationally necessary.

Management interfaces should reside on a dedicated management network or logically isolated plane. Firewalls, jump hosts, role-based access control, command logging, configuration backups, and out-of-band recovery reduce the impact of compromise. NTP should also be secured and monitored because inaccurate time damages certificate validation, log correlation, and incident investigation.

9. Diagnosing Protocol-Level Weaknesses

Diagnosis begins with asset and data-flow mapping. Identify clients, servers, resolvers, gateways, management systems, trust anchors, and external dependencies. Then inspect:

1. Protocol selection: Is Telnet, FTP, HTTP, SNMPv2c, or unauthenticated DNS exposure still present?
2. Transport protection: Is TLS or SSH enabled, correctly negotiated, and validated?
3. Identity: Are certificates, keys, service accounts, and trust stores managed correctly?
4. Authorisation: Does a successful connection provide more access than necessary?
5. Exposure: Is a service reachable from the Internet when it should be internal?
6. Resilience: Are rate limits, redundancy, backups, and recovery procedures tested?
7. Evidence: Are logs sufficient to establish who did what and when?

Packet captures can reveal clear-text credentials, unexpected protocols, certificate errors, DNS responses, retransmissions, and downgrade attempts. Tools such as protocol analyzers, port scanners, configuration checkers, vulnerability scanners, and log-search platforms are useful only when used with authorisation and within a defined scope. A scan must not become an uncontrolled denial-of-service event.

A weakness should be described precisely. “The network is insecure” is not useful; “the Internet-facing server accepts TLS 1.0 and sends session cookies without the Secure attribute” identifies a protocol and configuration problem that can be prioritised and corrected.

10. Illustrative Case Study: Regional Retailer

Background

A fictional regional retailer operates a public online store, internal point-of-sale systems, a mail platform, and 35 branch networks. A review discovers HTTP administration pages, SNMPv2c monitoring, shared SSH passwords, unrestricted recursive DNS, and an exposed RDP service.

Problem and analysis

Packet captures show credentials crossing a branch network in clear text. DNS logs show queries from the public Internet, indicating that the resolver may be abused for reflection or reconnaissance. The shared SSH password means one compromised account could provide access to many devices. The exposed RDP service is subject to continuous password-guessing attempts.

Approach and solution

The retailer creates a prioritised programme:

• HTTP administration is replaced with HTTPS and administrative access is restricted to a management VLAN.
• SNMPv2c is replaced by SNMPv3 with authentication and privacy; write access is removed.
• SSH keys, individual accounts, multi-factor authentication, and a jump host replace shared passwords.
• Recursive DNS is restricted to internal addresses, and DNSSEC validation is enabled.
• RDP is removed from the public Internet and made available only through a hardened access gateway.
• Mail submission requires TLS and authentication; SPF, DKIM, and DMARC are deployed progressively.
• Logs from DNS, VPN, identity, web, and endpoint systems are centralised and reviewed.

Outcome

The exposed attack surface is reduced, authentication becomes attributable to individuals, and incident responders can correlate events. The changes do not eliminate risk: compromised endpoints, phishing, application flaws, and supply-chain threats remain possible. The retailer therefore schedules patching, access reviews, recovery exercises, and periodic configuration testing.

Learning points

Security improvement is usually a coordinated service programme rather than a single product purchase. Encryption, identity, segmentation, least privilege, monitoring, and operational maintenance must reinforce one another.

11. Governance, Ethics, and Contemporary Developments

Secure protocol deployment must respect privacy, proportionality, and law. DNS and email monitoring may reveal sensitive personal information. Organisations should define legitimate purposes, retention periods, access controls, and disclosure procedures. Penetration testing requires written authorisation, rules of engagement, data-handling safeguards, and a responsible reporting process.

Automation and artificial intelligence can assist with certificate expiry detection, anomalous DNS analysis, phishing classification, and log correlation. However, automated decisions may produce false positives, expose confidential data to an external service, or be manipulated by adversarial input. Human review, explainable procedures, data minimisation, and audit trails remain necessary.

Contemporary practice is moving toward encrypted-by-default services, phishing-resistant authentication, automated certificate lifecycle management, software supply-chain signing, zero-trust access, and continuous configuration assessment. These trends improve security only when organisations understand their trust assumptions and maintain reliable recovery processes.

Best-Practice Checklist

• Prefer HTTPS, SSH, SFTP, IPsec, DNSSEC, SNMPv3, and TLS-protected mail services over legacy alternatives.
• Remove or isolate Telnet, FTP, HTTP administration, SNMPv1/v2c, and public RDP.
• Use individual identities, least privilege, multi-factor authentication, and managed keys.
• Validate certificates, restrict trust stores, and automate renewal monitoring.
• Segment user, server, guest, IoT, and management networks.
• Restrict recursive DNS and administrative interfaces to authorised sources.
• Monitor protocol versions, failed authentication, unusual DNS activity, and configuration drift.
• Test backups, incident response, and restoration—not merely connectivity.
• Document exceptions, owners, review dates, and compensating controls.

Chapter Summary

Internet services are secure only when their protocols, configurations, identities, and operating environments are secure together. DNSSEC authenticates DNS data but does not encrypt queries; encrypted DNS protects transport but does not replace DNSSEC. DHCP requires local-network controls such as snooping and segmentation. TLS protects connections, while HTTPS applies TLS to HTTP and still requires secure application design. SPF, DKIM, and DMARC improve email-domain authentication, whereas S/MIME and OpenPGP can provide message-level protection. SFTP and FTPS are safer file-transfer choices than FTP, while SSH replaces Telnet for remote administration. IPsec protects IP traffic and supports VPN designs, but a VPN does not replace authorisation or segmentation. SNMPv3 provides stronger management security than earlier versions. Effective diagnosis combines traffic inspection, configuration review, identity analysis, exposure assessment, and logging. Ethical, legal, and governance controls are essential because monitoring and administration affect people and organisational trust.

Key Terms

DNS: Distributed system that maps domain names to resource records.

DNSSEC: Extensions that provide authentication and integrity for DNS data.

DHCP: Protocol that supplies hosts with network configuration information.

TLS: Cryptographic protocol that protects communication channels.

HTTPS: HTTP transported through TLS.

SMTP: Protocol used primarily to transfer email between mail systems.

SPF: DNS-based declaration of authorised email-sending hosts.

DKIM: Cryptographic signature mechanism for email messages.

DMARC: Email policy framework using domain alignment and authentication results.

FTP: Traditional file-transfer protocol with insecure legacy operation.

SFTP: File-transfer subsystem operating over SSH.

SSH: Secure protocol for remote login, command execution, and tunnelling.

IPsec: Network-layer security architecture for authenticated and encrypted IP traffic.

SNMPv3: Version of SNMP supporting authenticated and private management communication.

Certificate authority: Trusted entity that issues and signs digital certificates.

Cache poisoning: Insertion of false information into a resolver or application cache.

HSTS: Browser policy requiring a site to use HTTPS.

Network segmentation: Division of a network into controlled security zones.

Least privilege: Principle of granting only the access needed for an authorised task.

Review Questions

Short Answer Questions

1. Why does DNSSEC not provide confidentiality for DNS queries?
2. State two security weaknesses of plain FTP.
3. What is the purpose of DKIM in email security?
4. How does SFTP differ conceptually from FTPS?
5. Why should SNMP management traffic be restricted to a management network?

Long Answer Questions

1. Explain the TLS handshake and discuss what TLS does and does not guarantee for a web user.
2. Compare Telnet, SSH, RDP, and IPsec VPNs as remote-access technologies.
3. Describe a layered security design for an organisation’s email service.
4. Explain the threats created by rogue DHCP servers and the controls available on managed switches.
5. Discuss how an administrator should diagnose a suspected protocol-level weakness from initial discovery through remediation.

Analytical / Application Questions

1. A company’s public DNS resolver accepts recursive queries from any Internet address. Analyse the risks and propose a staged remediation plan.
2. A packet capture shows a login over HTTP, while the organisation claims that its website uses HTTPS. Identify possible explanations and specify the evidence needed to verify each one.
3. A branch office requires secure connectivity to headquarters, but compromised branch devices must not obtain unrestricted access to internal servers. Recommend a design using IPsec, identity controls, segmentation, and monitoring, and justify each element.

Multiple Choice Questions with Answers

Question 1

Which security property does DNSSEC primarily provide for DNS records?

A. Confidentiality of DNS queries

B. Origin authentication and integrity

C. Automatic DHCP address assignment

D. Encryption of email attachments

Correct Answer: B. Origin authentication and integrity

Question 2

Which protocol is the most appropriate replacement for Telnet for secure command-line administration?

A. FTP

B. HTTP

C. SSH

D. TFTP

Correct Answer: C. SSH

Question 3

What is the principal purpose of SPF?

A. To encrypt the complete email message end to end

B. To identify authorised sending hosts for a domain

C. To provide secure remote desktop access

D. To sign DNS zone transfers

Correct Answer: B. To identify authorised sending hosts for a domain

Question 4

Which statement about HTTPS is correct?

A. It prevents all web application vulnerabilities

B. It is HTTP protected by TLS

C. It guarantees that the website owner is honest

D. It eliminates the need for authorisation controls

Correct Answer: B. It is HTTP protected by TLS

Question 5

Which technology provides secure file transfer as a subsystem of SSH?

A. SFTP

B. TFTP

C. SNMPv2c

D. POP3

Correct Answer: A. SFTP

Question 6

A rogue DHCP server is most likely to attack which configuration element directly?

A. The client’s default gateway or DNS resolver

B. The server’s TLS certificate chain

C. The email DKIM private key

D. The SSH host-key algorithm

Correct Answer: A. The client’s default gateway or DNS resolver

Question 7

Which SNMP version supports authenticated and private management communication?

A. SNMPv1

B. SNMPv2c

C. SNMPv3

D. SNMP over TFTP

Correct Answer: C. SNMPv3

Question 8

What does DMARC primarily help a receiving mail system determine?

A. Whether a message’s domain authentication aligns with the visible From domain

B. Whether an IPsec tunnel is using tunnel mode

C. Whether an FTP server permits passive connections

D. Whether a DHCP lease has expired

Correct Answer: A. Whether a message’s domain authentication aligns with the visible From domain

Question 9

Why is a VPN not a complete security policy?

A. A VPN cannot encrypt any traffic

B. A VPN may provide connectivity without limiting what an admitted device can access

C. A VPN operates only with email protocols

D. A VPN automatically disables segmentation

Correct Answer: B. A VPN may provide connectivity without limiting what an admitted device can access

Question 10

Which observation most clearly indicates a clear-text protocol weakness?

A. A certificate is renewed automatically

B. A packet capture contains readable usernames and passwords

C. A DNS record has a short TTL

D. A firewall logs a blocked connection

Correct Answer: B. A packet capture contains readable usernames and passwords