Web Foundations, Standards and Architecture

Introduction

The modern web is a global software platform built on interoperable networks, standardized protocols, distributed computing, and human-oriented interfaces. A web application is not merely a collection of pages. It is an ecosystem in which browsers, application servers, databases, content-delivery networks, identity systems, and network protocols cooperate across administrative boundaries.

Understanding these foundations is essential for postgraduate IT engineers because architectural decisions made at this level influence performance, security, accessibility, maintainability, scalability, and compliance. Frameworks and programming languages change frequently, but the principles of addressing, communication, rendering, state management, caching, and standards remain fundamental.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the evolution of the Internet and the World Wide Web.

2. Distinguish client–server, peer-to-peer, and distributed architectures.

3. Interpret URLs, DNS resolution, HTTP methods, status codes, and headers.

4. Describe the complete request–response lifecycle.

5. Explain the roles of browsers, web servers, application servers, and rendering engines.

6. Analyze cookies, sessions, caching, and content delivery.

7. Explain HTTPS and the security properties supplied by TLS.

8. Evaluate web standards and progressive enhancement as engineering strategies.

9. Identify contemporary architectural, ethical, accessibility, and governance concerns.

1. From the Internet to the Web

The Internet is a worldwide interconnection of networks that communicate using the Internet Protocol suite. It provides the underlying transport infrastructure. The World Wide Web is one application that operates over the Internet, using resources identified by URLs and transferred primarily through HTTP.

The Internet developed from packet-switching research and early wide-area networks. The important architectural idea was that data could be divided into packets and routed independently, allowing networks to interconnect without requiring a single central controlling system. TCP provided reliable, ordered delivery for applications that required it, while IP provided addressing and routing.

The Web was proposed by Tim Berners-Lee at CERN in 1989. Its original conceptual foundations were:

• a uniform way to identify resources;
• hypertext links between resources; and
• a protocol for transferring resources.

These ideas became URLs, HTML, and HTTP. Early websites were largely static documents. Later developments introduced server-side scripting, databases, browser scripting, asynchronous requests, mobile web applications, cloud platforms, APIs, and highly distributed services.

A useful distinction is therefore:

• Internet: the network infrastructure and protocols connecting devices.
• Web: a resource and application system operating over that infrastructure.
• Web application: software delivered through web technologies, often combining browser code, servers, APIs, and data stores.

The Web has evolved from document delivery toward interactive, data-intensive, and increasingly real-time systems. Nevertheless, compatibility remains a central design objective because users access services through diverse devices, browsers, networks, and assistive technologies.

2. Architectural Models

2.1 Client–server architecture

In a client–server model, a client requests a service and a server provides it. A browser is commonly the client, while a web server or application server processes requests. The roles are logical rather than necessarily physical: one machine may act as both client and server for different interactions.

The model offers centralized control of business rules, data, authorization, and deployment. Its limitations include server bottlenecks, network dependence, and the need to manage failures across the connection.

A typical web request passes through several layers:

1. The user agent creates a request.

2. DNS maps a domain name to an address.

3. Network infrastructure routes packets.

4. A front-end server or proxy receives the request.

5. Application logic executes.

6. Data may be read from a database or cache.

7. A response returns to the browser.

8. The browser parses and renders the result.

2.2 Distributed systems

A distributed system consists of components located on multiple networked computers that cooperate to provide a service. Web platforms commonly distribute responsibilities among reverse proxies, load balancers, application services, databases, queues, object stores, and content-delivery networks.

Distribution can improve availability, scalability, and geographic performance. It also introduces partial failure, network latency, consistency problems, observability requirements, and more complex security boundaries. A distributed architecture should not be adopted merely because it is fashionable; its operational cost must be justified by requirements.

2.3 Peer-to-peer and hybrid models

In peer-to-peer systems, participants can both request and provide resources. Content-distribution and communication applications may use hybrid designs in which centralized services provide identity and coordination while peers exchange data directly. Most commercial web applications are predominantly client–server or distributed client–server systems rather than purely peer-to-peer systems.

2.4 Layering and separation of concerns

Web engineering benefits from separating concerns. Presentation, application logic, data access, transport security, and infrastructure operations should have clearly defined responsibilities. Layering reduces coupling and allows one layer to evolve without rewriting the entire system. However, excessive layering can increase latency and make debugging difficult. Good architecture balances modularity with operational simplicity.

3. URLs, URIs, and DNS

A Uniform Resource Identifier (URI) identifies a resource. A URL is a URI that also indicates how the resource can be located or accessed. A typical URL is:

`https://shop.example.com:443/catalog/item?id=42#reviews`

Its components are:

• Scheme: `https`, indicating the access protocol.
• Host: `shop.example.com`, the domain name.
• Port: `443`, explicitly specified here; HTTPS commonly uses this default.
• Path: `/catalog/item`, identifying a resource path.
• Query: `id=42`, supplying parameters to the resource or application.
• Fragment: `reviews`, identifying a location within the retrieved representation. The fragment is normally processed by the client and is not sent in the HTTP request.

URLs should be designed for stability, clarity, and security. Sensitive information should not be placed in query strings because URLs may be stored in browser history, logs, analytics systems, or referrer data.

DNS resolution

The Domain Name System translates human-readable names into resource records, commonly including IP addresses. Resolution may involve the browser cache, operating-system cache, recursive resolver, root servers, top-level-domain servers, and authoritative name servers.

DNS is hierarchical and distributed. A domain name such as `www.example.com` is interpreted from right to left: the root, the `.com` top-level domain, the `example` domain, and the `www` host label. DNS records include A records for IPv4 addresses, AAAA records for IPv6 addresses, CNAME aliases, MX records for mail routing, and TXT records used for various verification and policy purposes.

DNS caching reduces lookup traffic and latency. A record's time to live (TTL) indicates how long a resolver may cache it, although actual behavior also depends on resolver policy. DNS is not itself a confidentiality mechanism; ordinary DNS queries can reveal requested domains unless protected by technologies such as DNS over HTTPS or DNS over TLS.

4. HTTP and HTTPS

HTTP is an application-layer protocol for transferring representations of resources. It follows a request–response model. HTTP is stateless at the protocol level: each request is independent unless applications add state using cookies, tokens, or other mechanisms.

An HTTP request generally contains:

• a method, such as GET or POST;
• a target path and protocol version;
• request headers; and
• an optional message body.

An HTTP response contains:

• a status code;
• response headers; and
• an optional representation body.

Methods

• GET: retrieves a representation and should be safe and idempotent.
• HEAD: requests headers equivalent to those of GET without the response body.
• POST: submits data for processing; repeated requests may create multiple effects.
• PUT: creates or replaces a representation at a specified target and is generally idempotent.
• PATCH: applies a partial modification; idempotence depends on the operation.
• DELETE: requests removal and is generally intended to be idempotent, although the outcome may depend on implementation.

“Safe” means that the method is intended not to change server state as a consequence of the request. “Idempotent” means that making the same request multiple times has the same intended effect as making it once; it does not mean that every response is identical.

Status codes

Status codes are grouped as follows:

• 1xx: informational;
• 2xx: successful, such as 200 OK and 201 Created;
• 3xx: redirection or conditional behavior, such as 301 and 304;
• 4xx: client-side error, such as 400, 401, 403, and 404;
• 5xx: server-side failure, such as 500 and 503.

A 401 response concerns missing or invalid authentication credentials, whereas 403 indicates that the server understood the request but refuses authorization. Confusing these meanings can produce poor client behavior and inaccurate monitoring.

HTTPS and TLS

HTTPS is HTTP transported over Transport Layer Security (TLS). TLS provides:

1. Confidentiality: outsiders should not read protected traffic.

2. Integrity: unauthorized modification should be detectable.

3. Server authentication: certificates help the client verify the server's identity.

TLS does not automatically make an application secure. It cannot prevent SQL injection, insecure authorization logic, malicious scripts inserted by the application, or abuse by an authenticated user. Proper certificate validation, secure key management, modern protocol configurations, and correct application design remain necessary.

5. The Request–Response Lifecycle

Consider a user navigating to an HTTPS page. The lifecycle may include the following stages:

1. URL interpretation: the browser parses the scheme, host, port, path, query, and fragment.

2. Cache examination: the browser checks whether a fresh usable response exists locally.

3. DNS resolution: the hostname is resolved, possibly using cached data.

4. Connection establishment: a TCP connection may be created; HTTP/3 can use QUIC over UDP instead. For HTTPS, a TLS handshake establishes cryptographic parameters and authenticates the server.

5. Request transmission: the browser sends method, target, headers, and possibly a body.

6. Intermediary processing: a proxy, firewall, CDN, or load balancer may inspect or route the request.

7. Server processing: application code authenticates the user, validates input, applies business rules, accesses data, and creates a response.

8. Response transfer: the response travels back, potentially through caches or compression layers.

9. Parsing: the browser parses HTML and discovers stylesheets, scripts, images, fonts, and other resources.

10. Rendering: the browser constructs internal representations and paints pixels.

11. Subsequent activity: scripts may issue asynchronous requests, update the document, or respond to user interaction.

The lifecycle is not necessarily linear. Resources can be requested concurrently, responses can be streamed, and service workers can intercept requests. Performance analysis therefore requires measurements such as DNS time, connection time, time to first byte, transfer duration, script execution time, and rendering milestones rather than a single vague notion of “page speed.”

6. Browsers, Servers, and Rendering

A browser is a complex execution environment. Major components include a networking stack, storage subsystem, security policy enforcement, JavaScript engine, layout engine, and rendering pipeline.

The browser typically parses HTML into a Document Object Model (DOM). CSS is parsed into a CSS object model. These structures contribute to a render tree, after which layout determines geometry and painting produces visual output. Compositing may move layers efficiently to the graphics processor.

The exact implementation differs among browser engines, but common performance principles include:

• minimize render-blocking resources;
• send appropriately sized images and scripts;
• avoid unnecessary layout recalculation;
• defer nonessential JavaScript;
• use semantic HTML and accessible controls; and
• measure real-user performance rather than relying only on laboratory tests.

A web server such as Nginx or Apache can serve static files, terminate TLS, apply routing rules, and proxy requests. An application server executes business logic, while a database manages persistent data. These roles may be combined in small systems or separated at scale.

7. State, Cookies, and Sessions

HTTP itself does not remember previous requests. Applications introduce state using several approaches.

A cookie is a small value associated with a domain and path that a browser can send with subsequent requests. Cookies may support preferences, analytics, or authentication sessions. Important attributes include:

• Secure: send the cookie only over HTTPS.
• HttpOnly: prevent ordinary client-side scripts from reading it.
• SameSite: control cross-site sending behavior and reduce some cross-site request risks.
• Expires or Max-Age: control persistence.
• Domain and Path: limit scope.

A session commonly stores server-side state indexed by a random session identifier held in a cookie. The identifier must be unpredictable, rotated after authentication where appropriate, protected from theft, and invalidated during logout or expiration. Session storage may be centralized, replicated, or placed in a distributed store.

Alternatives include stateless signed tokens, client-side storage, and explicit parameters. Each has trade-offs. Stateless tokens can reduce server-side session lookup but may be difficult to revoke and can become large. Browser storage is accessible to scripts and must not be treated as a secure vault for secrets. Authentication design should apply least privilege, secure transport, strong credential handling, and appropriate expiration.

8. Caching and Content Delivery

Caching stores a response for possible reuse. It can reduce latency, bandwidth, server load, and cost. Common cache locations include the browser, an institutional proxy, a CDN, a reverse proxy, and an application-level cache.

HTTP caching uses headers such as `Cache-Control`, `ETag`, `Last-Modified`, and `Vary`. For example, a client may send an ETag received earlier in an `If-None-Match` request header. If the representation has not changed, the server can return `304 Not Modified`, allowing the client to reuse its stored body.

Caching is safe only when the response's reuse conditions are understood. Personalized or confidential responses require careful cache directives. A common deployment strategy is content fingerprinting: a file such as `app.8f31c.js` receives a long freshness period because a content change produces a new filename. HTML documents often receive shorter caching periods because they refer to changing assets.

A CDN places content near users geographically. It is particularly effective for static assets and cacheable responses. Dynamic, personalized, or rapidly changing data may require revalidation, bypass rules, or edge-compute logic. Cache invalidation remains difficult because multiple layers may hold different versions.

9. Standards and Interoperability

Web standards allow independent vendors and developers to build compatible systems. The World Wide Web Consortium (W3C), WHATWG, IETF, and related organizations maintain specifications and recommendations covering HTML, CSS, HTTP, URLs, accessibility, and security.

Standards-based development involves more than using familiar tags. It includes:

• valid and meaningful document structure;
• semantic HTML;
• standards-compliant CSS;
• accessible names, labels, keyboard operation, and focus behavior;
• correct character encoding, normally UTF-8;
• responsive design for different screens and input methods;
• secure protocol and policy configuration; and
• testing across relevant browsers and assistive technologies.

Validation tools can detect structural problems, but conformance alone does not guarantee usability. A technically valid page may still be confusing, slow, inaccessible, or insecure. Engineering quality must therefore be evaluated through standards, testing, user research, security review, and operational monitoring.

10. Progressive Enhancement

Progressive enhancement begins with a usable, semantic, accessible core and adds richer presentation and behavior where the environment supports them. Its related strategy, graceful degradation, begins with a feature-rich design and attempts to make it work in less capable environments. Progressive enhancement is often more robust because it treats advanced capabilities as additions rather than prerequisites.

A practical example is a search form implemented with a normal HTML form action. JavaScript can enhance it with instant suggestions and asynchronous results, but users can still submit the form if scripts fail, are blocked, or are unavailable. Similarly, CSS can improve layout without making essential information inaccessible.

Progressive enhancement does not mean refusing modern technologies. It means identifying the minimum functional requirement, detecting capabilities rather than assuming them, providing meaningful fallback behavior, and preserving accessibility. This approach supports resilience under slow networks, browser limitations, content blockers, and assistive technology use.

11. Security, Ethics, and Governance

Web systems process identity data, financial information, health records, communications, and behavioral traces. Responsible engineering therefore includes privacy by design, data minimization, transparent consent practices, retention controls, access governance, and breach response planning.

Common technical risks include cross-site scripting, cross-site request forgery, injection, broken access control, insecure direct object references, weak session management, and supply-chain compromise. Defenses include output encoding, parameterized queries, CSRF protections where relevant, authorization checks on every protected operation, secure cookie attributes, content security policies, dependency management, logging, and security testing.

Accessibility is both an ethical responsibility and, in many jurisdictions, a legal obligation. WCAG principles emphasize that content should be perceivable, operable, understandable, and robust. Privacy and accessibility should be considered during architecture rather than added after implementation.

AI-enabled web features introduce additional concerns: personal-data exposure, opaque automated decisions, prompt injection, generated misinformation, model bias, and uncertain provenance. AI should be integrated with human oversight, access controls, evaluation datasets, auditability, and clear disclosure where appropriate.

12. Contemporary Developments

Current web architecture includes HTTP/2 multiplexing, HTTP/3 over QUIC, service workers, web components, edge computing, server-side rendering, static generation, and hybrid rendering. These approaches address different requirements:

• Server-side rendering can improve initial content availability and search indexing.
• Client-side rendering supports rich interaction after JavaScript loads.
• Static generation provides efficient, highly cacheable output for suitable content.
• Hybrid rendering selects a strategy per page or request.
• Service workers can support offline behavior, caching, and background tasks, but require careful update and security management.
• Edge computing moves selected processing closer to users, potentially reducing latency while increasing deployment complexity.

No approach is universally superior. Selection should reflect content volatility, interaction requirements, device constraints, operational skills, security needs, and measurable user outcomes.

Applied Case: Illustrative Online University Portal

Background

An illustrative university is replacing a portal used for course registration, fee payment, and academic records. The existing system is a single server that becomes slow during registration periods and frequently loses user sessions.

Problem analysis

The bottleneck is not necessarily the database alone. A complete analysis examines DNS, TLS negotiation, application processing, database queries, session storage, static assets, and cache behavior. Session loss may result from in-memory sessions on multiple servers, while slow pages may be caused by unoptimized queries and large client-side bundles.

Approach

The university introduces a load balancer and multiple application instances. Sessions are stored in a shared, fault-tolerant store, and authentication cookies are marked Secure, HttpOnly, and appropriately SameSite. Static assets are served through a CDN using content fingerprints. Registration operations use validated idempotency strategies where repeated submissions must not create duplicate transactions. Database indexes and query plans are reviewed, while rate limits protect authentication and registration endpoints.

The portal uses semantic HTML and server-generated core forms. JavaScript enhances timetable filtering and live seat availability but does not remove the basic registration workflow. Monitoring records latency percentiles, error rates, cache effectiveness, database saturation, and authentication failures without unnecessarily collecting personal data.

Outcome

The architecture improves resilience during peak demand, reduces repeated downloads, and makes the service more usable on mobile devices and slower networks. However, the university must continue capacity testing, dependency patching, privacy reviews, accessibility testing, and incident rehearsals. Distribution solves some bottlenecks but creates new operational responsibilities.

Learning points

1. Performance is an end-to-end property.

2. Shared state must be designed explicitly in a multi-instance system.

3. Caching requires correct freshness and privacy rules.

4. Progressive enhancement improves resilience and accessibility.

5. Technical architecture must be evaluated together with governance and user impact.

Challenges and Limitations

Web architecture involves unavoidable trade-offs. Strong caching may conflict with content freshness. Stateless designs can simplify scaling but complicate revocation. Encryption protects network traffic but does not correct authorization errors. Microservices can support independent deployment but increase operational and debugging overhead. Progressive enhancement improves robustness but may require additional design and testing effort. Standards improve interoperability, yet browser differences and evolving specifications still require practical compatibility testing.

Engineering decisions should therefore be evidence-based. Teams should define service objectives, threat models, accessibility requirements, data classifications, and performance budgets before selecting technologies. Measurement, review, and iterative improvement are more reliable than assuming that a particular framework or architecture guarantees quality.

Best-Practice Checklist

• Use HTTPS for all application traffic.
• Design stable, meaningful URLs.
• Validate and authorize all input and operations on the server.
• Use semantic, accessible HTML as the functional foundation.
• Configure cookies with appropriate Secure, HttpOnly, and SameSite attributes.
• Apply explicit cache-control policies to public and personalized content.
• Use content fingerprinting for versioned static assets.
• Measure real-user performance and monitor failures.
• Keep dependencies and TLS configurations current.
• Minimize collected personal data and define retention policies.
• Test across browsers, devices, networks, and assistive technologies.
• Prefer architectural complexity only when requirements justify it.

Chapter Summary

The Internet supplies the network foundation on which the World Wide Web operates. Web applications use URLs, DNS, HTTP, TLS, browsers, servers, and distributed infrastructure to deliver resources and services. Client–server and distributed models provide useful ways to allocate responsibilities, but distribution introduces latency, partial failure, consistency, and operational challenges.

The request–response lifecycle includes URL parsing, cache checks, DNS resolution, connection establishment, TLS negotiation, server processing, response transfer, parsing, and rendering. HTTP is stateless, so applications add state through cookies, sessions, tokens, and storage mechanisms. Caching and CDNs improve performance when freshness, privacy, and invalidation are carefully controlled.

Standards promote interoperability, while semantic HTML, accessibility, responsive design, and progressive enhancement improve resilience and inclusion. HTTPS supplies confidentiality, integrity, and server authentication but does not replace secure application logic. Contemporary architectures include HTTP/3, service workers, edge computing, and hybrid rendering; each must be selected according to measurable requirements. Professional web engineering integrates technical design with security, privacy, accessibility, governance, and continuous operational evaluation.

Key Terms

Internet: Global interconnected network using Internet Protocols.

World Wide Web: Hyperlinked resource system operating over the Internet.

Client: Component that requests a service or resource.

Server: Component that provides a service or resource.

Distributed system: Cooperative system whose components run on multiple networked computers.

URI: Identifier for a resource.

URL: URI that identifies a resource and indicates how it may be accessed.

DNS: Hierarchical system that maps names to network-related records.

HTTP: Application protocol using requests and responses to transfer representations.

HTTPS: HTTP protected by Transport Layer Security.

TLS: Protocol providing confidentiality, integrity, and authenticated connections.

Cookie: Browser-managed value sent with matching requests.

Session: Application state associated with a user or interaction period.

Cache: Storage of reusable responses or data.

CDN: Distributed network that delivers content near users.

DOM: Browser representation of an HTML document as a document tree.

Progressive enhancement: Building a functional core and adding richer capabilities progressively.

Idempotent: Having the same intended effect when repeated.

Semantic HTML: HTML whose elements convey structure and meaning.

Content Security Policy: Browser-enforced policy restricting permitted content sources and behavior.

Review Questions

Short Answer Questions

1. Distinguish the Internet from the World Wide Web.

2. What information is represented by DNS?

3. Explain the difference between a 401 and a 403 HTTP response.

4. State two security attributes of cookies and their purposes.

5. What is progressive enhancement?

Long Answer Questions

1. Explain the complete lifecycle of an HTTPS request from URL entry to browser rendering.

2. Compare client–server and distributed architectures, including their benefits and limitations.

3. Explain HTTP caching, conditional requests, ETags, and CDN use.

4. Discuss how browsers construct and render a web document.

5. Explain why HTTPS is necessary but insufficient for web application security.

Analytical / Application Questions

1. Design an architecture for a university portal expected to experience tenfold demand during registration. Identify likely bottlenecks and justify caching, session, and scaling decisions.

2. A personalized account page is accidentally served from a shared CDN cache. Analyze the failure and propose corrective controls.

3. Design a progressively enhanced course-search interface that remains usable when JavaScript is unavailable and accessible to keyboard and screen-reader users.

Multiple Choice Questions with Answers

Question 1

Which statement most accurately distinguishes the Internet from the Web?

A. The Internet is a browser, while the Web is a database.

B. The Internet is the network infrastructure, while the Web is an application system operating over it.

C. The Internet uses only HTTPS, while the Web uses only FTP.

D. The Internet is limited to private networks, while the Web is limited to public networks.

Correct Answer: B. The Internet is the network infrastructure, while the Web is an application system operating over it.

Question 2

What is the primary role of DNS?

A. To encrypt HTTP response bodies

B. To render HTML documents

C. To map domain names to network-related records such as IP addresses

D. To store browser cookies permanently

Correct Answer: C. To map domain names to network-related records such as IP addresses

Question 3

Which URL component is normally processed by the browser and not sent in the HTTP request?

A. Scheme

B. Host

C. Query

D. Fragment

Correct Answer: D. Fragment

Question 4

Which property is generally associated with the HTTP GET method?

A. It is intended to be safe and idempotent.

B. It must create a new server-side resource.

C. It always includes a request body.

D. It is used only for authentication.

Correct Answer: A. It is intended to be safe and idempotent.

Question 5

What does a 304 Not Modified response normally indicate?

A. The server has permanently deleted the resource.

B. The client should use its cached representation.

C. The request lacks authentication credentials.

D. The server encountered an internal failure.

Correct Answer: B. The client should use its cached representation.

Question 6

Which TLS property helps detect unauthorized modification of transmitted data?

A. Integrity

B. Compression

C. Caching

D. Address resolution

Correct Answer: A. Integrity

Question 7

What is the purpose of the HttpOnly cookie attribute?

A. It forces cookies to be sent over HTTP rather than HTTPS.

B. It prevents the cookie from being sent to the server.

C. It prevents ordinary client-side scripts from reading the cookie.

D. It makes the cookie available to every domain.

Correct Answer: C. It prevents ordinary client-side scripts from reading the cookie.

Question 8

Why is content fingerprinting useful for static web assets?

A. It allows browsers to execute scripts without parsing them.

B. It permits long caching while changed content receives a new filename.

C. It removes the need for HTTPS certificates.

D. It converts DNS names into IP addresses.

Correct Answer: B. It permits long caching while changed content receives a new filename.

Question 9

Which statement best describes progressive enhancement?

A. Designing only for the newest browser capabilities

B. Starting with a usable core and adding richer features when supported

C. Disabling CSS to improve accessibility

D. Replacing all server processing with client-side scripting

Correct Answer: B. Starting with a usable core and adding richer features when supported

Question 10

Which limitation remains even when a web application uses HTTPS?

A. Network eavesdroppers can always read the encrypted body.

B. HTTPS automatically prevents all authorization errors.

C. HTTPS does not by itself prevent flaws such as injection or broken access control.

D. HTTPS prevents browsers from storing any cookies.

Correct Answer: C. HTTPS does not by itself prevent flaws such as injection or broken access control.