2. IoT Architectures and Distributed Computing

Introduction and Context

An Internet of Things (IoT) system is not merely a collection of connected sensors. It is a distributed information system in which physical objects generate data, communication networks transport that data, computational resources interpret it, and software or human actors use the resulting knowledge to influence the physical world. Architecture determines where these responsibilities are located, how they communicate, and how the system behaves when devices, networks, services, or users fail.

Architecture is therefore a decision about more than technology. It expresses assumptions about latency, reliability, ownership, security, data movement, autonomy, cost, and evolution. A temperature-monitoring application for a university building may tolerate several seconds of delay and use a centralized cloud platform. An industrial safety controller, by contrast, may need a local decision within milliseconds even when the external network is unavailable. Both are IoT applications, but their architectural requirements are fundamentally different.

This chapter examines major architectural approaches for IoT and wireless systems: layered, service-oriented, event-driven, cloud-centric, edge, fog, mist, and serverless architectures. These approaches are not always mutually exclusive. A production system may use a layered conceptual model, service-oriented interfaces, event-driven messaging, edge analytics, fog coordination, and serverless cloud functions simultaneously. The central task is to understand their assumptions and evaluate their trade-offs rather than treating them as competing labels.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Explain the architectural role of sensing, networking, processing, application, and management layers in IoT systems.

2. Distinguish service-oriented, event-driven, cloud-centric, edge, fog, mist, and serverless architectures.

3. Critically compare architectures using scalability, latency, resilience, cost, autonomy, and maintainability.

4. Select an appropriate distributed architecture for a given IoT requirement.

5. Analyse security, governance, operational, and ethical implications of architectural choices.

6. Evaluate hybrid architectures and emerging developments in distributed IoT computing.

Architecture as a Distribution of Responsibility

The most useful way to study IoT architecture is to ask six questions. Where is data generated? Where is it transmitted? Where is it stored? Where is computation performed? Where are decisions enforced? Who controls the infrastructure and the data? Answers to these questions reveal the real architecture even when a vendor uses a different terminology.

IoT systems operate across a continuum. At one end, computation is concentrated in remote cloud data centres. At the other, processing is performed directly on a sensor, actuator, gateway, or embedded controller. Intermediate positions include local edge servers and fog nodes distributed through buildings, factories, cellular networks, or regional facilities. The continuum is not simply geographical; it also concerns administrative control, software deployment, trust, and timing.

A sound architecture separates concerns without ignoring physical constraints. Wireless links may be intermittent, energy-constrained, or congested. Devices may have limited memory and processing capacity. Cloud platforms offer elasticity but introduce dependence on wide-area networks and external providers. Consequently, architectural quality is achieved through deliberate placement of functions, not by maximizing centralization or decentralization.

The Layered View: A Foundation for Reasoning

Layered architecture organizes system functions according to abstraction and responsibility. A commonly used IoT interpretation includes a perception or device layer, a network layer, a processing or platform layer, an application layer, and a management or business layer.

The perception layer contains sensors, actuators, embedded processors, identification mechanisms, and physical interfaces. It interacts with the environment and therefore faces problems such as calibration, energy consumption, physical tampering, noise, and device heterogeneity. The network layer provides connectivity through technologies such as IEEE 802.15.4, Wi-Fi, Bluetooth Low Energy, LPWAN, Ethernet, 5G, and the Internet. It must address routing, addressing, mobility, quality of service, and intermittent connectivity.

The processing or platform layer stores, filters, aggregates, and analyses data. It may be implemented by a gateway, edge server, fog node, cloud platform, or a combination of these. The application layer provides domain functions such as predictive maintenance, environmental monitoring, smart logistics, or healthcare supervision. A management layer cuts across the others, providing identity management, device provisioning, policy enforcement, observability, software updates, billing, and governance.

Layering improves conceptual clarity, interoperability, testing, and maintainability. A communication protocol can sometimes be changed without rewriting application logic. However, strict layering can be problematic in IoT. Energy-aware routing may require knowledge of application priorities; safety decisions may require direct access to local device states; and security policies must cross all layers. Thus, layering is best treated as a model for separation of concerns rather than an instruction that every interaction must pass through a rigid sequence.

Service-Oriented and Event-Driven Architectures

A service-oriented architecture (SOA) represents capabilities as services with defined interfaces. A device-management service, location service, analytics service, or alerting service can be used by multiple applications. Services may communicate through RESTful APIs, message brokers, publish-subscribe protocols such as MQTT, or industrial interfaces such as OPC UA.

SOA promotes reuse, loose coupling, organizational ownership, and independent evolution. It is particularly useful when an enterprise integrates devices from several vendors or supports multiple applications over a common platform. Its weaknesses include interface complexity, service discovery problems, distributed failure modes, network overhead, and the possibility that a nominally modular system becomes a tightly coupled collection of dependencies.

Event-driven architecture organizes computation around events: notifications that something has occurred or that a state has changed. Examples include “motor temperature exceeded threshold,” “vehicle entered zone,” and “occupancy changed.” Producers publish events, while consumers subscribe to those relevant to their work. A broker may buffer, route, transform, or persist events.

Event-driven systems are well suited to asynchronous IoT environments because producers and consumers need not operate simultaneously. They support fan-out, temporal decoupling, and rapid reaction. Yet event semantics require careful design. Engineers must define delivery guarantees, ordering, duplication handling, retention, schemas, and idempotency. Exactly-once processing is difficult in distributed systems; many practical systems use at-least-once delivery and require consumers to make repeated processing harmless.

SOA and event-driven architecture are complementary. A service may expose a request-response API for configuration while also publishing events about changes. A useful design distinguishes commands, which request an action, from events, which record that an action or condition has occurred.

Cloud-Centric IoT Architecture

In a cloud-centric architecture, substantial storage, analytics, device management, and application logic are located in centralized or regional cloud platforms. Devices transmit data through gateways or Internet connectivity, and the cloud provides elastic compute, managed databases, stream processing, digital twins, dashboards, and machine-learning services.

Cloud centralization offers several advantages. Large-scale data can be consolidated for longitudinal analysis, model training, cross-site comparison, and enterprise reporting. Capacity can expand rapidly, and managed services reduce the need to maintain physical infrastructure. Central policy enforcement and centralized observability can also simplify administration.

The costs are equally important. Wide-area transmission may produce latency, bandwidth consumption, and recurring data charges. Connectivity becomes a dependency, and outages can affect operations. Sensitive data may cross organizational or national boundaries. Cloud services can also create provider lock-in through proprietary APIs, data formats, identity systems, and deployment models. Cloud architecture is therefore strongest for workloads that require global aggregation, substantial computation, or delayed analysis, but weaker for immediate control and disconnected operation.

Edge, Fog, and Mist Computing

Edge computing places computation near the data source or the point at which action is required. An industrial gateway may inspect vibration data locally; a camera may perform object detection without sending continuous video to the cloud; a vehicle may make safety decisions using on-board processors. Edge processing reduces latency, bandwidth use, and exposure of raw data. It can also preserve operation during cloud or Internet disruption.

Fog computing extends distributed processing across a hierarchy between devices and the cloud. Fog nodes may be located in access networks, cellular infrastructure, local servers, or regional facilities. They can coordinate multiple edge devices, perform stream aggregation, enforce policy, and support mobility. Fog is especially valuable where local context spans several devices or sites but a centralized cloud is too distant for operational requirements.

Mist computing pushes limited intelligence directly into constrained end devices. A sensor may perform thresholding, compression, feature extraction, or anomaly detection before transmission. Mist computing reduces radio use and energy consumption, although it is constrained by processor capacity, memory, battery life, software-update difficulty, and physical exposure.

The terms edge, fog, and mist are used inconsistently in research and industry. Their practical distinction should be based on placement and responsibility: mist generally refers to computation at or very near the device, edge to computation at the network edge or gateway, and fog to a broader distributed layer between edge devices and cloud resources. These models are often combined in a hierarchy rather than deployed independently.

Serverless Computing in IoT

Serverless computing allows developers to deploy functions or managed services without directly administering servers. A function may be triggered by an incoming message, object creation, database change, or scheduled event. The provider allocates resources dynamically and charges according to execution or service consumption.

For IoT, serverless functions can validate telemetry, transform messages, invoke alerts, update digital twins, and initiate machine-learning workflows. Serverless designs can accelerate development and scale uneven workloads efficiently. They also encourage small, independently deployable components.

However, serverless is not equivalent to edge computing. Most serverless functions execute in provider-managed cloud infrastructure, although increasingly similar models are being offered at the edge. Cold-start delay, execution limits, vendor-specific interfaces, unpredictable cost under event storms, observability difficulties, and state-management challenges must be considered. Serverless functions are generally inappropriate for hard real-time control unless a local runtime with suitable guarantees is used.

A Comparative Evaluation Framework

Architecture selection should be evidence-based. The following criteria provide a practical and research-oriented framework.

Scalability concerns the ability to support more devices, messages, locations, users, and analytical workloads. Cloud and serverless platforms usually offer strong horizontal elasticity, while device-level systems scale physically but may be difficult to manage. Edge and fog architectures distribute load and reduce central bottlenecks, but require coordinated deployment.

Latency includes sensing, transmission, queuing, computation, and actuation delay. Mist and edge arrangements usually provide the lowest local latency. Cloud-centric systems may be adequate for monitoring but unsuitable for closed-loop control. Average latency is insufficient; tail latency and worst-case behaviour are essential in safety-critical systems.

Resilience refers to continued operation under device, network, service, or site failure. Local autonomy improves resilience to cloud disconnection, whereas centralized platforms may simplify recovery and replication. A resilient architecture needs graceful degradation, buffering, redundant paths, state recovery, and explicit failure handling.

Cost includes hardware, connectivity, cloud consumption, energy, software engineering, maintenance, security, and opportunity costs. Sending raw data to the cloud may appear simple but become expensive at scale. Edge processing requires distributed hardware and operational expertise. Serverless reduces infrastructure administration but can generate unexpected charges during high-volume events.

Autonomy is the capacity to make decisions without external services. It is important for remote, mobile, safety-critical, or privacy-sensitive environments. Autonomy may be local and temporary rather than complete: a device can enforce safety limits locally while depending on the cloud for optimization.

Maintainability includes software updates, testing, observability, configuration, compatibility, and lifecycle management. Centralized deployments are often easier to update, while distributed systems increase the number of versions and failure locations. Containerization, remote attestation, digital twins, automated deployment, and fleet-management platforms can reduce this burden but do not eliminate it.

A useful decision matrix scores each candidate architecture against these criteria using application-specific weights. For example, a smart-building analytics application may weight cost and maintainability highly, while an autonomous industrial controller may weight latency, resilience, and autonomy above all else. No architecture is universally optimal.

A Reference Hybrid Architecture

A robust IoT design commonly follows a staged data path. Sensors perform calibration, filtering, and basic feature extraction. An edge gateway authenticates devices, translates protocols, buffers data, and applies immediate rules. A fog or site platform aggregates streams from several gateways and coordinates local services. The cloud stores selected historical data, trains models, manages fleets, and provides enterprise applications. Serverless functions may handle sporadic integration and notification tasks.

This arrangement should not be adopted automatically. Each function should be placed according to its timing, data sensitivity, computational demand, failure consequences, and update frequency. Raw video, for example, may remain local while derived events are sent to the cloud. Safety interlocks may be implemented redundantly at the device and edge levels, while optimization decisions are delegated to a cloud model.

Applied Case: Distributed Monitoring in a Cold-Chain Network

Background

The following is an illustrative case involving a pharmaceutical distributor operating refrigerated vehicles and storage facilities. Temperature, humidity, door status, location, and power conditions are measured throughout the supply chain.

Situation and Problem

The initial design transmitted all sensor readings to a cloud dashboard. Managers received useful reports, but cellular coverage was unreliable on some routes. Large volumes of repeated data increased connectivity costs, and a cloud outage meant that local personnel received no immediate warning when temperatures exceeded safe limits. The organization also needed an auditable record of who changed thresholds and how alerts were handled.

Analysis

The system required local safety decisions, intermittent-network operation, fleet-wide analysis, and centralized governance. A purely cloud-centric architecture failed the latency and autonomy requirements. A completely decentralized design would complicate model management, reporting, and compliance. The requirements therefore supported a hybrid architecture.

Approach

Sensors performed calibration and discarded clearly invalid readings. An in-vehicle edge gateway authenticated sensors, maintained a local time source, calculated rolling statistics, and triggered a local alarm when temperature remained outside the permitted range. It buffered records during connectivity loss and used an event-driven protocol to publish prioritized messages when a connection became available.

A regional fog service aggregated vehicles within a distribution area, detected repeated refrigeration faults, and coordinated maintenance notifications. The cloud platform stored signed records, provided dashboards, trained predictive-maintenance models, and managed device identities and software releases. Serverless functions integrated high-priority events with messaging and ticketing systems. The cloud did not replace the local alarm; it supplemented it with coordination and analysis.

Outcome

The architecture reduced unnecessary transmission, preserved alarm capability during disconnection, and allowed analysts to compare conditions across vehicles and facilities. It also introduced new responsibilities: gateways required secure physical installation, software updates needed staged rollout, and event consumers had to handle duplicate messages.

Learning Points

The case demonstrates that architectural decisions should follow operational consequences rather than fashionable terminology. Local processing is justified by timing and autonomy requirements; cloud processing is justified by aggregation and coordination; event-driven communication is useful for asynchronous alerts; and governance must cover every layer of the distributed system.

Risks, Limitations, and Common Design Errors

A frequent error is treating the cloud as the default location for every function. This can create excessive latency, privacy exposure, and dependence on connectivity. The opposite error is moving computation to the edge without providing secure updates, monitoring, backup, or sufficient hardware capacity.

Other common problems include uncontrolled protocol translation, inconsistent device identities, unversioned event schemas, lack of clock synchronization, unbounded message retention, failure to design for duplicate events, and dashboards that report system health without measuring data freshness. Distributed deployments may also suffer from configuration drift: nominally identical gateways gradually acquire different software, policies, or certificates.

Security must be designed across the architecture. Device identity, mutual authentication, encrypted communication, least-privilege authorization, secure boot, signed updates, key rotation, network segmentation, logging, and vulnerability response are mutually reinforcing controls. Privacy engineering may require data minimization, local inference, anonymization, retention limits, and purpose restrictions. In regulated sectors, organizations must establish accountability for automated decisions, audit trails, data residency, safety assurance, and third-party dependencies.

Architects should also examine environmental and social costs. Additional edge hardware consumes materials and energy, while frequent wireless transmission consumes battery power. Automated monitoring can affect worker privacy and produce discriminatory outcomes if models are trained on incomplete or biased data. Technical efficiency does not by itself establish ethical legitimacy.

Emerging Directions

Several developments are reshaping distributed IoT architecture. Tiny machine-learning models are enabling inference on microcontrollers and gateways, reducing the need to transmit raw data. Federated learning allows models to be trained across sites while retaining much of the data locally, although communication, poisoning attacks, and model leakage remain concerns.

Digital twins increasingly connect physical assets with models of state, history, and expected behaviour. Their value depends on data quality, synchronization, semantic interoperability, and clear ownership. Declarative orchestration and infrastructure-as-code are helping organizations manage heterogeneous edge and cloud resources. Confidential computing, remote attestation, and hardware-backed identities seek to strengthen trust in shared infrastructure.

Research also continues on intent-based networking, time-sensitive wireless systems, 5G and 6G-native edge services, energy-aware orchestration, and serverless functions deployed closer to devices. These developments do not remove architectural trade-offs. They make systematic evaluation more important because resource placement, trust, and operational responsibility become increasingly dynamic.

Best Practices for Architectural Decision-Making

Begin with requirements expressed as measurable constraints: maximum acceptable latency, outage tolerance, energy budget, data-retention period, throughput, recovery time, and security obligations. Map data and control flows before selecting platforms. Separate hard real-time control from supervisory analytics. Minimize raw-data movement when derived features are sufficient.

Use open protocols and explicit schemas where interoperability matters. Make event processing idempotent, observable, and tolerant of reordering when possible. Design local fallback modes and test them under realistic network failures. Automate provisioning, certificate rotation, testing, deployment, and rollback. Maintain an inventory of devices, software versions, dependencies, and data flows.

Finally, evaluate the architecture through failure injection, load testing, wireless interference, power-loss scenarios, security assessment, and total-cost modelling. An architecture is credible only when it performs acceptably not just in normal operation but also under degraded and adversarial conditions.

Chapter Summary

IoT architecture distributes sensing, communication, computation, storage, decision-making, and management across devices, networks, edge and fog nodes, cloud platforms, and services. Layered models clarify responsibilities, while service-oriented designs organize reusable capabilities and event-driven designs support asynchronous interaction. Cloud-centric architectures provide aggregation and elasticity; mist, edge, and fog computing provide progressively more local processing and autonomy; serverless computing offers event-triggered managed execution but does not automatically provide real-time or local operation.

The principal evaluation criteria are scalability, latency, resilience, cost, autonomy, and maintainability. Their relative importance depends on the application. Hybrid architectures are often most effective, provided that security, observability, lifecycle management, interoperability, privacy, and failure handling are designed across all layers. Sound architectural practice begins with measurable requirements and validates decisions through operational, economic, ethical, and technical analysis.

Key Terms

Layered architecture: An arrangement that separates system functions into conceptual or operational layers.

Perception layer: The IoT layer containing sensors, actuators, identifiers, and physical interfaces.

Service-oriented architecture: An architecture that exposes reusable capabilities through defined service interfaces.

Event-driven architecture: An architecture organized around the production, transmission, and processing of events.

Cloud-centric architecture: A design in which major storage, computation, and management functions reside in cloud platforms.

Edge computing: Processing performed near the data source or point of action.

Fog computing: Distributed processing and coordination across intermediate nodes between devices and cloud resources.

Mist computing: Very local processing performed within or immediately adjacent to constrained end devices.

Serverless computing: Provider-managed execution in which developers deploy functions or services without administering servers directly.

Latency: The time required for data or a command to travel through a system and be processed.

Resilience: The ability of a system to continue or recover operation under faults and disruption.

Autonomy: The capacity of a system to operate and make decisions without external infrastructure.

Idempotency: The property that repeated processing of the same operation produces the same effective result.

Digital twin: A digital representation of a physical asset or process that is maintained through data and models.

Fog node: An intermediate computational resource coordinating devices or edge systems.

Data minimization: Collecting and retaining only the data necessary for a defined purpose.

Configuration drift: Unintended divergence among systems that were intended to have equivalent configurations.

Graceful degradation: Continued operation with reduced functionality when resources or components fail.

Review Questions

Short Answer Questions

1. What responsibilities are commonly associated with the perception layer?

2. How does event-driven architecture differ from request-response interaction?

3. Define edge computing and state one reason for using it in IoT.

4. Why can serverless computing create cost uncertainty?

5. What is configuration drift in a distributed IoT deployment?

Long Answer Questions

1. Explain the layered IoT architecture and critically discuss why strict layering may be insufficient for some IoT applications.

2. Compare cloud-centric, edge, fog, and mist computing in terms of latency, autonomy, resilience, and maintainability.

3. Discuss how service-oriented and event-driven architectures can be combined in an IoT platform.

4. Explain the principal architectural risks associated with centralized cloud processing.

5. Develop a framework for selecting an IoT architecture using the six evaluation criteria presented in this chapter.

Analytical / Application Questions

1. A mining operation must detect dangerous gas concentrations despite intermittent connectivity. Which functions should be placed at the device, edge, fog, and cloud levels, and why?

2. An organization sends continuous camera video to the cloud for occupancy analytics. Analyse the likely cost, privacy, latency, and resilience problems and propose an alternative architecture.

3. A distributed event system occasionally delivers the same equipment-alert message twice and sometimes delivers messages out of order. What design principles should the application use to remain reliable?

22. Multiple Choice Questions with Answers

Question 1

Which architectural placement is most appropriate for a safety decision that must be made within milliseconds during Internet disconnection?

A. Central cloud service

B. Local device or edge controller

C. Cloud-based batch analytics

D. Remote enterprise dashboard

Correct Answer: B. Local device or edge controller

Question 2

What is the primary architectural distinction between an event and a command?

A. An event records something that occurred, whereas a command requests an action

B. An event always requires a response, whereas a command never does

C. An event is local, whereas a command must be cloud-based

D. An event is encrypted, whereas a command is not

Correct Answer: A. An event records something that occurred, whereas a command requests an action

Question 3

Which feature most directly supports scalability in a serverless IoT workload with highly variable message volume?

A. Manual installation of fixed servers

B. Dynamic provider-managed resource allocation

C. Permanent execution on every sensor

D. Elimination of all network communication

Correct Answer: B. Dynamic provider-managed resource allocation

Question 4

A gateway filters sensor data, buffers it during network outages, and triggers a local alarm. Which architectural approach does this best illustrate?

A. Cloud-only processing

B. Edge computing with local autonomy

C. Centralized batch processing

D. Presentation-layer isolation

Correct Answer: B. Edge computing with local autonomy

Question 5

Why should an event consumer often be idempotent in an IoT system?

A. Wireless networks guarantee duplicate-free delivery

B. At-least-once delivery can cause the same event to be processed repeatedly

C. Idempotency eliminates the need for authentication

D. It ensures that all events arrive in chronological order

Correct Answer: B. At-least-once delivery can cause the same event to be processed repeatedly

Question 6

Which criterion is most directly concerned with continued operation when the cloud or wide-area network is unavailable?

A. Autonomy

B. Interoperability

C. Data visualization

D. Storage density

Correct Answer: A. Autonomy

Question 7

A system aggregates data from multiple local gateways and coordinates site-level services before sending selected information to the cloud. What does this most closely represent?

A. Mist computing only

B. Fog computing

C. A purely layered network model

D. A standalone sensor network

Correct Answer: B. Fog computing

Question 8

Which is a significant limitation of a purely cloud-centric IoT architecture?

A. It cannot store historical data

B. It necessarily prevents centralized governance

C. It may introduce wide-area latency and dependence on connectivity

D. It cannot support machine learning

Correct Answer: C. It may introduce wide-area latency and dependence on connectivity

Question 9

Which practice best addresses configuration drift across distributed gateways?

A. Allowing each gateway to be manually customized indefinitely

B. Using automated provisioning, version control, and fleet management

C. Disabling software updates

D. Removing device identities

Correct Answer: B. Using automated provisioning, version control, and fleet management

Question 10

In the cold-chain case, why was a hybrid architecture preferable to either a fully centralized or fully decentralized design?

A. It combined local safety response with centralized analysis and governance

B. It eliminated the need for wireless communication

C. It made all devices independent of software updates

D. It guaranteed that no messages could be duplicated

Correct Answer: A. It combined local safety response with centralized analysis and governance