Real-Time, Embedded, Mobile, Edge, and AI Systems

Introduction and Context

Operating systems for general-purpose computers are usually optimized for throughput, convenience, and fairness. Systems used in vehicles, industrial controllers, medical devices, smartphones, autonomous robots, and AI appliances have additional requirements. A control system may need to respond before a deadline; a wearable device may operate for days on a small battery; an edge server may process sensor data without sending it to a cloud; and an AI accelerator may require carefully coordinated memory transfers and scheduling.

These environments expose tensions among predictability, performance, energy efficiency, flexibility, security, and cost. This chapter examines the operating-system principles used to manage those tensions. It focuses on real-time guarantees, embedded constraints, mobile power management, edge platforms, GPU and accelerator scheduling, heterogeneous memory, and OS support for machine-learning training and inference.

Learning Objectives

After studying this chapter, a student should be able to:

1. Distinguish hard, firm, and soft real-time requirements.

2. Analyze response-time and schedulability concepts using standard assumptions.

3. Explain the design constraints of embedded and mobile operating systems.

4. Describe OS mechanisms for power, thermal, and resource management.

5. Compare cloud, edge, and device-local computing.

6. Explain GPU, NPU, and accelerator scheduling challenges.

7. Describe heterogeneous memory and data-movement costs.

8. Identify OS support required by AI training and inference workloads.

9. Evaluate trade-offs involving safety, security, privacy, and performance.

1. Real-Time Operating-System Concepts

1.1 What makes a system real-time?

A real-time system is one in which the correctness of computation depends not only on the result but also on when the result is produced. The central concern is therefore bounded timing behavior rather than merely high average speed.

A task is commonly described by:

• Release time: when the task becomes ready.
• Execution time: the processor time required under stated assumptions.
• Deadline: the latest acceptable completion time.
• Period: the interval between releases of a periodic task.
• Response time: elapsed time from release until completion.
• Jitter: variation in release or completion timing.

A hard real-time deadline must never be missed, as in an airbag controller. A firm real-time result has no value after its deadline, but occasional misses may be tolerated. A soft real-time system experiences degraded quality when deadlines are missed, as in video playback or speech recognition.

Real-time does not necessarily mean “fast.” A slower processor can be suitable if its timing is predictable, whereas a faster processor with uncontrolled cache, interrupt, or virtual-memory delays may be unsuitable for a hard deadline.

1.2 Scheduling models

A real-time scheduler assigns processors to tasks according to policies designed to meet deadlines. Common policies include:

• Rate Monotonic (RM): fixed priority; shorter-period periodic tasks receive higher priority.
• Deadline Monotonic: fixed priority; shorter relative deadlines receive higher priority.
• Earliest Deadline First (EDF): dynamic priority; the ready task with the nearest absolute deadline runs first.
• Fixed-priority preemptive scheduling: priorities are assigned in advance, and a higher-priority task may interrupt a lower-priority task.
• Partitioned or global multiprocessor scheduling: tasks are assigned to processors or selected dynamically across processors.

For a set of independent periodic tasks scheduled by preemptive EDF on one processor, a standard idealized condition is that total utilization must not exceed one:

U = Σ(Cᵢ/Tᵢ) ≤ 1

Here, Cᵢ is the worst-case execution time of task i and Tᵢ is its period. This condition assumes known execution times, negligible scheduling overhead, independent tasks, and deadlines equal to periods. It is not by itself a complete test when tasks have arbitrary deadlines, blocking, release jitter, nonpreemptive sections, or shared resources.

For fixed-priority systems, response-time analysis is more informative. Under common assumptions, the response time of task i can be found by iterating:

Rᵢ = Cᵢ + Bᵢ + Σ (over higher-priority tasks j) ceil(Rᵢ/Tⱼ) Cⱼ

Cᵢ is task i’s execution time, Bᵢ is blocking caused by lower-priority tasks, and the summation represents interference from higher-priority tasks. The calculation is repeated until it converges or exceeds the task’s deadline Dᵢ. A task is schedulable if Rᵢ ≤ Dᵢ. The formula is an analysis model, not a universal law; interrupt costs, context-switch costs, release jitter, multiprocessor effects, and operating-system overhead must be included when relevant.

1.3 Worked schedulability example

Consider three independent periodic tasks on one processor:

• Task A: C = 1 ms, T = 4 ms
• Task B: C = 1 ms, T = 5 ms
• Task C: C = 2 ms, T = 10 ms

The utilization is:

U = 1/4 + 1/5 + 2/10 = 0.25 + 0.20 + 0.20 = 0.65.

Under the ideal EDF assumptions and deadlines equal to periods, utilization of 0.65 is below one, so the task set passes this sufficient feasibility condition. This does not prove that a real implementation will meet deadlines if device interrupts, locking, cache misses, or execution-time variation are omitted. Engineering certification requires a more conservative analysis and measurements under worst-case conditions.

1.4 Predictability and kernel design

A real-time kernel reduces unbounded delays by using:

• bounded interrupt and dispatch latency;
• priority-based preemption;
• deterministic synchronization protocols;
• carefully controlled memory allocation;
• timers with known resolution and drift;
• avoidance or management of paging;
• high-resolution clocks;
• admission control and resource reservations.

Priority inversion occurs when a high-priority task waits for a lock held by a low-priority task while medium-priority tasks continue executing. Priority inheritance temporarily raises the lock holder’s priority. Priority ceiling protocols restrict which tasks may acquire a resource and can bound blocking more tightly. These protocols are essential in safety-critical systems.

Real-time Linux configurations, commercial RTOSs, and microkernels differ in architecture, certification evidence, driver support, and timing guarantees. A general-purpose kernel can offer excellent average latency without providing a formal hard-real-time guarantee. Claims must therefore distinguish measured performance from proven bounds.

2. Embedded Operating Systems

Embedded systems are computing systems integrated into a larger product. They often have a narrowly defined function, constrained memory, limited energy, specialized peripherals, and long product lifetimes. Examples include engine controllers, smart meters, thermostats, cameras, and industrial sensors.

2.1 Resource constraints

Embedded OS design commonly addresses:

• small RAM and nonvolatile storage;
• low-power processors;
• limited display and input facilities;
• specialized buses and sensors;
• predictable boot time;
• restricted update opportunities;
• environmental conditions such as heat and vibration;
• safety and reliability requirements.

A monolithic embedded OS places many services in one privileged address space, reducing communication overhead but increasing fault-containment risk. A microkernel keeps only minimal mechanisms in kernel mode and moves drivers and services into separate processes, improving isolation at the cost of interprocess communication and design complexity. Many contemporary systems use hybrid approaches.

2.2 Device drivers and interrupts

Drivers convert hardware events into OS-managed operations. In an embedded controller, a sensor interrupt may initiate a short, high-priority handler. Good practice is to perform minimal urgent work in the interrupt context and defer longer processing to a schedulable task. Excessively long interrupt handlers delay other events and make timing analysis difficult.

Direct Memory Access (DMA) allows a peripheral to transfer data without continuous CPU copying. DMA improves throughput and reduces processor load, but it introduces cache-coherency, buffer-ownership, and memory-protection issues. The OS must ensure that a device cannot write outside its authorized buffer.

2.3 Reliability, safety, and updates

Embedded products may remain deployed for ten or twenty years. Reliability requires watchdog timers, fault logging, defensive programming, redundancy where justified, and recovery modes. Secure boot establishes that only authenticated software executes. Signed updates and rollback protection help prevent malicious or corrupted firmware from being installed.

Safety and security are related but distinct. Safety protects people and equipment from accidental failure; security protects against intentional misuse. A secure system can still be unsafe because of timing defects, and a safe design can be vulnerable to cyberattack.

3. Mobile Operating Systems and Energy Management

Mobile devices operate under a limited energy budget while supporting demanding workloads such as cameras, wireless networking, graphics, and AI inference. Battery capacity is finite, and heat cannot be continuously dissipated like that of a desktop server.

3.1 Energy and power

Power is the rate of energy consumption. Over an interval, energy can be expressed as:

E = ∫ P(t) dt

where E is energy and P(t) is instantaneous power. If power is approximately constant, E = P × t. Battery life depends on workload, radio conditions, display brightness, temperature, battery aging, and conversion losses; it cannot be predicted accurately from processor frequency alone.

Dynamic Voltage and Frequency Scaling (DVFS) changes clock frequency and supply voltage. Lowering frequency can reduce dynamic power, while lowering voltage often has an especially strong effect, although the exact relationship depends on the processor. DVFS may increase execution time, so the OS must evaluate energy per task, deadline risk, and thermal conditions together.

3.2 Mobile resource policies

Mobile OSs use mechanisms such as:

• idle states and suspend-to-RAM;
• background execution limits;
• batching of sensor and network requests;
• radio power-state management;
• CPU and GPU frequency scaling;
• thermal throttling;
• application quotas and wakelocks;
• user-aware scheduling.

Batching several sensor operations can reduce repeated wake-up costs, but it increases latency and may reduce freshness. Thermal throttling protects hardware but can cause performance variability. A well-designed scheduler considers deadlines, user interaction, battery state, and skin temperature rather than optimizing only CPU utilization.

Mobile isolation uses application sandboxes, permission systems, cryptographic storage, verified boot, and policy-controlled access to location, microphones, cameras, and health data. These controls are central to trustworthy AI applications that process personal information.

4. Edge Computing Platforms

Edge computing places computation near data sources or users rather than exclusively in a remote cloud. The edge may be a gateway, cellular base station, factory server, vehicle computer, or household appliance.

4.1 Why use the edge?

Edge processing can reduce network latency, bandwidth consumption, and exposure of raw data. It may continue operating during cloud disconnection and can support local control loops. However, edge devices are often more heterogeneous, physically exposed, intermittently connected, and difficult to administer than centralized data centers.

A useful architecture is a hierarchy:

1. Device edge: sensors, cameras, wearables, and embedded controllers.

2. Local edge: gateways and on-premises servers.

3. Regional edge: carrier or metropolitan computing infrastructure.

4. Cloud: large-scale centralized computing and storage.

Workloads can be partitioned across levels. For example, a camera may perform object detection locally, send only metadata to a gateway, and upload selected clips to the cloud for model improvement.

4.2 Edge OS responsibilities

An edge OS must support container or virtual-machine isolation, device discovery, local storage, intermittent connectivity, remote management, secure provisioning, and resource quotas. It should expose telemetry for CPU, memory, temperature, accelerator utilization, network quality, and battery state.

Placement decisions should consider latency, energy, privacy, model size, network cost, and availability. Moving a computation to the cloud may save local energy but introduce communication delay and confidentiality risks. Keeping it local may improve responsiveness but exceed local memory or thermal capacity.

5. GPU and Accelerator Scheduling

AI and graphics workloads use GPUs, NPUs, TPUs, DSPs, FPGAs, and other accelerators. These devices often execute thousands of parallel operations and have their own command queues, memory spaces, and synchronization mechanisms.

5.1 Scheduling challenges

An accelerator scheduler must manage:

• multiple applications sharing one device;
• command submission and completion;
• context switching or partitioning;
• fairness and priority;
• memory isolation;
• deadlines and quality-of-service requirements;
• thermal and power limits;
• data transfer between host and device.

A GPU context switch can be expensive because state may include registers, caches, page tables, and command buffers. Some devices support preemption at fine granularity; others preempt only between kernels. Consequently, a high-priority inference request can be delayed by a long-running training kernel.

Possible policies include time slicing, space sharing, static partitioning, priority queues, admission control, and reservation-based scheduling. A reservation can guarantee a portion of accelerator capacity, but only if execution and transfer times are sufficiently characterized.

5.2 GPU virtualization and isolation

Virtualization may expose an accelerator to multiple virtual machines or containers. Isolation can be implemented through hardware address translation, mediated devices, partitioned engines, or software enforcement. Security is particularly important because accelerator memory may contain model parameters and personal data. Memory should be cleared or protected when ownership changes.

For real-time inference, average throughput is not enough. Tail latency—the slowest portion of requests—may determine whether a service meets its contract. Queueing, memory contention, thermal throttling, and background kernels can produce large tail delays.

6. Heterogeneous Memory and Data Movement

Modern systems contain several memory levels: CPU caches, system DRAM, GPU memory, NPU-local SRAM, nonvolatile storage, and sometimes remote or pooled memory. They differ in capacity, bandwidth, latency, persistence, and energy cost.

A unified virtual address space can simplify programming by allowing processors to refer to a common address range. It does not necessarily mean that all processors have equal access time or that data movement is free. A discrete GPU may still require explicit transfers across PCIe or another interconnect. A shared-memory accelerator may avoid copies but create cache-coherency and contention costs.

The OS and runtime must manage:

• page placement and migration;
• pinned memory for DMA;
• cache coherence;
• address translation and IOMMUs;
• memory compression;
• huge pages;
• zero-copy buffers;
• out-of-memory behavior;
• protection between tenants.

For AI, a model may fit in system memory but not in accelerator memory. Techniques such as quantization, pruning, tensor sharding, operator fusion, checkpointing, and offloading reduce capacity pressure or transfer volume. Offloading can make a model executable but may increase latency and energy.

7. Operating-System Support for AI/ML

7.1 Training systems

Training workloads require high throughput, large datasets, distributed communication, checkpointing, and fault recovery. The OS supports them through:

• accelerator drivers and runtime APIs;
• high-bandwidth storage and I/O paths;
• DMA and pinned buffers;
• process and container isolation;
• CPU and accelerator affinity;
• distributed networking;
• checkpoint and restart mechanisms;
• telemetry and resource accounting.

Training jobs are often elastic and batch-oriented, whereas inference services may have strict latency requirements. A cluster scheduler must prevent a large training job from monopolizing resources needed by interactive inference. This may involve separate pools, priority classes, quotas, preemption, or reserved accelerators.

7.2 Inference systems

Inference emphasizes predictable latency, high availability, energy efficiency, and model confidentiality. The OS and runtime may support batching, model caching, request admission, asynchronous I/O, operator fusion, quantized execution, and accelerator selection.

Batching improves device utilization but increases waiting time. Dynamic batching should therefore use a maximum batch delay or deadline. Model servers may maintain several model versions and route requests according to accuracy, hardware capability, or regulatory policy.

AI systems also need observability. Logs and metrics should record model version, hardware, input and output timing, failures, and resource consumption without unnecessarily retaining sensitive input data. Reproducibility requires recording software versions, drivers, model artifacts, and relevant configuration.

8. Detailed Applied Case: Illustrative Edge Vision Controller

Background

Consider a fictional but realistic factory robot that uses cameras to detect misplaced components on a production line. The robot must stop or divert an item within 80 ms of image capture. It has a multicore CPU, an NPU, limited local storage, an industrial network connection, and a fanless enclosure.

Problem

An initial design sends images to a cloud service. Average response time is acceptable, but network congestion causes occasional delays above 80 ms. Continuous transmission also exposes sensitive production data and increases energy use. Running the complete vision model on the CPU causes thermal throttling.

Analysis

The requirement is effectively hard or near-hard real-time for the control action. Average latency is insufficient; the design must analyze worst-case or high-percentile response time. The image pipeline includes capture, preprocessing, memory transfer, NPU execution, postprocessing, and actuator communication. Each stage contributes latency and possible jitter.

The OS must give the control task suitable priority, isolate it from maintenance activities, use bounded queues, and prevent storage or network work from starving the inference path. The NPU scheduler must prevent long background kernels from delaying urgent inference. DMA buffers must be protected and correctly synchronized.

Approach and solution

The revised design performs initial detection on the edge device. A compact quantized model runs on the NPU; only metadata and selected anomalous images are sent to the cloud. The system reserves one CPU core for control-related processing, limits inference queue depth, and rejects or defers nonurgent requests when the device is thermally constrained. A watchdog restarts failed inference components, while a safe fallback controller stops the line if confidence or timing requirements are not met.

Model updates are signed, tested offline, and deployed gradually. The device records timing statistics, model identity, temperature, and failure codes. Raw images are retained only under a defined investigation policy.

Outcome

Local execution reduces network dependence and improves privacy. NPU execution reduces CPU load, but the system still requires thermal testing, accelerator-driver validation, and timing analysis. The fallback behavior ensures that an uncertain AI result does not directly create an unsafe action.

Learning points

The case demonstrates that an AI model is only one component of a dependable system. OS scheduling, memory movement, thermal management, security updates, observability, and safe failure behavior are equally important. Edge deployment is an architectural decision involving timing, privacy, energy, and lifecycle management.

9. Challenges, Ethics, and Governance

Heterogeneous systems are difficult to test because behavior depends on hardware revisions, firmware, drivers, temperature, workload interference, and network conditions. Formal timing guarantees may be impossible for complex AI models whose execution varies with control paths or hardware state. Benchmark results should therefore identify workload, batch size, precision, temperature, and measurement method.

Privacy risks arise when cameras, microphones, location sensors, or health devices process personal data. Data minimization, local processing, encryption, access control, retention limits, and documented consent are important safeguards. Edge devices are physically accessible, making debug ports, keys, and firmware protection significant security concerns.

AI in safety-related systems requires human accountability, traceability, validation data, monitoring for distribution shift, and an explicit response to low-confidence predictions. Regulatory requirements vary by sector and jurisdiction; engineers should consult applicable functional-safety, cybersecurity, privacy, medical-device, automotive, and AI-governance standards rather than treating compliance as a purely technical feature.

10. Best Practices

1. Specify deadlines, jitter limits, availability, thermal limits, and energy budgets explicitly.

2. Distinguish measured average performance from a defensible worst-case bound.

3. Include interrupts, drivers, locks, memory transfers, and thermal throttling in timing analysis.

4. Use priority inheritance or ceiling protocols for shared real-time resources.

5. Keep interrupt handlers short and validate DMA ownership and buffer boundaries.

6. Separate safety-critical, latency-sensitive, and best-effort workloads.

7. Measure accelerator tail latency, not only peak throughput.

8. Minimize data movement through locality, fusion, quantization, and suitable memory placement.

9. Secure boot, signed updates, least privilege, and rollback protection should be designed from the beginning.

10. Provide safe fallback behavior when an AI result is late, unavailable, or uncertain.

11. Record model, driver, hardware, and configuration versions for reproducibility.

12. Test across realistic temperature, power, connectivity, and interference conditions.

Contemporary Developments

Current systems increasingly combine CPUs with GPUs, NPUs, and specialized inference engines. Confidential computing, memory pooling, chiplet-based accelerators, real-time containerization, and energy-aware AI runtimes are active areas of development. TinyML moves inference onto microcontrollers, while federated and split learning reduce the need to centralize raw data. These developments increase capability but also complicate scheduling, verification, memory management, and governance.

Chapter Summary

Real-time operating systems are concerned with bounded timing and deadline satisfaction, not simply average speed. Task periods, execution times, deadlines, utilization, response time, blocking, and interference provide a vocabulary for schedulability analysis, but formulas apply only under stated assumptions. Embedded systems emphasize limited resources, specialized hardware, reliability, long lifetimes, and secure updates. Mobile operating systems balance responsiveness with battery and thermal constraints through idle states, DVFS, batching, and application policies.

Edge computing places processing near data sources to reduce latency, bandwidth use, and cloud dependence, but it introduces heterogeneity, physical exposure, and management challenges. GPUs and other accelerators require specialized scheduling, isolation, memory management, and attention to tail latency. Heterogeneous memory makes data placement and transfer costs central to system performance. AI training needs throughput, distributed resources, and checkpointing; inference needs predictable latency, efficient execution, isolation, and safe failure behavior. Across all these systems, performance must be evaluated together with security, privacy, safety, energy, maintainability, and governance.

Key Terms

Real-time system: A system whose correctness depends on both results and their timing.

Hard real-time: A system in which missing a deadline is unacceptable.

Soft real-time: A system in which deadline misses degrade service quality.

Jitter: Variation in the timing of events or task execution.

Worst-case execution time: A justified upper bound on task execution under defined conditions.

Schedulability: The ability of a task set to meet its timing constraints.

Priority inversion: Blocking of a higher-priority task by lower-priority activity.

Priority inheritance: A protocol that temporarily raises a lock holder’s priority.

Embedded system: A computer integrated into a larger product for a dedicated function.

DMA: Direct transfer between a device and memory with limited CPU copying.

DVFS: Dynamic adjustment of processor voltage and frequency.

Thermal throttling: Reducing performance to keep temperature within safe limits.

Edge computing: Processing data near its source or users.

Accelerator: Hardware specialized for a class of computations.

Tail latency: High-percentile request latency, often important for service guarantees.

Heterogeneous memory: Memory resources with different latency, capacity, bandwidth, or persistence.

Quantization: Representing numerical values with reduced precision to save resources.

Admission control: Deciding whether a workload may enter a system while preserving guarantees.

Secure boot: A mechanism that authenticates software before execution.

Model drift: Change in data conditions that reduces an AI model’s effectiveness.

Review Questions

Short Answer Questions

1. How does hard real-time differ from soft real-time?

2. What information is represented by a task’s period and execution time?

3. Why can a faster processor still be unsuitable for a hard real-time task?

4. What problem does DVFS address in mobile systems?

5. Why is tail latency important for AI inference?

Long Answer Questions

1. Explain fixed-priority and EDF scheduling, including the assumptions behind utilization-based analysis.

2. Discuss the principal operating-system constraints of embedded systems.

3. Explain how mobile operating systems balance performance, battery life, and thermal limits.

4. Analyze the benefits and risks of placing AI inference at the edge.

5. Describe how heterogeneous memory and accelerator scheduling affect AI workload performance.

Analytical / Application Questions

1. A single processor executes three periodic tasks with execution times of 2, 1, and 1 ms and periods of 8, 5, and 10 ms. Calculate total utilization and explain what an ideal EDF utilization test can and cannot establish.

2. Design an OS policy for a mobile device that must run camera inference while preserving battery life. Identify scheduling, memory, thermal, and privacy decisions.

3. For an industrial AI controller with an 80 ms deadline, identify all major sources of latency and propose a validation plan that distinguishes average performance from dependable timing behavior.

Multiple Choice Questions with Answers

Question 1

Which property is central to a hard real-time system?

A. Maximum average throughput

B. Never missing specified deadlines

C. Unlimited memory capacity

D. Continuous cloud connectivity

Correct Answer: B. Never missing specified deadlines

Question 2

Under ideal single-processor EDF assumptions with deadlines equal to periods, which condition is commonly used?

A. Σ(Cᵢ/Tᵢ) must be at most 1

B. Σ(CᵢTᵢ) must be at least 1

C. Every task must have the same period

D. The processor must remain idle half the time

Correct Answer: A. Σ(Cᵢ/Tᵢ) must be at most 1

Question 3

What is the purpose of priority inheritance?

A. To reduce disk fragmentation

B. To prevent a low-priority task from holding a resource while indefinitely delaying a high-priority task

C. To increase virtual-memory size

D. To encrypt network traffic

Correct Answer: B. To prevent a low-priority task from holding a resource while indefinitely delaying a high-priority task

Question 4

Which technique commonly reduces mobile energy consumption by allowing hardware to sleep between groups of operations?

A. Batching

B. Busy waiting

C. Disabling all caches

D. Increasing interrupt frequency

Correct Answer: A. Batching

Question 5

What is a principal benefit of edge computing?

A. It guarantees unlimited compute capacity

B. It can reduce latency and dependence on wide-area networks

C. It eliminates all security risks

D. It always lowers local energy use

Correct Answer: B. It can reduce latency and dependence on wide-area networks

Question 6

Why can a long-running GPU kernel threaten urgent inference?

A. GPUs cannot execute arithmetic operations

B. Accelerator preemption may be coarse-grained or expensive

C. Inference never uses memory

D. The operating system cannot observe any process

Correct Answer: B. Accelerator preemption may be coarse-grained or expensive

Question 7

Which statement about a unified virtual address space is correct?

A. It guarantees identical access time for all processors

B. It eliminates all data-transfer costs

C. It can simplify addressing without making all memory accesses equally fast

D. It prevents memory protection

Correct Answer: C. It can simplify addressing without making all memory accesses equally fast

Question 8

Which workload characteristic is especially associated with AI training rather than latency-sensitive inference?

A. Checkpointing large jobs and maximizing throughput

B. Strict response deadlines for individual requests

C. Immediate safe fallback after one missed request

D. Small fixed input buffers only

Correct Answer: A. Checkpointing large jobs and maximizing throughput

Question 9

What does secure boot primarily do?

A. Authenticates software before allowing it to execute

B. Increases GPU clock frequency

C. Compresses neural-network tensors

D. Guarantees model accuracy

Correct Answer: A. Authenticates software before allowing it to execute

Question 10

In the illustrative factory case, why was a fallback controller important?

A. It made cloud connectivity mandatory

B. It ensured an unavailable, late, or uncertain AI result would not directly create an unsafe action

C. It removed the need for timing analysis

D. It increased image retention for all users

Correct Answer: B. It ensured an unavailable, late, or uncertain AI result would not directly create an unsafe action