Processes and Threads

Introduction

A modern operating system must manage many activities that appear to execute simultaneously: user applications, background services, device drivers, network handlers, and system tasks. The central abstraction used to represent a running program is the process. A process provides an execution environment containing a program’s instructions, data, resources, and current execution state. A thread is the unit within a process that the operating system schedules for execution.

Understanding processes and threads is essential for designing responsive applications, exploiting multicore processors, managing resources safely, and diagnosing failures. In artificial intelligence and machine-learning systems, processes and threads are used to coordinate data preparation, model training, inference requests, accelerators, and communication with distributed services.

This chapter examines process states, process control blocks, context switching, process creation and termination, user and kernel threads, multithreading models, signals, and thread-level parallelism.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the process abstraction and distinguish a process from a program and a thread.

2. Interpret process states and state transitions.

3. Describe the contents and purpose of a process control block.

4. Explain context switching and its performance costs.

5. Compare process-creation and process-termination mechanisms.

6. Distinguish user-level and kernel-level threads and explain common multithreading models.

7. Describe signal delivery, handling, and limitations.

8. Analyze thread-level parallelism, synchronization concerns, and practical design trade-offs.

1. Programs, Processes, and Threads

A program is a passive collection of instructions stored on secondary storage. A process is an active instance of a program. When a program is loaded, the operating system establishes an address space, initializes registers and other control information, allocates resources, and schedules execution.

Two processes may execute the same program while maintaining independent data and execution states. For example, two users can run the same editor without normally sharing its memory. A process typically contains:

• a text or code segment;
• initialized and uninitialized data segments;
• a heap used for dynamic allocation;
• one or more stacks;
• CPU-register values and a program counter;
• open files, communication endpoints, and other resources; and
• security credentials and scheduling attributes.

A process may contain one or many threads. A thread has its own program counter, register set, and stack, but threads in the same process share code, global data, heap memory, and many operating-system resources. This sharing makes communication efficient but creates risks such as data races.

The distinction is therefore both conceptual and practical: a process is primarily a resource-protection and ownership abstraction, whereas a thread is primarily an execution and scheduling abstraction.

2. Process States and Transitions

Operating systems commonly describe a process using a state model. The basic five-state model contains:

• New: the process is being created.
• Ready: it can execute but is waiting for a processor.
• Running: its instructions are currently executing on a processor.
• Waiting or Blocked: it cannot continue until an event occurs, such as completion of I/O or arrival of a message.
• Terminated: execution has ended and operating-system resources are being reclaimed.

Typical transitions are:

• New → Ready: admission is complete.
• Ready → Running: the scheduler dispatches the process.
• Running → Ready: preemption occurs, often because a time slice expires or a higher-priority process becomes ready.
• Running → Waiting: the process requests I/O, waits for synchronization, or waits for another event.
• Waiting → Ready: the event completes.
• Running → Terminated: the process exits or is forcibly terminated.

Some systems add suspended states, such as Ready Suspended and Blocked Suspended, when a process is temporarily moved out of main memory. A blocked process cannot run because it is waiting for an event; a suspended process may be ready but has been removed from memory or otherwise excluded from immediate execution. These are different conditions and should not be conflated.

A process can be ready while another process is running, but on a single-core processor only one thread can execute at an instant. On a multicore processor, several threads may be running simultaneously, subject to the number of available cores and scheduling constraints.

State-transition example

Suppose a process reads a file. It begins in the Ready state, is dispatched, and enters Running. When it issues a blocking read operation, it enters Waiting rather than consuming CPU time. After the storage device supplies the data, the process moves to Ready. It eventually runs again and may later terminate.

This model helps administrators interpret monitoring data. A large ready queue may indicate processor saturation, whereas many blocked processes may indicate slow storage, network delays, lock contention, or an application intentionally waiting for external events.

3. The Process Control Block

The operating system maintains a process control block (PCB) for each process. The PCB is a kernel data structure containing the information required to manage, suspend, resume, schedule, protect, and terminate the process. Exact fields differ across operating systems, but common categories include:

1. Identification: process identifier (PID), parent-process identifier, user and group identifiers.

2. Process state: current state and relevant event or wait information.

3. CPU context: program counter, general-purpose registers, stack pointer, processor-status information, and sometimes floating-point or vector-register state.

4. Scheduling information: priority, scheduling class, CPU affinity, accumulated CPU time, and queue links.

5. Memory-management information: page-table or segment references, address-space identifiers, and memory limits.

6. Resource information: open-file references, devices, signal dispositions, accounting data, and interprocess-communication references.

7. Security and control information: privileges, credentials, isolation attributes, and resource quotas.

The PCB is not necessarily one contiguous structure. A kernel may maintain related structures in separate tables or objects. Nevertheless, the abstraction is useful: without saved control information, the operating system could not reliably pause one execution context and resume it later.

A process identifier is an operating-system handle, not a permanent identity. PIDs can be reused after a process terminates. Software that tracks processes should therefore avoid assuming that a PID alone identifies the same process indefinitely.

4. Context Switching

A context switch occurs when the processor changes from one execution context to another. The kernel saves the current thread’s processor context and restores the context of a selected thread. The saved information may include registers, the program counter, stack pointer, processor flags, memory-management references, and architecture-specific state.

A simplified sequence is:

1. An interrupt, system call, exception, or scheduler event transfers control to the kernel.

2. The current context is saved in its thread or process control structure.

3. The scheduler selects another runnable thread.

4. The next context is restored.

5. Execution returns to the selected thread.

Context-switch time is overhead: during the switch, useful application instructions are generally not progressing. The cost depends on processor architecture, operating-system implementation, register state, cache and translation-lookaside-buffer effects, security mechanisms, and whether the address space changes. Switching between threads in one process may avoid some address-space changes, but it is not automatically free.

Context switching should be distinguished from mode switching. A system call can switch from user mode to kernel mode and later return to the same thread; that is not necessarily a change to another process or thread. A context switch changes the execution context, while a mode switch changes privilege level.

Worked timing example

Assume a workload performs 1,000 context switches per second, and each switch consumes 4 microseconds of processor time. The overhead is:

`1,000 × 4 microseconds = 4,000 microseconds = 0.004 seconds per second`.

Thus, the switching overhead is 0.4% of one processor’s time under these assumptions. The calculation applies only to the stated average rate and switch cost; cache misses, I/O, and other overheads are excluded. If the switch cost rises or switches become more frequent, the lost capacity rises proportionally.

5. Process Creation and Termination

5.1 Process creation

A process may create another process, often called a child. The parent may continue concurrently, wait for the child, or communicate with it. Operating systems provide different interfaces, but two broad creation strategies are common:

• Create and execute: establish a new process and then load a selected program into it.
• Spawn: create a process and specify the program and initial configuration as part of one higher-level operation.

In Unix-like systems, `fork()` traditionally creates a child with a logical copy of the parent’s address space, followed commonly by `exec()` to replace the child’s program image. Modern implementations frequently use copy-on-write: physical memory pages are shared until one process writes to a page. This reduces immediate copying but still requires careful management of page tables and inherited resources.

The `exec` operation does not normally create another process. It replaces the calling process’s program image while preserving its process identity. This distinction is important when tracing process trees.

In Windows, the `CreateProcess` family generally creates a new process and loads a specified executable in one operation. The precise inheritance of handles, security attributes, and environment information is explicitly controlled through the API.

Process creation involves policy decisions. Unrestricted creation can exhaust memory, process-table entries, file descriptors, or other resources. Operating systems therefore apply permissions, quotas, and limits.

5.2 Process termination

A process may terminate normally by returning from its main function or calling an exit operation. It may terminate abnormally because of an unhandled exception, illegal memory access, resource failure, or an external termination request.

Termination requires the operating system to release memory mappings, close or reassign resources, record an exit status, notify interested processes, and remove control structures. In Unix-like systems, a terminated child whose status has not been collected can remain as a zombie: it no longer executes but retains limited bookkeeping information. A parent collects that status using a wait operation. If the parent exits first, the child may be adopted by a system process or supervisor.

A process can also be terminated by a supervisor, administrator, or parent. Forced termination is potentially unsafe because application-level cleanup may not occur. Systems that require reliable cleanup should use cooperative shutdown protocols where possible.

6. User-Level and Kernel-Level Threads

User-level threads

User-level threads are managed primarily by a runtime library rather than directly by the kernel. Advantages include fast creation and switching, customizable scheduling, and portability. A library can implement specialized mechanisms such as user-space coroutines or task schedulers.

A major limitation is blocking. In a purely user-level design, if one thread performs a blocking system call and the kernel sees only one process execution context, the entire process may block. User-level scheduling also cannot independently place threads on multiple cores unless the kernel provides an appropriate execution interface.

Kernel-level threads

Kernel-level threads are known to and scheduled by the operating system. A blocking operation by one thread need not block unrelated threads in the same process. Kernel scheduling can distribute threads across processors and enforce system-wide priorities and accounting.

The costs include kernel data structures, system calls or kernel involvement in management, and potentially more expensive creation and switching. Contemporary systems commonly use kernel-supported threads, while application runtimes may layer user-level tasks over them.

7. Multithreading Models

The relationship between user threads and kernel threads is described by several models.

Many-to-one

Many user threads map to one kernel thread. The model is inexpensive and permits user-defined scheduling, but one blocking system call can block all user threads and true parallelism is unavailable on multiple cores.

One-to-one

Each user thread maps to one kernel thread. This supports genuine parallelism and allows one thread to block without stopping others. The disadvantages are higher management costs and possible limits on the number of threads.

Many-to-many

Many user threads are multiplexed over a pool of kernel threads. The runtime can support many logical activities while the kernel provides parallel execution. The model is flexible but complex, particularly for scheduling, synchronization, signal behavior, and debugging.

Two-level model

A two-level design resembles many-to-many but permits selected user threads to be bound permanently to particular kernel threads. Binding is useful when an activity requires predictable kernel-level treatment, although it increases management complexity.

In practice, systems often combine one-to-one kernel threads with user-level task pools. A server may create a bounded number of worker threads and schedule thousands of lightweight tasks over them. This avoids creating an unbounded number of kernel threads while retaining multicore execution.

8. Signals and Asynchronous Events

A signal is a software-generated notification of an event. Signals may be generated by hardware exceptions, the kernel, another process, a terminal, or an application. Examples include requests to terminate, stop, continue, or handle an exceptional condition.

Signal handling generally involves:

1. generating or sending a signal;

2. delivering it to a process or, where supported, a particular thread;

3. applying the configured disposition; and

4. possibly invoking a handler or terminating, stopping, or ignoring execution.

A process may specify default handling, ignore a signal where permitted, or install a handler. Some signals cannot safely be caught or ignored because the operating system reserves them for essential control.

Signals are asynchronous, so they may arrive at an inconvenient point in program execution. Handlers should therefore perform minimal, well-defined work. Many library operations are not safe to call from a signal handler because they may hold internal locks or update shared state non-atomically. A safer pattern is often to record a simple notification and let the main event loop perform substantial processing.

In multithreaded programs, signal semantics require care. A process-directed signal may be delivered to an eligible thread, whereas a thread-directed signal targets a particular thread. Applications should document which thread handles signals and should avoid relying on unspecified delivery choices.

Signals are not a general replacement for reliable message queues. They carry limited information, may be coalesced depending on the signal type and operating system, and can be difficult to use for high-volume data transfer.

9. Thread-Level Parallelism

Thread-level parallelism (TLP) is the execution of independent or partially independent threads at the same time. It can improve throughput, responsiveness, or latency, but only when the workload contains sufficient independent work and synchronization costs are controlled.

Amdahl’s law provides a useful upper-bound model. If fraction `s` of a workload is inherently serial and the remaining fraction `1 − s` can be parallelized across `p` processors, the ideal speedup is:

`S(p) = 1 / [s + (1 − s)/p]`.

Here, `S(p)` is speedup relative to one processor, `s` is between 0 and 1, and `p` is the number of processors. The model assumes equal processor performance, perfect load balance, no parallelization overhead, and no additional effects such as memory contention. Real speedup is normally lower.

For example, if 20% of a workload is serial and 80% is parallelizable, using four processors gives:

`S(4) = 1 / [0.20 + 0.80/4] = 1 / 0.40 = 2.5`.

Although four processors are available, ideal speedup is only 2.5 because the serial portion limits progress.

Important practical barriers include:

• Data races: two threads access shared data concurrently and at least one access is a write, without adequate synchronization.
• Deadlock: threads wait indefinitely for one another.
• Starvation: a thread is repeatedly denied progress.
• Livelock: threads remain active but make no useful progress.
• False sharing: independent variables occupy the same cache line, causing unnecessary coherence traffic.
• Load imbalance: some workers finish early while others remain busy.
• Contention: threads compete for locks, memory bandwidth, I/O, or accelerator resources.

Synchronization tools include mutexes, semaphores, condition variables, read-write locks, barriers, atomic operations, and message passing. A sound design minimizes shared mutable state, establishes a consistent lock-ordering policy, uses bounded worker pools, and measures actual performance rather than assuming that more threads are better.

10. Processes and Threads in AI and ML Systems

AI workloads often combine several forms of concurrency. A data-loader may use threads for prefetching, a training service may use processes for isolation, and accelerator libraries may launch device kernels asynchronously. Processes are valuable when separate models or tenants require protection from memory corruption and incompatible dependencies. Threads are useful for sharing large in-memory datasets or coordinating I/O without copying data between processes.

However, the best choice depends on the runtime. Some language implementations use a global interpreter lock or similar mechanism that limits parallel execution of interpreter-managed code, although native numerical libraries may release such a lock. Processes can provide parallelism in that situation but may incur serialization and memory costs. GPU or accelerator parallelism is also distinct from CPU thread parallelism: launching more CPU threads does not automatically increase device throughput.

A production ML service should measure CPU utilization, queueing delay, memory bandwidth, accelerator utilization, context-switch rates, and tail latency. It should also define shutdown behavior so that in-flight requests, model state, and temporary data are handled safely.

11. Applied Case: A Multithreaded Model-Inference Service

The following is an illustrative case.

Background

A university deploys an image-classification API. An initial implementation creates one operating-system process for each request. The model is loaded separately in each process, producing high memory use and unacceptable startup latency.

Problem

The service must handle concurrent requests while protecting the model from accidental modification, maintaining predictable latency, and shutting down without losing accepted requests.

Analysis

Profiling shows that model loading is expensive, image decoding is partly I/O-bound, and inference uses a native numerical library that can exploit multiple cores. Creating an unlimited number of processes would duplicate memory and cause scheduling and cache contention. Creating an unlimited number of threads would increase queueing, memory use, and synchronization pressure.

Approach

The team adopts a supervisor process and a bounded worker pool. A small number of worker processes each loads a model once. Within each worker, a controlled number of threads performs input preparation and coordinates inference. Requests enter a bounded queue; when the queue is full, the service applies back-pressure rather than allocating unlimited work.

The service records request identifiers, uses timeouts, and handles termination cooperatively. On shutdown, the supervisor stops accepting new requests, allows a defined grace period for accepted requests, and then terminates workers that fail to exit. Shared configuration is read-only after initialization, reducing data races. Signal handling is centralized in the supervisor, which communicates shutdown instructions through explicit channels.

Outcome

Memory consumption becomes predictable because the number of model copies is bounded. Throughput improves because workers reuse loaded models and overlap input preparation with computation. Tail latency becomes more stable because the queue and worker count are controlled. The design does not assume that every additional thread improves performance; it is validated using load tests and resource measurements.

Learning points

The case illustrates that process isolation and thread sharing solve different problems. It also shows why concurrency must be designed together with resource limits, failure handling, observability, and shutdown semantics.

12. Challenges, Governance, and Best Practices

Concurrency introduces governance concerns as well as technical ones. A process or thread may handle personal data, credentials, or regulated workloads. Least-privilege credentials, memory isolation, audit logging, and controlled access to files and devices are therefore essential. A signal or termination request must not be used to bypass authorization or destroy evidence in a managed environment.

Recommended practices include:

• define ownership of every shared resource;
• prefer immutable data or message passing where practical;
• keep thread pools bounded and based on measured workload characteristics;
• document whether APIs are thread-safe and reentrant;
• use timeouts and cancellation policies deliberately;
• establish lock ordering to reduce deadlock risk;
• test under contention, failure, overload, and shutdown;
• use race detectors, sanitizers, profilers, and tracing tools;
• separate safety-critical control paths from best-effort background work; and
• monitor not only average throughput but also tail latency and resource saturation.

Emerging systems increasingly use user-space scheduling, asynchronous runtimes, lightweight tasks, heterogeneous processors, and container-based isolation. These technologies do not eliminate the underlying issues. They change where scheduling occurs and how resources are represented, making clear process and thread models even more important.

Chapter Summary

A process is an active program instance with an address space, resources, and execution state; a thread is a schedulable execution path within a process. Processes move among New, Ready, Running, Waiting, and Terminated states, with suspended states added in some systems. The process control block stores identity, state, CPU context, scheduling information, memory references, resources, and security data.

Context switching permits multiple execution contexts to share processors but consumes time and may disturb caches and address-translation state. Process creation and termination involve inheritance, program-image replacement, resource management, exit status, and safe cleanup. User-level threads offer flexible and inexpensive management, while kernel threads support blocking independence and multicore scheduling. Many-to-one, one-to-one, many-to-many, and two-level models represent different mappings between them.

Signals provide asynchronous notifications but require disciplined handling, especially in multithreaded programs. Thread-level parallelism can improve performance, yet serial work, synchronization, contention, load imbalance, and memory effects limit gains. Effective systems combine appropriate process isolation, bounded thread concurrency, explicit communication, measurement, and robust failure and shutdown policies.

Key Terms

Process: An active instance of a program.

Program: A passive set of instructions and data.

Thread: A schedulable execution unit within a process.

Process state: The current execution condition of a process.

Ready: Able to run but waiting for a processor.

Waiting/Blocked: Unable to proceed until an event occurs.

Process control block: Kernel structure containing process-management information.

Context switch: Change from one execution context to another.

Preemption: Removal of a running thread from a processor by the scheduler.

PID: Operating-system process identifier.

Fork: Operation that creates a child process in Unix-like systems.

Exec: Operation that replaces a process’s program image.

Zombie: Terminated child retaining exit-status information until collected.

User-level thread: Thread managed primarily by a runtime library.

Kernel-level thread: Thread known and scheduled by the operating system.

Signal: Asynchronous software notification of an event.

Data race: Conflicting unsynchronized concurrent accesses to shared data.

Deadlock: Indefinite waiting caused by circular resource dependence.

Thread-level parallelism: Concurrent execution of multiple threads.

Amdahl’s law: Model relating speedup to serial and parallel workload fractions.

Review Questions

Short Answer Questions

1. What is the difference between a program and a process?

2. List the five basic process states.

3. What information is stored in a process control block?

4. How does a user-level thread differ from a kernel-level thread?

5. Why can a signal handler be unsafe for general-purpose library operations?

Long Answer Questions

1. Explain the process-state model and illustrate the transitions caused by dispatching, preemption, blocking, and event completion.

2. Describe context switching, distinguish it from a mode switch, and discuss its performance costs.

3. Compare many-to-one, one-to-one, many-to-many, and two-level multithreading models.

4. Explain process creation and termination, including copy-on-write, program-image replacement, exit status, and zombies.

5. Discuss the benefits and risks of thread-level parallelism in a multicore AI or ML service.

Analytical / Application Questions

1. A workload is 15% serial and 85% parallelizable. Using the ideal Amdahl model, calculate the speedup with five processors and explain why it is less than five.

2. Design a process-and-thread architecture for a model-inference service that must limit memory use and provide graceful shutdown. Justify the worker counts, communication method, and signal strategy.

3. A multithreaded program shows high CPU utilization but little throughput improvement as threads are added. Analyze at least four possible causes and propose measurements or experiments to distinguish them.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a process from a thread?

A. A process is always smaller than a thread.

B. A process provides resource ownership and protection, while a thread is an execution unit within it.

C. A thread has its own complete address space, while a process does not.

D. Processes cannot contain more than one execution path.

Correct Answer: B. A process provides resource ownership and protection, while a thread is an execution unit within it.

Question 2

A running process requests blocking disk I/O. Which state transition normally occurs?

A. Running to Waiting

B. Running to New

C. Ready to Terminated

D. Waiting to Running

Correct Answer: A. Running to Waiting

Question 3

Which item is normally saved as part of a thread or process CPU context?

A. The source-code comments

B. The program counter

C. The user’s monitor resolution

D. The file contents of every open file

Correct Answer: B. The program counter

Question 4

What is the principal performance effect of a context switch?

A. It guarantees linear speedup.

B. It consumes overhead that does not directly execute application work.

C. It permanently increases the process priority.

D. It converts a process into a program.

Correct Answer: B. It consumes overhead that does not directly execute application work.

Question 5

What does an exec-style operation generally do in a Unix-like system?

A. Creates a second PID without changing the program image

B. Replaces the calling process’s program image

C. Converts a thread into a kernel

D. Terminates the parent and all unrelated processes

Correct Answer: B. Replaces the calling process’s program image

Question 6

Which multithreading model maps each user thread to a separate kernel thread?

A. Many-to-one

B. One-to-one

C. Many-to-many

D. Zero-to-one

Correct Answer: B. One-to-one

Question 7

Why can a purely many-to-one user-thread system fail to exploit multiple cores?

A. Its threads have no stacks.

B. The kernel sees only one schedulable execution context.

C. Its processes cannot perform I/O.

D. Its threads are all permanently terminated.

Correct Answer: B. The kernel sees only one schedulable execution context.

Question 8

If 20% of a workload is serial, what is the ideal speedup with four processors under Amdahl’s law?

A. 1.25

B. 2.00

C. 2.50

D. 4.00

Correct Answer: C. 2.50

Question 9

Which condition is a data race?

A. Two threads read immutable data concurrently.

B. A thread waits for a file event.

C. Two threads access shared data concurrently, with at least one write, without adequate synchronization.

D. A process exits normally.

Correct Answer: C. Two threads access shared data concurrently, with at least one write, without adequate synchronization.

Question 10

Why should signal handlers generally perform minimal work?

A. Signals can only be delivered to terminated processes.

B. Many library operations are not safe to call asynchronously and may use internal locks.

C. Handlers automatically run on every processor.

D. Signals always carry complete application messages.

Correct Answer: B. Many library operations are not safe to call asynchronously and may use internal locks.