6. Processes, Services and Resource Control

Introduction and Context

A Linux system is not merely a collection of files and commands. It is a dynamic execution environment in which programs compete for processor time, memory, storage bandwidth, devices, and network resources. The operating system creates and manages processes, coordinates their execution, provides services through background daemons, records events in logs, and applies controls that prevent one workload from overwhelming others.

This chapter examines process management as a unified subject. Process states and signals explain how execution is controlled; scheduling explains how processor time is distributed; daemons and systemd explain how services are started and supervised; timers and logs provide automation and observability; namespaces and control groups establish isolation and resource governance. These mechanisms are important both to system administrators and to software engineers deploying reliable applications.

A useful distinction is between isolation and allocation. Namespaces determine what a process can see—for example, which processes or network interfaces are visible. Control groups, or cgroups, determine how much of a resource a group may consume. Modern containers depend on both mechanisms, although neither one alone constitutes a complete security boundary.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Explain Linux process creation, identity, states, relationships, and termination.

2. Interpret process information and use signals safely.

3. Describe scheduling classes, priorities, and practical CPU allocation.

4. Distinguish foreground processes, daemons, services, and systemd units.

5. Create and analyze systemd services and timers.

6. Use journald and related tools for operational diagnosis.

7. Explain namespaces and cgroups and their role in containers.

8. Apply CPU, memory, and I/O controls to realistic workloads.

9. Analyze operational, security, and governance risks in resource management.

Processes: the Executable Unit of Linux

A process is an instance of a running program together with its address space, registers, open files, credentials, environment, and kernel-managed metadata. The program is passive code stored on disk; the process is the active execution of that code. Each process has a process identifier (PID), while its parent process is identified by a parent PID (PPID). The kernel also tracks user and group ownership, process group membership, session membership, and scheduling attributes.

Linux commonly creates processes through the `fork()` or `clone()` system calls, followed frequently by `execve()`, which replaces the calling process image with a new program. A shell illustrates this sequence: it creates a child process, the child executes the requested command, and the parent shell waits for completion unless the command is placed in the background. Modern applications may use more specialized process-creation interfaces, but the conceptual distinction between creating a process and loading a new program remains important.

A process may be described through several relationships. A process group supports job control, allowing a shell to send a signal to a pipeline as a unit. A session groups related process groups, particularly for terminal management. A thread is an execution context within a process; threads share the process address space and many resources but possess individual scheduling state and thread identifiers. The `/proc` pseudo-filesystem exposes much of this information. For example, `ps`, `top`, `htop`, and files such as `/proc/<pid>/status` provide different views of process activity.

Process States and Transitions

The kernel represents execution using states that describe whether a task is running, waiting, stopped, or terminating. The exact display symbols vary slightly among tools, but the principal states are:

• Running or runnable (R): executing on a CPU or ready to be scheduled.
• Interruptible sleep (S): waiting for an event and able to respond to most signals.
• Uninterruptible sleep (D): generally waiting for kernel I/O or another critical operation. A process in this state may not respond immediately to signals.
• Stopped (T): suspended by job-control activity or a stopping signal.
• Zombie (Z): terminated, but its parent has not yet collected its exit status.

A zombie has released most resources but retains a process-table entry. It cannot be killed because it is already dead; the parent must call a waiting operation such as `wait()` or `waitpid()`. If the parent terminates, an ancestor process—normally a system service manager—can adopt the child and collect its status. A persistent accumulation of zombies indicates defective child-process management.

The state label alone is insufficient for diagnosis. A process in sleeping state may be perfectly healthy because it is waiting for a request. Conversely, a process consuming CPU continuously may be inefficient, malicious, or simply performing legitimate computation. Diagnosis requires correlating state, CPU time, memory use, open files, system calls, logs, and application behavior.

Signals and Controlled Termination

A signal is a small asynchronous notification delivered by the kernel or another authorized process. Signals are used for termination, interruption, stopping, continuation, and selected application-defined events. Common examples include `SIGTERM` (15), a request for orderly termination; `SIGKILL` (9), immediate kernel-enforced termination; `SIGINT` (2), conventionally generated by an interrupt such as Ctrl-C; `SIGHUP`, historically associated with terminal disconnection and often used to request configuration reload; and `SIGSTOP` and `SIGCONT`, which stop and resume execution.

The normal administrative sequence is to send `SIGTERM`, allow the application to close files and release resources, observe whether it exits, and use `SIGKILL` only when necessary. `SIGKILL` cannot be caught or handled, so it prevents cleanup routines from running. The command `kill` sends a signal to a PID, while `pkill` and `killall` select processes by attributes such as name. Selection commands require care: a broad pattern can terminate unrelated workloads.

Signals do not solve every process-management problem. A process blocked in uninterruptible kernel sleep may remain present until the underlying operation completes. In addition, a service may automatically restart after termination, making it appear that a signal had no effect. In such cases, the service manager, parent process, kernel wait condition, and relevant logs must be examined.

Scheduling and CPU Allocation

Linux scheduling determines which runnable task receives a CPU and for how long. The general-purpose Completely Fair Scheduler model seeks to distribute processor time fairly among ordinary tasks, using virtual runtime and task weights derived partly from the nice value. A lower nice value gives a task greater weight and therefore a larger relative share; increasing the nice value lowers its preference. The commands `nice` and `renice` adjust this attribute, subject to privilege restrictions.

Scheduling policy is distinct from priority. Ordinary tasks normally use `SCHED_OTHER`, while real-time policies such as `SCHED_FIFO` and `SCHED_RR` can give selected tasks precedence over ordinary work. Real-time scheduling can be essential for bounded-latency workloads, but careless use may starve the operating system and other applications. It should be combined with explicit limits and careful testing.

CPU affinity restricts a process or thread to selected logical CPUs. Affinity may improve cache locality, isolate latency-sensitive work, or reserve processors for particular workloads. However, pinning tasks unnecessarily can reduce the scheduler's flexibility and produce idle CPUs alongside overloaded ones. Tools such as `taskset`, `sched_setaffinity()`, and systemd's `CPUAffinity=` implement affinity controls.

CPU utilization must be interpreted with context. A high value may be expected for batch computation, whereas latency, run-queue length, throttling, context switches, and application response time may reveal the real problem. In a cgroup-based environment, CPU quota can limit aggregate runtime, while CPU weight determines relative competition when groups contend.

Services, Daemons, and systemd

A daemon is a background process that provides a continuing function, often without direct terminal interaction. Examples include logging, secure remote access, name resolution, scheduling, and web serving. A service is the operational function supplied by one or more processes. The terms overlap but are not identical: a service may be implemented by a daemon, a socket-activated process, a kernel facility, or a transient task.

On many contemporary Linux distributions, systemd is the service manager and, commonly, the first userspace process with PID 1. It initializes the system, starts units according to dependencies, supervises services, captures status and output, manages sockets and devices, and integrates with cgroups. A systemd unit is a managed object described by a unit file. Important unit types include `.service`, `.socket`, `.timer`, `.mount`, `.target`, `.path`, and `.slice`.

A service unit describes how a program is launched and supervised. Typical directives include:

```ini

[Unit]

Description=Example worker

After=network-online.target

[Service]

Type=simple

ExecStart=/usr/local/bin/example-worker

Restart=on-failure

User=worker

NoNewPrivileges=yes

[Install]

WantedBy=multi-user.target

```

`ExecStart` specifies the command, `User` reduces privilege, and `Restart=on-failure` provides limited recovery. `After=` expresses ordering, not necessarily a dependency; `Requires=` or `Wants=` expresses dependency relationships with different strength. A unit is enabled to start automatically at boot, while starting it immediately is a separate operation. Typical commands include `systemctl start`, `stop`, `restart`, `enable`, `disable`, `status`, and `daemon-reload`.

A robust service definition should specify a stable user, working directory, environment policy, restart behavior, timeout policy, resource controls, and security restrictions. Applications should generally log to standard output or error when managed by systemd, allowing the journal to collect records without requiring ad hoc daemonization code.

Timers and Operational Automation

A systemd timer is a unit that activates another unit, usually a service. Unlike a traditional cron entry, a timer is represented as a manageable object and can participate in dependencies, logging, resource controls, and missed-run policies. `OnCalendar=` schedules wall-clock events; `OnBootSec=` and `OnUnitActiveSec=` schedule relative intervals. `Persistent=true` can cause a missed calendar activation to run after the system becomes available again.

Timers are appropriate for maintenance, reports, cache cleanup, certificate checks, and periodic synchronization. They should be designed to be idempotent: running the task twice should not corrupt data or produce an unintended cumulative effect. Concurrency controls, timeouts, lock files, and clear failure logging are important in production environments.

Logs as Evidence of Process Behavior

Operational diagnosis depends on evidence. The systemd journal stores structured records from services, the kernel, and other sources. `journalctl -u service-name` filters by unit; `journalctl -b` restricts results to the current boot; `journalctl -f` follows new entries; and priority, time, identifier, and boot filters support more focused analysis. `systemctl status` provides a concise view of state, recent logs, the main PID, and exit information.

Logs should be correlated with resource measurements. A service restart may be caused by a configuration error, an out-of-memory event, a failed dependency, or an administrator action. Kernel messages can reveal memory pressure, device errors, or cgroup enforcement. Journal retention, access permissions, rate limiting, time synchronization, and forwarding to centralized systems are governance concerns rather than mere convenience features.

Namespaces: Controlling Visibility

Linux namespaces isolate particular views of system resources. A process in a PID namespace sees a separate process-number hierarchy; a mount namespace has its own filesystem mount view; a network namespace has interfaces, routes, sockets, and firewall context of its own. User namespaces map user and group identifiers and can allow a process to possess capabilities inside a namespace without having the same authority on the host. UTS, IPC, and cgroup namespaces isolate hostname, interprocess communication, and cgroup views respectively.

Namespaces are foundational to containers, but isolation is selective. A process may be isolated from the host process list while still consuming host CPU and memory. Privileged capabilities, vulnerable kernel interfaces, mounted host paths, device access, and unsafe configuration can weaken isolation. Therefore, namespaces must be combined with least privilege, capability reduction, seccomp policies, read-only filesystems where possible, and careful device and network configuration.

Cgroups and Resource Governance

A control group organizes processes hierarchically so that resource usage can be measured and constrained. The current cgroup v2 model provides a unified hierarchy and controllers such as CPU, memory, I/O, and process-count control. Systemd places services into cgroups automatically, making service-level accounting and enforcement practical.

CPU controls commonly include a relative weight and a maximum quota. Weight matters when groups compete; quota imposes a ceiling over a period. Memory controls include a high threshold that signals sustained pressure and a maximum that limits usage. Exceeding a memory limit may invoke cgroup-level reclaim or an out-of-memory kill. Memory accounting includes more than anonymous application data: page cache, kernel-related allocations, and shared pages may affect observed values depending on the controller and measurement interface.

I/O controls can assign weights or limits to block-device traffic, although actual behavior depends on the storage stack and device capabilities. Process-count limits prevent fork bombs and uncontrolled worker creation. In systemd units, directives such as `CPUQuota=`, `MemoryMax=`, `MemoryHigh=`, `IOWeight=`, `TasksMax=`, and `LimitNOFILE=` express policy. Limits should be validated under realistic workloads; overly strict values can cause failures that resemble application defects.

Applied Scenario: Protecting a Multi-Service Web Host

Background

Consider an illustrative organization hosting a public API, a background image-processing worker, and a nightly data-export task on one Linux server. The API requires predictable latency, whereas image processing is CPU-intensive and the export can generate substantial disk traffic.

Situation and Analysis

During peak traffic, API latency increases. Process inspection shows image workers consuming most CPUs, while export activity produces high I/O wait. Service logs show no application exception, and ordinary CPU utilization appears near saturation. The central issue is not simply that one process is “using too much CPU”; multiple workloads are competing for different shared resources, and the interactive service lacks explicit priority and limits.

Approach

The administrator places each workload in a separate systemd service and assigns stable service users. The API receives an appropriate CPU weight and a conservative memory maximum. Image processing receives a CPU quota so that it cannot consume the entire machine. The export service is scheduled by a timer and receives an I/O weight suitable for background work. `TasksMax=` limits accidental worker explosion. The services use `Restart=on-failure`, while logs are collected through journald and monitored centrally.

The administrator then tests normal load, burst load, memory pressure, timer overlap, and service restart behavior. CPU and memory settings are adjusted from observed latency and failure data rather than arbitrary percentages.

Outcome

API latency becomes more predictable, although total throughput for image processing declines slightly. The export continues to run but yields resources during interactive demand. The organization gains clearer ownership, auditability, and evidence for capacity planning.

Learning Points

Resource control is a policy problem as well as a technical problem. Relative weights alone cannot guarantee latency, quotas can create throttling, and memory limits can trigger abrupt termination. Effective design combines isolation, allocation, observability, testing, and explicit service ownership.

Challenges, Risks, and Professional Governance

The most common mistake is killing processes without understanding their parent service or restart policy. Another is confusing high memory usage with a leak: cached data may be reclaimable, whereas steadily increasing resident memory may indicate a defect. Zombie processes are also misdiagnosed because `kill -9` cannot remove them. Resource limits may create cascading failures when a service is configured below its legitimate peak requirement.

Security requires least privilege in both process and service design. Services should avoid unnecessary root execution, restrict capabilities, use private temporary directories where appropriate, and protect credentials from command lines and world-readable environment files. Administrative actions should be authorized, logged, and reviewed. Resource exhaustion is also a security concern: fork bombs, memory exhaustion, log flooding, and I/O saturation can reduce availability even when confidentiality is not affected.

From a governance perspective, organizations should define service ownership, retention policies for logs, change-control procedures for unit files, incident escalation paths, and recovery objectives. Automated restarts should not conceal repeated failures; alerting must distinguish successful recovery from an unhealthy restart loop.

Current Developments and Best Practices

Contemporary Linux deployments increasingly use cgroup v2, rootless containers, declarative service definitions, and orchestration platforms that translate workload policies into namespaces and cgroups. Resource-aware scheduling is becoming more important as systems consolidate heterogeneous workloads, including latency-sensitive services, batch analytics, and machine-learning inference.

Recommended practice includes:

• inspect before acting: identify the PID, owner, parent, state, and service unit;
• prefer graceful termination and controlled restart policies;
• use systemd units instead of informal background processes for long-running services;
• set explicit users, dependencies, timeouts, and resource limits;
• make timers idempotent and protect against overlapping execution;
• correlate journal records with CPU, memory, I/O, and process-count metrics;
• test limits under failure and peak-load conditions;
• treat namespaces as isolation mechanisms, not automatic security guarantees;
• review capabilities, devices, mounts, and host integrations for containerized workloads;
• document ownership and audit changes to operational policy.

Chapter Summary

Linux manages active computation through processes and threads, each with identity, relationships, state, credentials, and resource requirements. Process states reveal whether execution is active, waiting, stopped, or awaiting collection, while signals provide controlled mechanisms for interruption and termination. Scheduling policies, nice values, affinity, and real-time priorities influence CPU allocation, but performance diagnosis requires more than observing utilization.

Daemons provide continuing background functions, and systemd organizes them as units with dependencies, supervision, logging, timers, and cgroup integration. Namespaces limit what processes can see, whereas cgroups measure and control what groups can consume. CPU quotas and weights, memory thresholds and limits, I/O policies, and task-count restrictions allow administrators to express service-level resource governance. Reliable Linux operations therefore depend on the combination of process knowledge, service management, observability, isolation, allocation, security, and tested policy.

Key Terms

Process: An active instance of a program managed by the kernel.

PID: A unique process identifier within a PID namespace.

PPID: The identifier of a process's parent.

Thread: An execution context sharing an address space with other threads in a process.

Daemon: A background process that provides an ongoing system or application function.

Signal: An asynchronous notification delivered to a process or process group.

Zombie: A terminated process whose exit status has not been collected by its parent.

Nice value: A scheduling attribute influencing the relative weight of an ordinary task.

CPU affinity: A restriction specifying which logical CPUs may execute a task.

systemd: A Linux initialization and service-management system.

Unit: A systemd-managed object such as a service, timer, socket, or mount.

Timer: A systemd unit that activates another unit according to a schedule.

Journal: A structured log managed by systemd-journald.

Namespace: A kernel isolation mechanism providing a distinct view of selected resources.

Cgroup: A hierarchical mechanism for accounting and controlling groups of processes.

CPU quota: A maximum amount of CPU time available to a cgroup over a period.

Memory pressure: A condition in which available memory is insufficient for demand.

I/O weight: A relative preference used when groups compete for block-device I/O.

Capability: A distinct unit of Linux privilege that can be granted or removed independently.

Review Questions

Short Answer Questions

1. What is the difference between a program and a process?

2. Why can a zombie process not be removed with `SIGKILL`?

3. Distinguish a daemon from a systemd service unit.

4. What is the purpose of a PID namespace?

5. Why should `SIGTERM` normally be attempted before `SIGKILL`?

Long Answer Questions

1. Explain Linux process states and discuss how each state contributes to diagnosis.

2. Describe Linux scheduling, including nice values, CPU affinity, and real-time policies.

3. Explain the structure and operational role of systemd units, dependencies, and timers.

4. Compare namespaces and cgroups in terms of isolation, allocation, and containerization.

5. Discuss how logs and resource metrics should be combined during service failure analysis.

Analytical / Application Questions

1. A service repeatedly restarts after receiving `SIGTERM`. Develop a systematic investigation using process information, systemd status, dependencies, and journal records.

2. Design a resource policy for an API, a CPU-intensive batch worker, and a disk-intensive backup task sharing one host. Justify CPU, memory, I/O, and scheduling choices.

3. A container cannot see host processes but can still exhaust host memory. Explain why this occurs and propose namespace, cgroup, and security controls to reduce the risk.

22. Multiple Choice Questions with Answers

Question 1

A process is shown in zombie state. Which action is most appropriate?

A. Send SIGKILL repeatedly to the zombie

B. Identify and correct the parent’s failure to collect its exit status

C. Increase the process nice value

D. Move the process to a network namespace

Correct Answer: B. Identify and correct the parent’s failure to collect its exit status

Question 2

Why is SIGTERM generally preferred before SIGKILL?

A. SIGTERM has higher scheduling priority

B. SIGTERM allows the application to perform orderly cleanup

C. SIGTERM can terminate kernel threads only

D. SIGTERM bypasses service-manager restart policies

Correct Answer: B. SIGTERM allows the application to perform orderly cleanup

Question 3

Which statement best distinguishes namespaces from cgroups?

A. Namespaces allocate CPU, while cgroups isolate filesystems

B. Namespaces control visibility, while cgroups control accounting and resource limits

C. Both mechanisms perform exactly the same function

D. Cgroups are used only for logging

Correct Answer: B. Namespaces control visibility, while cgroups control accounting and resource limits

Question 4

A systemd timer is primarily used to:

A. Replace the Linux scheduler

B. Activate another unit according to a schedule

C. Create a new user namespace

D. Increase a process’s memory allocation automatically

Correct Answer: B. Activate another unit according to a schedule

Question 5

Which directive most directly imposes a maximum memory boundary on a systemd service in cgroup v2 environments?

A. MemoryMax=

B. CPUWeight=

C. After=

D. Restart=

Correct Answer: A. MemoryMax=

Question 6

Two ordinary workloads compete continuously for CPU. Which control primarily expresses their relative share rather than an absolute ceiling?

A. CPU weight

B. MemoryMax

C. TasksMax

D. SIGSTOP

Correct Answer: A. CPU weight

Question 7

A process in uninterruptible sleep most commonly indicates that it is:

A. Waiting in a kernel operation such as I/O

B. Already terminated and awaiting collection

C. Running with real-time priority

D. Executing in a stopped terminal session

Correct Answer: A. Waiting in a kernel operation such as I/O

Question 8

Which command is most suitable for examining recent logs belonging to a particular systemd service?

A. journalctl -u service-name

B. nice service-name

C. taskset -p service-name

D. renice -u service-name

Correct Answer: A. journalctl -u service-name

Question 9

Why can an unrestricted background worker harm an API even when the worker is functioning correctly?

A. It may compete for shared CPU, memory, or I/O resources

B. It automatically disables all namespaces

C. It converts all API processes into zombies

D. It prevents the kernel from recording logs

Correct Answer: A. It may compete for shared CPU, memory, or I/O resources

Question 10

Which practice is most appropriate for a recurring maintenance timer?

A. Make the task idempotent and protect against overlapping runs

B. Always run it as root without limits

C. Use SIGKILL as its normal completion method

D. Disable logging to reduce resource use

Correct Answer: A. Make the task idempotent and protect against overlapping runs