7. Concurrency and Parallel Programming

Introduction and Context

A sequential Java program executes one instruction stream at a time. Concurrency changes this model by allowing multiple activities to make progress during overlapping periods, while parallelism executes multiple activities simultaneously, usually on different processor cores. The distinction matters: a concurrent program may run on a single core through task interleaving, whereas a parallel program requires simultaneous execution to obtain computational speed-up.

Modern software routinely performs several activities at once: a web server handles many clients, an application reads files while updating its user interface, and a data-processing system divides a large computation among processors. Java provides language constructs, memory-model rules, synchronization mechanisms, executor frameworks, concurrent collections, asynchronous abstractions, and virtual threads to support these requirements. These facilities increase performance and responsiveness, but they also introduce nondeterminism, visibility problems, race conditions, deadlocks, starvation, and cancellation difficulties.

Effective concurrent programming therefore requires more than creating threads. The programmer must define ownership of mutable state, establish safe communication, select an appropriate execution model, and reason about ordering guarantees. This chapter develops those abilities from the Java Memory Model to modern structured concurrency concepts.

Learning Objectives

After studying this chapter, a learner should be able to:

1. Explain thread states, lifecycle transitions, interruption, and termination.

2. Distinguish concurrency from parallelism and identify suitable task decomposition strategies.

3. Apply the Java Memory Model, happens-before relationships, synchronization, locks, and atomic variables.

4. Diagnose race conditions, deadlocks, livelocks, starvation, and visibility failures.

5. Select among raw threads, executors, futures, CompletableFuture, and virtual threads.

6. Use concurrent collections and non-blocking algorithms appropriately.

7. Design cancellation, error handling, and resource management policies.

8. Evaluate structured concurrency as a disciplined model for concurrent subtasks.

A Conceptual Model of Concurrent Execution

A thread is an independently schedulable path of execution within a process. Threads in the same Java Virtual Machine share heap memory and therefore can communicate efficiently, but shared memory also creates synchronization obligations. Each thread has its own stack, program counter, and local variables. Objects stored in the heap may be accessed by several threads unless their visibility and mutation are controlled.

Concurrency is best understood in terms of tasks rather than merely threads. A task is a unit of work, such as processing a request or calculating a subtotal. A thread is an execution resource that may run a task. Separating these concepts allows an application to use a bounded pool, queue tasks, apply back-pressure, and manage failure independently of the task’s business logic.

Parallelism may be data parallel or task parallel. Data parallelism applies the same operation to different portions of a dataset. Task parallelism performs different operations concurrently, such as fetching data, validating credentials, and constructing a response. Speed-up is limited by the sequential portion of a program, communication overhead, synchronization, and available hardware. Consequently, creating more threads does not necessarily improve performance.

Thread Lifecycle and Interruption

A Java thread moves through states represented by Thread.State: NEW, RUNNABLE, BLOCKED, WAITING, TIMED_WAITING, and TERMINATED. NEW describes a thread whose start method has not been invoked. RUNNABLE includes a thread that is ready to run and one currently executing; the JVM does not expose a separate RUNNING state. BLOCKED means that the thread is waiting to acquire an intrinsic monitor. WAITING indicates indefinite waiting for another action, commonly through join, wait, or park. TIMED_WAITING results from timed sleep, wait, join, or park. TERMINATED indicates completion or abrupt failure of the thread’s run method.

Calling start creates a new execution path. Calling run directly does not; it is an ordinary method invocation on the current thread. A thread should generally be allowed to terminate naturally. The deprecated stop, suspend, and resume operations are unsafe because they can leave shared objects in inconsistent states.

Interruption is a cooperative cancellation mechanism. Thread.interrupt sets an interruption status, or causes certain blocking methods to throw InterruptedException. Code that catches InterruptedException should normally restore the status with Thread.currentThread().interrupt() after performing necessary cleanup, unless it can complete cancellation at a higher level. Interruption is a request, not a guarantee that a thread will stop immediately.

The Java Memory Model: Visibility and Ordering

The Java Memory Model (JMM) specifies how threads interact through memory. It permits each thread to use cached or reordered values provided that the resulting behavior conforms to the model’s rules. A write performed by one thread is not automatically visible to another. Thus, a program can fail even when each individual statement appears correct.

The central relation is happens-before. If action A happens-before action B, the effects of A are visible to B and the ordering is respected. Important happens-before relationships include:

• Unlocking a monitor happens-before a later lock of the same monitor.
• A write to a volatile variable happens-before a subsequent read of that variable.
• A call to start on a thread happens-before actions in the started thread.
• All actions in a thread happen-before another thread successfully returns from join.
• Actions before submitting a task to an Executor happen-before that task begins execution.

A data race occurs when two conflicting accesses to the same variable are not ordered by happens-before and at least one access is a write. Data races can produce stale values, lost updates, and surprising instruction-order effects. The volatile modifier provides visibility and ordering for a variable, but it does not make compound operations such as count++ atomic. Volatile is appropriate for independent state flags or publication protocols, not for arbitrary invariants involving several fields.

Safe publication ensures that another thread sees a properly initialized object. It can be achieved through immutable objects, final fields used correctly, volatile references, static initialization, synchronized access, locks, or concurrent collections. Publishing a mutable object by placing it in an unsynchronized ordinary collection does not by itself establish safe visibility.

Mutual Exclusion, Intrinsic Locks, and Explicit Locks

The synchronized keyword provides mutual exclusion and memory visibility. A synchronized instance method locks the receiver; a synchronized static method locks the Class object; a synchronized block locks the specified object. Only one thread at a time may hold a given intrinsic lock. Exiting the block releases the lock even when an exception occurs.

For example:

```java

class Counter {

private int value;

public synchronized void increment() {

value++;

}

public synchronized int get() {

return value;

}

}

```

Both methods protect the same state with the same monitor. Synchronizing increment but not get would permit a visibility failure or an inconsistent observation.

The java.util.concurrent.locks package provides more flexible mechanisms. ReentrantLock supports explicit lock and unlock operations, tryLock, timed acquisition, interruptible acquisition, and optional fairness. The lock must be released in a finally block:

```java

lock.lock();

try {

updateState();

} finally {

lock.unlock();

}

```

ReadWriteLock separates shared reads from exclusive writes when reads substantially dominate writes and the protected operation is sufficiently expensive. StampedLock provides optimistic reads, but optimistic results must be validated and the mechanism is not reentrant. A lock is useful when a clearly defined invariant must be protected. It is not a substitute for a design that unnecessarily shares mutable state.

Condition objects allow a thread to wait for a logical condition associated with a lock. Waiting must occur in a loop because a wake-up does not prove that the condition is true:

```java

while (queue.isEmpty()) {

notEmpty.await();

}

```

The loop also handles spurious wake-ups and competition from other waiting threads.

Atomics and Lock-Free Coordination

Atomic classes such as AtomicInteger, AtomicLong, AtomicReference, and LongAdder support thread-safe operations without a conventional mutual-exclusion lock. They commonly rely on compare-and-set (CAS): a value is replaced only if it still equals an expected value. CAS is effective for counters, state transitions, and references to immutable snapshots.

Atomicity is not the same as invariants. Two separately atomic updates may still violate a relationship between variables. For highly contended counters, LongAdder can provide better throughput by distributing updates internally, although its value is not an instantaneous transactionally consistent snapshot. AtomicReference can implement immutable-state replacement, but careless retry loops may cause excessive contention or the ABA problem, in which a location changes from A to B and back to A. Where identity of intermediate changes matters, stamped or versioned references may be needed.

Coordination with Executors and Futures

Creating an unbounded number of platform threads can exhaust memory or overwhelm external services. ExecutorService separates task submission from thread management. Fixed pools bound concurrency; cached pools may create threads according to demand; scheduled executors support delayed and periodic work. The chosen queue, pool size, rejection policy, and shutdown procedure are part of system design rather than incidental configuration.

A Callable returns a result and may throw checked exceptions; its execution can be represented by Future. Future.get waits for completion and either returns the result or reports failure, cancellation, or interruption. Blocking on a future inside a small pool can cause thread starvation when the task needed to complete the future is waiting for a worker from the same pool.

Executor shutdown should be explicit. shutdown permits submitted work to finish, whereas shutdownNow attempts interruption and returns tasks that never commenced. Neither operation guarantees that arbitrary code will respond to interruption. Production systems need timeouts, cancellation policies, and resource cleanup.

CompletableFuture and Asynchronous Pipelines

CompletableFuture represents a result that may become available later and supports composition. thenApply transforms a successful result; thenCompose flattens an asynchronous operation; thenCombine joins independent results; exceptionally and handle provide failure processing. Methods ending in Async may use a default common pool or a supplied executor. Supplying an executor is often preferable when blocking I/O, workload isolation, latency control, or security boundaries matter.

For example:

```java

CompletableFuture<User> user = fetchUserAsync(id, ioPool);

CompletableFuture<Account> account = user.thenCompose(

u -> fetchAccountAsync(u.accountId(), ioPool));

CompletableFuture<Report> report = account.thenCombine(

fetchLimitsAsync(id, ioPool),

(a, limits) -> buildReport(a, limits));

```

Asynchronous composition does not automatically make blocking code non-blocking. A method that performs blocking database access still consumes a thread. Exceptions should be handled deliberately, and timeouts should be applied at meaningful boundaries. A pipeline should also specify what happens when one component fails: fail fast, use a fallback, retry, or return partial results.

Concurrent Collections and Thread-Safe Design

Concurrent collections provide defined coordination for common access patterns. ConcurrentHashMap supports concurrent retrieval and updates without locking the entire map. Its computeIfAbsent method is useful for atomic initialization, but the mapping function should be short and should not recursively modify the same map. CopyOnWriteArrayList is efficient when reads are frequent and writes are rare; each write creates a new backing array. BlockingQueue supports producer-consumer designs by coordinating insertion and removal. ConcurrentLinkedQueue is non-blocking and appropriate when producers and consumers should not wait for capacity.

A concurrent collection does not make a compound application operation atomic. The sequence “check whether key exists, then insert” may race unless expressed through an atomic collection method or protected by another mechanism. Immutable values, message passing, ownership transfer, and confinement often yield simpler designs than shared mutable collections.

Diagnosing Failure: Races, Deadlocks, and Starvation

A race condition occurs when the outcome depends on uncontrolled timing. A lost update in a shared counter is a basic example. A check-then-act sequence, lazy initialization, and iteration concurrent with mutation are common forms. Testing may fail to expose a race because scheduling varies across machines and runs. Code review should identify shared mutable state and its happens-before guarantees; stress testing and tools such as thread dumps and race-analysis tools can supplement review.

Deadlock occurs when threads wait forever for one another. The classic conditions are mutual exclusion, hold-and-wait, no pre-emption, and circular wait. Deadlock can be prevented by imposing a global lock order, acquiring all required locks together, using timed tryLock, reducing lock scope, or replacing nested locking with message passing. Lock ordering must be documented and consistently applied.

Starvation occurs when a thread is indefinitely denied progress, perhaps because of unfair scheduling or a continuously favored lock holder. Livelock occurs when threads remain active but repeatedly respond to one another without completing useful work. Excessive contention, priority misuse, blocking within asynchronous callbacks, and unbounded queues are additional hazards.

Virtual Threads and Structured Concurrency

Virtual threads are lightweight JVM-managed threads designed to support large numbers of mostly blocking tasks. They make a thread-per-request style practical for many I/O-bound services. They are not automatically faster for CPU-bound computation; CPU work remains limited by processor capacity. Blocking while holding a scarce resource, such as a database connection, can still exhaust that resource. Thread-local usage, synchronization patterns, native calls, and pinning considerations must be reviewed in virtual-thread applications.

Virtual threads change the economics of thread creation, not the rules of safe shared-state access. A race remains a race, and a deadlock remains a deadlock. Executors and structured task scopes remain valuable for lifecycle and failure management.

Structured concurrency treats concurrently forked subtasks as children of a lexical parent operation. The parent waits for, cancels, and observes those children as a group. This model prevents unfinished background tasks from escaping their request context and makes cancellation and error propagation explicit. In modern Java, structured concurrency has been developed as a preview API in recent releases, so exact classes and status depend on the JDK version. The underlying design principle—concurrency should have a clear lifetime and ownership boundary—is useful even when the API is unavailable.

An Applied Case: Parallel Product Recommendations

Background

An online retail service produces a product page by obtaining product information, inventory, customer recommendations, and delivery estimates. The service must respond quickly while several remote calls are potentially slow.

Problem

The first implementation creates a new platform thread for every request and uses a shared HashMap as a cache. It also waits synchronously for each remote call in sequence. Under peak traffic, thread count and memory usage rise sharply. The cache occasionally loses entries, and some requests remain blocked after one remote service becomes unavailable.

Analysis

The design combines three problems. First, unbounded thread creation provides no admission control. Second, HashMap is not safe for concurrent mutation. Third, sequential waiting makes total latency approximately the sum of all remote-call latencies. Failure and cancellation boundaries are also unclear.

Strategy and Solution

The service uses a bounded executor for CPU-intensive response assembly and virtual threads for request-oriented blocking I/O, subject to connection-pool limits. Independent calls are started concurrently and combined with CompletableFuture or an equivalent structured task scope. A ConcurrentHashMap uses computeIfAbsent for cache initialization, with an expiration policy and bounded storage. Each remote call has a timeout, and a failed recommendation service produces a documented fallback rather than blocking the entire response. If the primary product lookup fails, dependent calls are cancelled. Shared counters use LongAdder, while immutable response objects are safely published.

The service also records task latency, timeout counts, queue depth, cancellation rates, and downstream saturation. These metrics distinguish a concurrency defect from an overloaded dependency.

Outcome

The revised design reduces avoidable sequential delay, prevents uncontrolled thread growth, and gives failures a defined propagation path. It does not claim unlimited scalability: database connections, remote-service quotas, CPU, and memory remain capacity constraints.

Learning Points

Concurrency design begins with task relationships and resource limits, not with a choice of thread class. Safe collections solve collection-level coordination but not every business invariant. Timeouts, cancellation, ownership, and observability are essential parts of a production design.

Professional and Governance Considerations

Concurrent software can affect financial transactions, medical records, access control, and safety-related decisions. A lost update may become a data-integrity incident; a retry without idempotency may duplicate a payment. Critical operations should define transaction boundaries, idempotency keys, audit records, authorization checks, and recovery procedures. Sensitive data should not be exposed through unsafe logging of asynchronous exceptions or thread-local context.

Teams should document executor ownership, maximum concurrency, lock ordering, interruption behavior, timeout assumptions, and shutdown responsibilities. Code review should ask which state is shared, who owns it, what establishes visibility, and what happens when a task fails or is cancelled. Load tests should include slow dependencies and partial failure rather than only successful, low-contention execution.

Best-Practice Design Principles

Prefer immutable data and confinement before adding synchronization. Represent work as tasks and use an appropriately bounded execution policy. Establish visibility through documented happens-before mechanisms. Keep critical sections short and avoid blocking while holding locks. Use atomic collection operations for compound actions. Never ignore interruption without a deliberate reason. Avoid blocking the common pool with long-running or blocking work. Apply deadlines, cancellation, and back-pressure. Give every concurrent activity a clear owner and lifetime. Measure contention and downstream saturation instead of assuming that more parallelism means better performance.

Current Developments

Java’s concurrency direction emphasizes lightweight virtual threads, structured task lifetimes, scoped context, and clearer composition of asynchronous operations. These developments reduce the accidental complexity associated with callback-heavy designs, but they do not eliminate the need for memory-model reasoning. Future-ready designs should therefore separate business operations from execution policy, avoid reliance on undocumented scheduler behavior, and test against the supported JDK version.

Chapter Summary

Concurrency allows overlapping progress, while parallelism uses simultaneous execution. Java threads have defined lifecycle states, and interruption provides cooperative cancellation. The Java Memory Model explains visibility and ordering through happens-before relationships; volatile establishes visibility but does not make compound operations atomic. synchronized, explicit locks, conditions, and atomic classes provide different forms of coordination.

Executors separate task submission from thread management, Future represents eventual results, and CompletableFuture supports asynchronous composition and failure handling. Concurrent collections provide scalable operations for common patterns but do not automatically make multi-step business operations atomic. Race conditions, deadlocks, starvation, and livelock must be prevented through ownership, lock ordering, bounded resources, and careful cancellation. Virtual threads make large numbers of blocking tasks practical, while structured concurrency gives related subtasks a coherent lifetime. Reliable concurrent systems combine correctness, capacity control, observability, and explicit failure policies.

Key Terms

Happens-before: A Java Memory Model ordering relation that guarantees visibility and ordering of one action before another.

Data race: Conflicting unsynchronized accesses to the same variable where at least one access is a write.

Thread: An independently schedulable execution path within a process.

Concurrency: Overlapping progress of multiple tasks.

Parallelism: Simultaneous execution of tasks, generally on multiple cores.

Monitor: The intrinsic lock and associated waiting mechanism of a Java object.

Volatile: A variable modifier providing visibility and ordering for individual reads and writes.

Atomicity: Completion of an operation as an indivisible unit with respect to other threads.

CAS: Compare-and-set operation that conditionally changes a value if it equals an expected value.

ExecutorService: An abstraction for submitting tasks and managing their execution resources.

Future: A handle representing a possibly incomplete computation and its eventual result.

CompletableFuture: A composable representation of an asynchronous result.

ConcurrentHashMap: A concurrent map designed for safe, scalable access by multiple threads.

Deadlock: Permanent waiting caused by a cycle of resource dependencies.

Starvation: Indefinite denial of a thread’s opportunity to make progress.

Livelock: Repeated activity without useful progress.

Virtual thread: A lightweight JVM-managed thread intended to support large-scale concurrent tasks.

Structured concurrency: A model in which related subtasks have a bounded lexical lifetime and coordinated cancellation.

Safe publication: Making an object’s initialized state reliably visible to other threads.

Back-pressure: Regulation that prevents producers from overwhelming consumers or limited resources.

Review Questions

Short Answer Questions

1. Distinguish concurrency from parallelism with one Java-oriented example.

2. What is the difference between calling start and calling run on a Thread?

3. Why does volatile not make count++ safe?

4. State two important happens-before relationships.

5. Why should InterruptedException generally lead to restoration of interruption status?

Long Answer Questions

1. Explain the Java Memory Model and show how synchronized, volatile, and thread start/join establish visibility.

2. Compare intrinsic locks, ReentrantLock, ReadWriteLock, StampedLock, and atomic classes.

3. Describe the design and failure-handling features of ExecutorService and CompletableFuture.

4. Explain the causes of deadlock, starvation, and livelock and present prevention techniques.

5. Discuss virtual threads and structured concurrency, including their benefits and limitations.

Analytical / Application Questions

1. A program protects writes to a shared counter with synchronized but reads the counter without synchronization. Analyze the possible defect and propose two correct designs.

2. A fixed pool of four workers submits four tasks, each of which waits for another task submitted to the same pool. Explain the likely failure and redesign the execution strategy.

3. Design a concurrency architecture for a service that performs three independent remote calls, has a strict response deadline, and must cancel unnecessary work after one essential call fails. Identify execution resources, coordination mechanisms, and observability measures.

22. Multiple Choice Questions with Answers

Question 1

Which statement most accurately describes a happens-before relationship?

A. It guarantees that two actions execute simultaneously.

B. It guarantees visibility and ordering of the earlier action to the later action.

C. It guarantees that every operation is lock-free.

D. It guarantees that a task cannot be interrupted.

Correct Answer: B. It guarantees visibility and ordering of the earlier action to the later action.

Question 2

Why is volatile insufficient for safely implementing count++ shared by multiple threads?

A. Volatile variables cannot be read by other threads.

B. Volatile prevents all forms of memory visibility.

C. The read-modify-write sequence is not one atomic operation.

D. Volatile can only be used with reference types.

Correct Answer: C. The read-modify-write sequence is not one atomic operation.

Question 3

What happens when Thread.run() is called directly rather than Thread.start()?

A. A new operating-system thread is always created.

B. The method executes in the calling thread.

C. The method is placed automatically in an executor queue.

D. The JVM rejects the call with InterruptedException.

Correct Answer: B. The method executes in the calling thread.

Question 4

Which design most directly prevents deadlock when two locks may be needed by multiple threads?

A. Acquire the locks in a globally consistent order.

B. Use an unbounded queue for all tasks.

C. Make every variable volatile.

D. Increase the number of worker threads.

Correct Answer: A. Acquire the locks in a globally consistent order.

Question 5

Which collection is particularly appropriate when reads are frequent and writes are rare because writes create a new backing array?

A. ConcurrentHashMap

B. CopyOnWriteArrayList

C. ConcurrentLinkedQueue

D. ArrayDeque

Correct Answer: B. CopyOnWriteArrayList

Question 6

A task blocks on Future.get inside a small executor pool, while the task needed to complete that future is queued in the same pool. What problem may result?

A. Safe publication

B. Thread starvation or pool deadlock

C. False sharing only

D. Automatic lock downgrading

Correct Answer: B. Thread starvation or pool deadlock

Question 7

Which CompletableFuture operation is most suitable for composing an asynchronous function that itself returns a CompletableFuture?

A. thenCompose

B. thenApply

C. thenAccept

D. isDone

Correct Answer: A. thenCompose

Question 8

What is a principal benefit of virtual threads?

A. They make CPU-bound work independent of processor count.

B. They eliminate all synchronization requirements.

C. They support large numbers of mostly blocking tasks with low thread-management overhead.

D. They guarantee that remote services never become saturated.

Correct Answer: C. They support large numbers of mostly blocking tasks with low thread-management overhead.

Question 9

Which statement about interruption is correct?

A. It forcibly terminates any Java thread immediately.

B. It is a cooperative cancellation request that blocking methods may report.

C. It can only be delivered to virtual threads.

D. It automatically rolls back all shared-state changes.

Correct Answer: B. It is a cooperative cancellation request that blocking methods may report.

Question 10

What is the central idea of structured concurrency?

A. Every task should run in the global common pool.

B. Related subtasks should have a bounded lifetime and coordinated failure and cancellation.

C. Locks should be nested to guarantee completion.

D. All asynchronous work should ignore parent-request cancellation.

Correct Answer: B. Related subtasks should have a bounded lifetime and coordinated failure and cancellation.