Concurrency and Asynchronous Programming

Introduction

A sequential Java program performs one instruction path at a time. Concurrency allows multiple tasks to make progress during overlapping periods, while parallelism allows tasks to execute simultaneously on multiple processors or cores. Asynchronous programming separates the initiation of an operation from the moment its result becomes available. These ideas are essential for responsive applications, high-throughput servers, data-processing systems, and interactive software.

Concurrency improves resource utilisation but introduces risks. Threads may interfere with one another, operations may occur in unexpected orders, and failures may be difficult to reproduce. Professional Java development therefore requires both knowledge of concurrency mechanisms and disciplined design practices.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish processes, threads, concurrency, parallelism, and asynchronous execution;
• create and manage tasks using Java threads and executors;
• explain race conditions, visibility, atomicity, and ordering;
• apply synchronization, locks, and concurrent collections appropriately;
• recognise and prevent deadlock, starvation, and livelock;
• use Future, CompletableFuture, and structured task composition;
• explain the role and limitations of virtual threads;
• design concurrent components that are safe, observable, and maintainable.

Processes, Threads, and Execution Models

A process is an operating-system unit containing an address space, resources, and one or more threads. A thread is an execution path within a process. Threads in the same process share heap memory and other resources, although each thread has its own stack and program counter.

Concurrency is a property of program structure: several tasks are in progress. Parallelism is a property of execution: several tasks run at the same instant. A single-core machine can provide concurrency through time-sharing but cannot execute two instructions simultaneously on one core.

A useful performance principle is Amdahl’s law. If fraction p of a workload can be parallelised and N processors are available, the ideal speedup is:

Here, S(N) is speedup relative to one processor, p is between 0 and 1, and N is the number of processors. The model assumes negligible communication and scheduling overhead. Real systems usually achieve less speedup because of contention, memory bandwidth, coordination, and unequal task sizes. If p = 0.9 and N = 4, the ideal speedup is 1 / (0.1 + 0.225) ≈ 3.08, not four times.

Creating Tasks and Threads

Modern Java code should normally express work as a Runnable or Callable, then delegate execution to an executor. A Runnable returns no value and cannot declare a checked exception. A Callable<V> returns a value of type V and may throw an exception.

Runnable task = () -> System.out.println("Running");
Thread thread = new Thread(task);
thread.start();       // schedules a new execution path
// thread.run();      // ordinary method call; does not create a new thread

Calling start() causes the JVM to arrange execution of run() on a new thread. A thread should not be started more than once. join() allows one thread to wait for another to terminate, while interrupt() requests interruption; it does not forcibly kill the thread.

Interruptible code should preserve the interruption signal:

try {
    Thread.sleep(1000);
} catch (InterruptedException ex) {
    Thread.currentThread().interrupt();
    return;
}

An interruption is a cooperative cancellation mechanism. Well-designed tasks periodically check interruption and release resources before terminating.

The Java Memory Model

Concurrent correctness is not only about the order in which statements appear in source code. The Java Memory Model defines when one thread’s writes become visible to another and which reorderings are permitted.

Three concepts are central:

• Atomicity: an operation appears indivisible. A read or write of a variable may be atomic, but a compound action such as count++ consists of read, addition, and write.
• Visibility: a thread observes a value written by another thread.
• Ordering: operations occur in an order constrained by the happens-before relationship.

Synchronization, entering and leaving a monitor, completing a task before Future.get(), and correctly using volatile can establish happens-before relationships. volatile ensures visibility and ordering for a variable, but it does not make compound operations atomic.

class UnsafeCounter {
    private int value;
    void increment() { value++; }
    int get() { return value; }
}

If two threads execute value++, both may read the same old value and overwrite one another’s update. The result can be lower than the number of increments. This is a race condition: the outcome depends on timing between conflicting operations.

A safe alternative is an atomic class:

import java.util.concurrent.atomic.AtomicInteger;

class SafeCounter {
    private final AtomicInteger value = new AtomicInteger();
    void increment() { value.incrementAndGet(); }
    int get() { return value.get(); }
}

Atomic classes are effective for independent state transitions. They are not automatically sufficient when several variables must change as one consistent unit.

Synchronization and Locks

The synchronized keyword protects a critical section using an object’s intrinsic monitor.

class Account {
    private int balance;

    public synchronized void deposit(int amount) {
        if (amount < 0) throw new IllegalArgumentException();
        balance += amount;
    }

    public synchronized int balance() {
        return balance;
    }
}

Only one thread at a time can execute synchronized methods guarded by the same object. Synchronization also provides visibility guarantees when entering and leaving the monitor.

A synchronized block can reduce the protected region:

synchronized (lock) {
    sharedState.update();
}

The java.util.concurrent.locks package provides more flexible locks. ReentrantLock supports timed acquisition, interruptible acquisition, and multiple condition variables.

Lock lock = new ReentrantLock();
try {
    lock.lockInterruptibly();
    // protected operation
} finally {
    lock.unlock();
}

The finally block is essential. Failure to unlock can permanently block other threads. A read-write lock may allow multiple simultaneous readers while excluding writers, but it is beneficial only when the workload and contention pattern justify its complexity.

Coordination Mechanisms

CountDownLatch allows one or more threads to wait until a count reaches zero. It is suitable for one-time startup or completion coordination. CyclicBarrier allows a fixed group of threads to meet repeatedly at a barrier. Semaphore limits the number of simultaneous users of a resource.

Blocking queues provide a powerful producer-consumer design:

BlockingQueue<String> queue = new ArrayBlockingQueue<>(100);

// producer
queue.put("job");

// consumer
String job = queue.take();

The bounded capacity provides backpressure: producers cannot add unlimited work when consumers are slow. This protects memory and makes overload visible rather than silently accumulating tasks.

Executors and Thread Pools

Creating a new platform thread for every request can exhaust operating-system resources. An executor separates task submission from thread management.

ExecutorService executor = Executors.newFixedThreadPool(4);
try {
    executor.submit(() -> processRequest());
} finally {
    executor.shutdown();
}

Common executor strategies include:

• newFixedThreadPool: a bounded number of worker threads, useful for controlled CPU or service workloads;
• newSingleThreadExecutor: serial execution with executor management;
• newScheduledThreadPool: delayed and periodic tasks;
• ForkJoinPool: divide-and-conquer tasks and work stealing;
• Executors.newVirtualThreadPerTaskExecutor(): one virtual thread per submitted task, available in modern Java versions supporting virtual threads.

Executors must be shut down. shutdown() stops accepting new tasks while allowing submitted tasks to finish. shutdownNow() requests interruption of tasks that have not completed; it cannot guarantee immediate termination.

Pool sizing is workload-dependent. CPU-bound tasks generally benefit from a number of workers near the available processor count. Blocking I/O workloads may need more concurrent tasks, but excessive threads create memory, scheduling, and downstream-service pressure. Measurement and load testing are preferable to an arbitrary fixed formula.

Futures and Asynchronous Composition

A Future<V> represents a result that may become available later.

Future<String> future = executor.submit(() -> loadDocument());
String document = future.get();

get() blocks until completion and may throw InterruptedException or ExecutionException. A timeout limits waiting:

String result = future.get(2, TimeUnit.SECONDS);

CompletableFuture supports non-blocking composition and explicit error handling.

CompletableFuture<String> user =
    CompletableFuture.supplyAsync(() -> loadUser(), executor);

CompletableFuture<String> message = user
    .thenApply(u -> "Welcome " + u)
    .exceptionally(ex -> "Unavailable");

Important operations include thenApply for transforming a result, thenCompose for flattening a dependent asynchronous operation, and thenCombine for combining independent operations.

CompletableFuture<Order> order = loadOrderAsync();
CompletableFuture<Customer> customer = loadCustomerAsync();

CompletableFuture<Invoice> invoice =
    order.thenCombine(customer, (o, c) -> createInvoice(o, c));

Asynchronous code is not automatically faster. If the caller immediately invokes join() after every submission, the design may merely obscure blocking. Benefits arise when independent operations overlap, resources are used efficiently, and completion stages are composed without unnecessary waiting.

Concurrent Collections

Standard collections such as ArrayList and HashMap are not generally safe for concurrent mutation. Options include:

• ConcurrentHashMap for highly concurrent map access;
• CopyOnWriteArrayList for many reads and rare writes;
• ConcurrentLinkedQueue for non-blocking queue operations;
• BlockingQueue when producers and consumers must coordinate;
• ConcurrentSkipListMap for concurrent sorted-map operations.

Concurrent collections provide stronger guarantees than wrapping a collection with a single external lock, but compound workflows still require care. This operation is unsafe if the map can change between the test and insertion:

if (!map.containsKey(key)) {
    map.put(key, value);
}

Use an atomic map operation instead:

map.putIfAbsent(key, value);

For aggregation, methods such as compute, computeIfAbsent, and merge express the intended atomic update more clearly.

Deadlock, Starvation, and Livelock

A deadlock occurs when threads wait indefinitely for one another. The classic conditions are mutual exclusion, hold-and-wait, no pre-emption, and circular wait. Consider two transfers acquiring account locks in opposite order:

// Thread 1: lock A, then B
// Thread 2: lock B, then A

A reliable prevention strategy is global lock ordering: assign each account a stable order and always acquire the lower-ranked lock first. Other strategies include timed tryLock, reducing lock scope, and avoiding nested locks.

Starvation occurs when a thread is continually denied access to resources. Livelock occurs when threads remain active but repeatedly respond to one another without completing useful work. Fair locks, bounded waiting, backoff, and simpler ownership protocols can reduce these problems.

Deadlock prevention should be complemented by diagnosis. Thread dumps, Java Flight Recorder, profilers, and executor metrics can reveal blocked threads, queue growth, long tasks, and lock contention. Logging should include request or task identifiers so that interleaved events can be reconstructed.

Virtual Threads

Virtual threads are lightweight threads managed by the Java runtime rather than directly corresponding one-for-one with operating-system threads. They are designed particularly for applications with many tasks that spend substantial time waiting for network or file I/O.

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    Future<String> f = executor.submit(() -> callRemoteService());
    System.out.println(f.get());
}

Virtual threads make thread-per-request programming practical at large scales, but they do not make CPU work execute faster than available processors. They also do not remove limits imposed by databases, remote services, file descriptors, or connection pools. Blocking inside a long synchronized region or in certain native operations may reduce scalability. Applications should therefore use appropriate timeouts, bounded external resources, cancellation, and load testing.

Virtual threads are a concurrency mechanism, not a substitute for sound synchronization. Shared mutable state remains subject to races regardless of the type of thread.

Safe Concurrent Design

A safe design begins by reducing shared mutable state. Prefer immutable objects, local variables, message passing, and ownership of data by one task or component. final references prevent reassignment but do not make referenced objects immutable.

Practical principles include:

1. Define the ownership of every mutable object.
2. Protect each invariant with one clearly documented synchronization policy.
3. Keep critical sections short and avoid calling unknown external code while holding a lock.
4. Prefer high-level concurrency utilities over manually implemented wait-notify protocols.
5. Use bounded queues and explicit backpressure.
6. Propagate cancellation and preserve interruption.
7. Give every remote or blocking operation a timeout where appropriate.
8. Do not use Thread.stop() or rely on forced termination.
9. Test under contention, repetition, and realistic load.
10. Monitor queue lengths, latency, rejected work, failures, and resource saturation.

Security and governance also matter. Concurrent audit logging must preserve event order or include timestamps and correlation identifiers. Shared caches must not expose partially constructed sensitive data. Cancellation and timeout policies should prevent denial-of-service through unbounded work. Personal data should not be accidentally copied into diagnostic logs merely because multiple threads make tracing convenient.

Illustrative Case: Concurrent Order Processing

Background

An online retailer receives orders through an HTTP service. Each order requires inventory validation, payment authorisation, and confirmation. The first implementation uses one shared HashMap for inventory and creates a new platform thread for every request.

Problem

Under load, inventory occasionally becomes negative, some requests time out, and memory usage increases sharply. Investigation finds a check-then-update race, unbounded thread creation, and calls to external payment services without timeouts.

Analysis

Inventory reservation is a compound operation: checking available quantity and decrementing it must be atomic. Payment and confirmation involve I/O and can overlap across different orders. However, the payment provider and database have bounded capacity, so unlimited concurrency would merely move the bottleneck downstream.

Approach and Solution

The service uses ConcurrentHashMap for inventory entries, with compute or an atomic reservation method to enforce the stock invariant. A bounded executor handles CPU-intensive validation, while virtual threads handle many request tasks that spend time waiting for I/O. A semaphore limits simultaneous payment calls. Each remote call has a timeout and supports cancellation. Order state transitions are represented by immutable records and persisted through an idempotent operation so that retries do not charge a customer twice.

A structured completion flow launches independent inventory and customer-data reads, combines their results, then performs payment only after validation succeeds. Metrics record reservation failures, payment latency, queue depth, and cancellation counts.

Outcome

The revised design prevents negative inventory, limits pressure on the payment provider, and makes failures observable. It does not guarantee unlimited throughput; instead, it turns overload into controlled rejection or delayed work. This is a desirable outcome because correctness and predictable service are more important than accepting work that cannot be completed safely.

Learning Points

The case illustrates that concurrency is an end-to-end systems concern. Choosing a concurrent collection solves only one part of the problem. Correct invariants, bounded resources, cancellation, idempotency, and monitoring must be designed together.

Challenges and Limitations

Concurrency increases testing difficulty because failures may depend on rare timing. Passing tests on one machine does not prove correctness. Race detectors are less standard in Java than in some languages, so developers combine code review, stress tests, randomized scheduling, static analysis, thread dumps, and production telemetry.

Parallel execution can also reduce performance when tasks are too small, synchronization is excessive, or data locality is poor. Asynchronous pipelines may complicate stack traces and error handling. Thread pools can hide overload until queues become enormous. Virtual threads reduce the cost of waiting but do not eliminate finite external resources. These limitations support a central engineering rule: measure actual workload behaviour rather than assuming that more concurrency is always beneficial.

Chapter Summary

Concurrency allows overlapping progress, while parallelism involves simultaneous execution. Processes provide isolation; threads share a process’s memory; virtual threads provide lightweight, JVM-managed execution particularly useful for high numbers of blocking I/O tasks. Race conditions arise when unsynchronised operations conflict, and correctness requires attention to atomicity, visibility, and ordering.

Java provides intrinsic synchronization, explicit locks, atomic variables, coordination utilities, executors, futures, asynchronous composition, and concurrent collections. Deadlock, starvation, and livelock require deliberate prevention and diagnosis. Safe designs minimise shared mutable state, establish ownership, use bounded resources, support cancellation and timeouts, and make failures observable. Executors and virtual threads improve scalability only when the entire system—including databases and remote services—is designed with appropriate limits.

Key Terms

Process: An operating-system unit with its own address space and resources.

Thread: An execution path within a process.

Concurrency: Overlapping progress of multiple tasks.

Parallelism: Simultaneous execution of multiple tasks.

Asynchronous programming: Initiating work without requiring the caller to wait immediately for completion.

Race condition: A timing-dependent result caused by unsynchronised conflicting operations.

Atomicity: The property that an operation appears indivisible.

Visibility: The ability of one thread to observe another thread’s updates.

Happens-before: A Java Memory Model ordering relationship that provides visibility and ordering guarantees.

Synchronization: Coordination that controls access to shared state and establishes memory guarantees.

Executor: An abstraction that accepts tasks and manages their execution.

Future: A representation of a result that may be available later.

CompletableFuture: A future supporting asynchronous composition and error handling.

Concurrent collection: A collection designed for safe concurrent access.

Deadlock: Indefinite waiting caused by a cycle of resource dependencies.

Starvation: Repeated denial of access to a required resource.

Livelock: Ongoing activity without useful progress.

Virtual thread: A lightweight Java thread managed primarily by the JVM.

Backpressure: A mechanism that limits producers when consumers or downstream resources are overloaded.

Idempotency: The property that repeating an operation produces the same intended effect as performing it once.

Review Questions

Short Answer Questions

1. Distinguish concurrency from parallelism.
2. Why is value++ not necessarily atomic?
3. What guarantee does volatile provide, and what does it not provide?
4. State two uses of CompletableFuture.
5. Why should executors be shut down?

Long Answer Questions

1. Explain the Java Memory Model concepts of atomicity, visibility, ordering, and happens-before.
2. Compare intrinsic synchronization, ReentrantLock, atomic variables, and concurrent collections.
3. Explain the causes of deadlock and describe two prevention strategies.
4. Discuss executor design for CPU-bound and I/O-bound workloads.
5. Explain how virtual threads change application design and identify their limitations.

Analytical / Application Questions

1. Design a producer-consumer system for processing uploaded files. Specify the queue type, capacity policy, cancellation behaviour, and monitoring metrics.
2. Two account-transfer operations acquire account locks in opposite orders. Explain the failure and propose a lock-ordering solution.
3. An application submits three independent remote calls and then waits for each result sequentially. Redesign the workflow using asynchronous composition, timeouts, and failure handling.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes concurrency from parallelism?

A. Concurrency requires multiple processors, but parallelism does not.

B. Concurrency concerns overlapping progress; parallelism concerns simultaneous execution.

C. Concurrency applies only to processes, while parallelism applies only to threads.

D. They are exactly the same concept.

Correct Answer: B. Concurrency concerns overlapping progress; parallelism concerns simultaneous execution.

Question 2

What is the principal problem with the expression count++ when shared by multiple threads?

A. It always causes a deadlock.

B. It cannot be compiled by Java.

C. It is a compound operation that may lose updates.

D. It automatically creates a new process.

Correct Answer: C. It is a compound operation that may lose updates.

Question 3

Which mechanism is most appropriate for an atomic insert-if-absent operation in a ConcurrentHashMap?

A. Calling containsKey and then put separately

B. Calling putIfAbsent

C. Calling Thread.sleep before put

D. Using an ordinary HashMap without coordination

Correct Answer: B. Calling putIfAbsent

Question 4

What does calling thread.run() directly do?

A. It always creates a new operating-system thread.

B. It starts a virtual thread.

C. It invokes the method in the current execution path.

D. It interrupts the thread.

Correct Answer: C. It invokes the method in the current execution path.

Question 5

Which condition is characteristic of deadlock?

A. A thread completes before another thread starts.

B. Threads wait in a circular dependency for resources.

C. A queue has bounded capacity.

D. A task returns a completed future.

Correct Answer: B. Threads wait in a circular dependency for resources.

Question 6

What is a principal purpose of a bounded blocking queue?

A. To eliminate all waiting in a system

B. To provide backpressure when producers outpace consumers

C. To guarantee that tasks run in parallel

D. To make every object immutable

Correct Answer: B. To provide backpressure when producers outpace consumers

Question 7

Which statement about virtual threads is correct?

A. They make CPU-bound work faster than the available processors allow.

B. They are useful for many tasks that spend time blocked on I/O.

C. They eliminate race conditions in shared state.

D. They remove limits imposed by databases and remote services.

Correct Answer: B. They are useful for many tasks that spend time blocked on I/O.

Question 8

What does Future.get() normally do?

A. Cancels the associated task immediately

B. Waits for and returns the task’s result, possibly throwing exceptions

C. Converts a platform thread into a virtual thread

D. Makes a non-thread-safe collection safe

Correct Answer: B. Waits for and returns the task’s result, possibly throwing exceptions

Question 9

Which practice best supports safe concurrent design?

A. Hold locks while calling arbitrary external services

B. Use unlimited queues to avoid rejection

C. Minimise shared mutable state and define ownership clearly

D. Ignore interruption because tasks should always finish

Correct Answer: C. Minimise shared mutable state and define ownership clearly

Question 10

If 90% of a workload is parallelisable and four processors are used, Amdahl’s ideal speedup is approximately:

A. 1.43

B. 2.00

C. 3.08

D. 4.90

Correct Answer: C. 3.08