Persistence, Concurrency, and Distributed Object Applications

Introduction

Object-oriented programs become substantially more useful when objects can outlive a single execution, perform work concurrently, react to events, and communicate across process or network boundaries. These capabilities also introduce hazards that do not arise in small, single-threaded programs. A program may lose data when it terminates, corrupt shared state when two threads update it simultaneously, or fail because a remote object is unavailable.

This chapter develops a practical model for persistence, concurrency, event-driven interaction, and distributed objects. The emphasis is on design reasoning: choosing an appropriate persistence mechanism, preserving object invariants, coordinating concurrent activities, and recognizing that a distributed object is not equivalent to a local object.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain serialization and distinguish it from durable persistence.

2. Implement file-based and database-backed persistence safely.

3. Describe object-relational mapping and its trade-offs.

4. Create threads or asynchronous tasks and coordinate their execution.

5. Identify race conditions, deadlock, starvation, and visibility problems.

6. Apply synchronization, immutability, transactions, and event-driven design.

7. Explain distributed-system boundaries, remote failure, and serialization risks.

8. Evaluate a persistence and concurrency design in a realistic application.

1. Object Lifetime and Persistence

An object normally exists in volatile memory. Its state disappears when the process ends unless it is transferred to a durable representation. Persistence is the preservation of data so that it can be recovered later. Persistence is broader than saving a file: it includes databases, object stores, message logs, and cloud storage.

A useful distinction is:

• Transient state: exists only during the current execution, such as a cache or open socket.
• Serialized state: converted into a representation suitable for storage or transmission.
• Durable state: stored in a medium intended to survive process and usually machine failure.
• Recoverable state: durable data that can be interpreted and restored consistently.

Serialization is therefore a mechanism, not a guarantee of durability. A serialized object written to a buffer is not durable until the underlying storage system has successfully committed it. Similarly, deserialization reconstructs an object representation but may not restore external resources such as file handles, locks, or network connections.

Object Identity, State, and Invariants

An object has identity, state, and behavior. Persistence generally records state, but identity must also be considered. If the same customer is loaded twice, the application may accidentally create two in-memory objects representing one database row. This can lead to inconsistent updates.

An invariant is a condition that must remain true for a valid object. For example, a bank account may require that its account number is nonempty and that a withdrawal cannot exceed the permitted balance. Persistence code should not bypass these rules by directly assigning arbitrary values. A robust design loads validated data through constructors, factories, or controlled restoration methods.

2. Serialization and File Persistence

Serialization transforms an object graph into bytes or text. Common formats include JSON, XML, binary formats, and language-specific formats. Text formats are often portable and inspectable; binary formats may be compact and faster but less transparent.

A serialized object graph may contain references between objects. A serializer must decide whether to preserve shared references, duplicate values, or reject cycles. It must also address versioning. A class may gain, remove, or rename fields after data has been stored.

Safe Serialization Design

Good practice includes:

• Serialize explicit data transfer objects rather than arbitrary runtime objects.
• Record a schema version.
• Validate input during deserialization.
• Treat serialized input as untrusted, especially when received over a network.
• Exclude secrets, credentials, caches, and operating-system resources.
• Define behavior for missing, additional, or renamed fields.
• Use atomic replacement when updating an important file.

An atomic file update commonly writes new content to a temporary file, flushes it as required by the platform, and then replaces the old file using an atomic rename operation where supported. This reduces the chance that a crash leaves a partially written document. It does not, however, eliminate all storage or hardware failure modes.

Illustrative Python Example

```python

from dataclasses import dataclass, asdict

import json

from pathlib import Path

@dataclass

class Task:

task_id: int

title: str

completed: bool = False

def to_record(self):

return asdict(self)

@classmethod

def from_record(cls, record):

if not isinstance(record.get("task_id"), int):

raise ValueError("invalid task identifier")

if not isinstance(record.get("title"), str):

raise ValueError("invalid title")

return cls(record["task_id"], record["title"],

bool(record.get("completed", False)))

def save_tasks(tasks, filename):

path = Path(filename)

temporary = path.with_suffix(path.suffix + ".tmp")

payload = {"version": 1,

"tasks": [task.to_record() for task in tasks]}

temporary.write_text(json.dumps(payload), encoding="utf-8")

temporary.replace(path)

def load_tasks(filename):

payload = json.loads(Path(filename).read_text(encoding="utf-8"))

if payload.get("version") != 1:

raise ValueError("unsupported file version")

return [Task.from_record(r) for r in payload["tasks"]]

```

The example separates the persistent record from the runtime object and validates input. In production, the replacement operation, file permissions, backup policy, and concurrent writers would also require attention.

3. Database Persistence and Object-Relational Mapping

A relational database stores data in tables consisting of rows and columns. An object-oriented program stores instances with encapsulated state and references. Object-relational mapping (ORM) maps classes and relationships to relational structures.

A simple mapping might represent:

• `Student` as a `students` table;
• an object identifier as a primary key;
• a field such as `email` as a column;
• a one-to-many relationship as a foreign key in another table.

ORMs reduce repetitive SQL and can provide identity management, change tracking, transactions, and query abstractions. However, they do not remove the conceptual difference between objects and relations. This difference is often called the object-relational impedance mismatch.

Mapping Relationships

For a one-to-many relationship, a `Department` may have many `Course` objects. The relational design normally places `department_id` in the `courses` table. A many-to-many relationship generally requires a join table, such as `enrolments(student_id, course_id)`.

Inheritance is more difficult. Strategies include:

• Single table inheritance: all subclasses share one table with a type column.
• Joined tables: common and subclass-specific fields occupy related tables.
• Table per concrete class: each concrete subclass has its own complete table.

Each strategy trades simplicity, storage efficiency, constraints, and query complexity.

Transactions and Consistency

A transaction groups database operations into one logical unit. The commonly used ACID properties are:

• Atomicity: all operations commit, or the transaction is rolled back.
• Consistency: defined database constraints and application invariants are preserved.
• Isolation: concurrent transactions do not expose unacceptable intermediate effects.
• Durability: committed data survives ordinary subsequent failures.

Isolation is not synonymous with complete serial execution. Database systems offer levels such as read committed, repeatable read, and serializable. Stronger isolation can reduce anomalies but may reduce concurrency or increase blocking. The appropriate choice depends on business requirements.

A transfer between two accounts illustrates the need for a transaction. Debiting one account and crediting another must not be treated as two unrelated commits. If a failure occurs between them, the database could create or destroy money. The transaction should validate the debit, perform both updates, and commit once.

ORM Hazards

A common ORM problem is the N+1 query problem: one query retrieves a list of parent objects, and then one additional query is issued for each parent when a related collection is accessed. Eager loading, explicit joins, batching, or carefully designed queries can reduce this cost.

Other hazards include stale objects, unintended cascaded deletes, excessive memory use, and queries generated without appropriate indexes. Developers should inspect generated SQL and measure performance rather than assuming that object-oriented syntax produces efficient database access.

4. Concurrency Models

Concurrency means that multiple activities make progress during overlapping periods. Parallelism means that activities execute simultaneously, typically on different processor cores. A program can be concurrent without being parallel, for example when a single processor interleaves tasks.

Common models include:

1. Thread-based concurrency: multiple threads execute within one process and share memory.

2. Asynchronous tasks: operations yield while waiting for input/output, allowing other tasks to run.

3. Process-based concurrency: separate processes isolate memory and communicate explicitly.

4. Actor or message-based concurrency: independent actors communicate through messages rather than shared mutable state.

Threads are useful for CPU work when the language runtime and hardware permit parallel execution, and for blocking operations when appropriately managed. Asynchronous tasks are particularly effective for large numbers of I/O-bound operations, such as network requests.

Task Decomposition and Execution

A task should have a clear unit of work, bounded resource requirements, and a defined error policy. Thread pools are usually preferable to creating an unbounded number of threads. A pool limits resource consumption and supports reuse.

An asynchronous function often returns a future or promise representing a result that may become available later. The caller can await that result, combine several futures, or apply timeout and cancellation policies. Asynchrony does not automatically make shared state safe; two asynchronous tasks can still interleave in unsafe ways.

5. Concurrency Hazards and Synchronization

A race condition occurs when the result depends on the timing of concurrent operations. Consider a shared counter initially equal to 10. An increment may involve reading 10, adding 1, and writing 11. If two threads read 10 before either writes, both may write 11, losing one increment.

The operation must be made atomic through a lock, an atomic variable, a database operation, or a design that avoids shared mutation.

Main Hazards

• Race condition: timing changes the result.
• Data race: conflicting unsynchronized accesses occur, at least one being a write; language memory rules may make behavior unpredictable.
• Deadlock: activities wait indefinitely for one another.
• Livelock: activities remain active but make no useful progress.
• Starvation: one activity repeatedly fails to obtain required resources.
• Visibility error: one thread does not observe another thread’s update under the memory model.
• Priority inversion: a high-priority activity waits for a resource held by a lower-priority one.

Synchronization Techniques

A mutex or lock provides mutual exclusion. A critical section should be as small as possible and should protect a clearly defined invariant. Locks must be acquired and released using structured mechanisms such as context managers or `finally` blocks.

A condition variable allows a thread to sleep until a state predicate becomes true. The predicate must be checked in a loop because wake-up does not necessarily mean that the condition remains true.

Semaphores control access to a bounded number of resources. Read-write locks may allow multiple readers but only one writer, although they can introduce fairness and complexity issues. Atomic operations are suitable for simple state transitions, but compound invariants usually require a larger synchronization strategy.

The safest strategy is often to minimize shared mutable state. Immutable objects, message passing, ownership transfer, and confinement of state to one worker reduce synchronization requirements.

Deadlock Avoidance

Deadlock commonly arises when thread A holds lock 1 and requests lock 2 while thread B holds lock 2 and requests lock 1. A practical prevention method is to establish a global lock ordering and always acquire locks in that order. Time-bounded lock acquisition and deadlock detection can provide recovery or diagnostics, but they do not replace sound design.

6. Event-Driven Object Interaction

In an event-driven system, objects react to events such as a user action, a completed network request, a message, or a timer. A publisher emits an event and subscribers register handlers. This reduces direct coupling, but it can make control flow less visible.

A sound event design specifies:

• event structure and versioning;
• ordering guarantees;
• whether delivery is synchronous or asynchronous;
• retry and failure behavior;
• duplicate-event handling;
• subscriber lifetime and unsubscription;
• security and authorization.

An event handler should usually be short and nonblocking. It can validate the event, update local state, and enqueue longer work. If delivery is at least once, handlers should be idempotent: processing the same event twice should produce the same intended result as processing it once. A unique event identifier and a processed-event record are common techniques.

7. Distributed Objects and System Boundaries

A distributed object application allows components in different processes or machines to cooperate. Remote procedure calls, web APIs, message brokers, and distributed object frameworks all provide forms of interaction across a boundary.

A remote call differs fundamentally from a local method call. It may experience:

• network delay or disconnection;
• server crash or restart;
• timeout after the server has already completed the operation;
• incompatible versions;
• partial failure, in which some components work and others do not;
• authentication, authorization, and confidentiality threats.

Therefore, remote interfaces should make failure explicit. They should use stable data contracts, bounded timeouts, retry policies, correlation identifiers, and clear error categories. Retrying a non-idempotent operation can duplicate its effect. For example, a client that times out after submitting a payment cannot safely assume that the payment was not processed.

Distributed systems also require careful consistency decisions. A local transaction cannot automatically make a remote service and a local database commit atomically. Patterns such as transactional outbox, sagas, compensating actions, and idempotent consumers address this problem, but each has trade-offs. A saga coordinates a sequence of local transactions and performs compensating actions when a later step fails; compensation is a business operation, not a magical reversal of every side effect.

Serialization across a boundary should use explicit contracts rather than exposing internal class layouts. This supports independent deployment and reduces security risk. Deserializing executable or arbitrary object graphs from untrusted sources is especially dangerous.

8. Integrated Case Study: University Assignment Platform

The following is an illustrative case.

Background and Problem

A university assignment platform stores submissions, accepts uploads, runs automated tests, and notifies students when grading is complete. Initially, the system stored submission metadata in JSON files and executed grading in request-handling threads. Under heavy use, files were overwritten, requests timed out, and two grading workers occasionally processed the same submission.

Analysis

The system had several independent problems:

1. File replacement was not coordinated between writers.

2. Grading was long-running work performed in a latency-sensitive request path.

3. Submission status was updated by multiple workers without an atomic state transition.

4. A notification could be emitted twice after a retry.

5. The design assumed that a remote grading service behaved like a local method.

Approach and Solution

Submission metadata was moved to a relational database. A submission row contained an identifier, student identifier, immutable upload location, status, version, and timestamps. A transaction changed status from `QUEUED` to `RUNNING` only when the current status was `QUEUED`. The worker that successfully performed this transition owned the job.

The web service placed a grading command in a durable queue and returned a tracking identifier. Worker processes consumed commands asynchronously. Each command included a unique identifier. Before applying a result, a worker checked whether the command had already been processed. Results were saved in a transaction, and an outbox record was created in the same transaction. A separate publisher delivered notification events and retried failures.

The grading service received a versioned submission contract rather than a serialized internal `Submission` object. Calls used timeouts and authenticated communication. Because grading could be repeated safely, the operation was designed to be idempotent for a given submission version.

Outcome

The request path became short and predictable, concurrent workers no longer claimed the same submission, and notification retries did not create duplicate student messages. The system still required monitoring, queue capacity planning, access control for submissions, retention rules, and handling of a permanently failing grader.

Learning Points

Persistence, concurrency, and distribution must be designed together. A database transaction protects local state, but not an entire distributed workflow. Explicit state transitions, durable messaging, idempotency, and failure-aware interfaces are more reliable than treating remote work as an ordinary method call.

9. Professional, Ethical, and Governance Considerations

Persistent systems process personal, academic, financial, or health information. Engineers should apply least-privilege access, encryption in transit and at rest where appropriate, secure credential management, audit logging, retention limits, and deletion procedures. Data should not be serialized merely because it is convenient; sensitive fields require purpose limitation and access controls.

Concurrency and distribution can also affect fairness. If a queue prioritizes some users, the policy should be explicit and defensible. Automated grading systems require versioned test environments, reproducible results, appeal procedures, and monitoring for systematic errors. AI-assisted grading or classification should not be introduced without validation, human oversight where appropriate, and compliance with institutional and legal requirements.

10. Contemporary Developments and Best Practices

Modern applications increasingly combine relational databases, caches, event streams, serverless functions, containers, and managed queues. These technologies improve scalability but increase the number of failure boundaries. Observability is therefore essential: logs should include correlation identifiers, metrics should record latency and queue depth, and traces should show cross-service operations.

Recommended practices are:

1. Define object invariants before choosing a storage mapping.

2. Use explicit, versioned serialization contracts.

3. Validate all deserialized or remote input.

4. Keep transactions short and make their boundaries visible.

5. Prefer immutable state, ownership, or message passing to unrestricted shared mutation.

6. Use bounded pools, queues, timeouts, and cancellation.

7. Establish lock ordering or avoid multiple locks.

8. Design event consumers to tolerate duplicates where delivery is at least once.

9. Test failure scenarios, not only successful executions.

10. Measure generated database queries and concurrent-system behavior.

11. Separate local domain objects from remote data-transfer contracts.

12. Document consistency, ordering, retry, and recovery guarantees.

Chapter Summary

Persistence allows object state to survive beyond one process, but serialization alone does not guarantee durability. Safe persistence requires explicit representations, validation, schema evolution, and preservation of object invariants. Relational databases provide transactions and durable constraints, while ORM tools simplify mapping at the cost of query and identity complexity.

Concurrency enables overlapping work through threads, asynchronous tasks, processes, or message-based models. Shared mutable state creates races, visibility problems, deadlock, starvation, and other hazards. Locks, atomic operations, conditions, immutability, ownership, and message passing are tools for controlling these risks. Event-driven systems reduce coupling but require defined delivery, ordering, retry, and idempotency policies.

Distributed objects cross boundaries where latency, partial failure, version mismatch, and security threats are normal. Remote interfaces should use explicit contracts, timeouts, authentication, correlation identifiers, and failure-aware workflows. Transactions, queues, outbox records, sagas, and idempotent consumers help coordinate work without pretending that a distributed system is a single local object space.

Key Terms

Persistence: Preservation of data beyond the lifetime of a process.

Serialization: Conversion of object state into a transferable or storable representation.

Deserialization: Reconstruction of data or objects from a serialized representation.

Object invariant: A condition that must remain true for an object to be valid.

Durability: The property that committed data survives ordinary subsequent failures.

ORM: A tool or technique that maps object structures to relational database structures.

Transaction: A logical unit of database work with defined commit and rollback behavior.

Atomicity: The all-or-nothing property of a transaction or operation.

Concurrency: Overlapping progress of multiple activities.

Parallelism: Simultaneous execution of activities.

Race condition: A timing-dependent behavior whose result varies with interleaving.

Deadlock: Indefinite waiting caused by circular resource dependencies.

Starvation: Repeated failure of an activity to obtain needed resources.

Mutex: A mutual-exclusion lock protecting shared state.

Idempotency: The property that repeated application has the same intended effect as one application.

Event-driven system: A system in which components respond to emitted events.

Partial failure: Failure of some distributed components while others continue operating.

Saga: A sequence of local transactions coordinated with compensating actions.

Transactional outbox: A pattern that stores an outgoing event with local database changes in one transaction.

Review Questions

Short Answer Questions

1. Why is serialization not equivalent to durable persistence?

2. What is the object-relational impedance mismatch?

3. Distinguish concurrency from parallelism.

4. Define a race condition and give one prevention technique.

5. Why should remote interfaces use explicit data contracts?

Long Answer Questions

1. Explain how object identity, invariants, and schema evolution affect persistence design.

2. Compare thread-based, asynchronous, process-based, and message-based concurrency.

3. Describe ACID transactions and discuss why stronger isolation may reduce concurrency.

4. Explain deadlock, starvation, visibility errors, and practical synchronization strategies.

5. Analyze the benefits and limitations of ORM in a medium-sized application.

Analytical / Application Questions

1. Design a persistence and concurrency strategy for a ticket-booking system in which many users may reserve the final available seat.

2. A remote payment call times out after the client sends the request. Explain the possible states and propose a safe retry and reconciliation design.

3. For the university assignment platform case, identify two additional failure scenarios and specify how monitoring, data design, or workflow changes could address them.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes serialization from persistence?

A. Serialization always guarantees recovery after a power failure.

B. Persistence is conversion of objects into memory addresses.

C. Serialization creates a representation, while persistence stores data for later recovery.

D. Persistence can occur only in relational databases.

Correct Answer: C. Serialization creates a representation, while persistence stores data for later recovery.

Question 2

Which practice best protects an important file from being left partially written?

A. Write directly to the original file without flushing.

B. Write a temporary file and replace the original after successful completion.

C. Serialize the object twice in the same buffer.

D. Store the file only in a thread-local variable.

Correct Answer: B. Write a temporary file and replace the original after successful completion.

Question 3

What is the primary purpose of a database transaction?

A. To ensure every query runs in parallel.

B. To group related operations into a controlled logical unit.

C. To eliminate the need for database constraints.

D. To convert relational tables into inheritance hierarchies.

Correct Answer: B. To group related operations into a controlled logical unit.

Question 4

Two threads increment a shared counter, but one increment is lost. What is the most likely cause?

A. A race condition in a non-atomic read-modify-write operation.

B. Schema versioning.

C. A serialization format mismatch.

D. An event being immutable.

Correct Answer: A. A race condition in a non-atomic read-modify-write operation.

Question 5

Which design most directly reduces deadlock risk when multiple locks are required?

A. Acquire locks in an established global order.

B. Allow each thread to choose its own lock order.

C. Hold locks indefinitely after use.

D. Replace every lock with a network request.

Correct Answer: A. Acquire locks in an established global order.

Question 6

Why should an at-least-once event consumer be idempotent?

A. Events are guaranteed to arrive exactly once.

B. Duplicate delivery should not create unintended repeated effects.

C. Idempotency encrypts event contents.

D. It prevents all network failures.

Correct Answer: B. Duplicate delivery should not create unintended repeated effects.

Question 7

Which is a characteristic of a distributed object call that a local call usually does not have?

A. It always executes without delay.

B. It cannot return a value.

C. It may time out after the remote operation has completed.

D. It never requires authentication.

Correct Answer: C. It may time out after the remote operation has completed.

Question 8

What problem does the N+1 query pattern commonly cause?

A. Too many database queries when related objects are accessed individually.

B. Loss of all primary keys during serialization.

C. Automatic deadlock in every transaction.

D. Inability to create immutable objects.

Correct Answer: A. Too many database queries when related objects are accessed individually.

Question 9

What does a transactional outbox help coordinate?

A. Saving a local database change together with a record of an outgoing event.

B. Acquiring two locks in reverse order.

C. Replacing all database transactions with file writes.

D. Running CPU instructions without an operating system.

Correct Answer: A. Saving a local database change together with a record of an outgoing event.

Question 10

Which statement about a saga is correct?

A. It guarantees global atomicity across all services.

B. It coordinates local transactions and may use compensating actions.

C. It eliminates the need for authentication.

D. It applies only to single-threaded programs.

Correct Answer: B. It coordinates local transactions and may use compensating actions.