Exceptions, Files, Serialization, and Debugging

Introduction

Reliable software must do more than produce correct results for ideal input. It must respond safely when a user enters invalid data, a file is missing, a network resource is unavailable, or a stored object no longer matches the current program design. This chapter examines four closely related areas of defensive programming: exception handling, file processing, serialization, and debugging.

Exceptions provide a structured way to report and handle abnormal conditions. Validation prevents unacceptable data from entering a program. File processing allows programs to exchange information with persistent storage. Serialization converts objects or data structures into a storable or transferable representation. Debugging and logging provide the evidence needed to locate and correct faults.

The examples use Python, Java, and C++ where their approaches differ significantly. The objective is not to memorise syntax, but to develop transferable professional practices: identify failure modes, preserve useful diagnostic information, avoid data corruption, and separate recovery from reporting.

Learning Objectives

After studying this chapter, you should be able to:

1. Distinguish syntax errors, runtime errors, logical errors, and exceptional conditions.
2. Validate input before using it and handle exceptions appropriately.
3. Read and write text, binary, CSV, and JSON files safely.
4. Explain serialization, deserialization, compatibility, and security risks.
5. Apply debugging techniques, assertions, breakpoints, and logging.
6. Design small programs that fail safely and provide useful diagnostic information.
7. Evaluate trade-offs among portability, performance, reliability, and security.

1. Errors, Failures, and Exceptions

An error is any condition that causes a program to produce an incorrect result or fail to complete its intended task. Errors are commonly classified as follows.

An exception is an event representing an abnormal condition that interrupts the normal flow of execution. For example, attempting to convert "abc" to an integer, opening a nonexistent file, or dividing by zero may raise an exception.

An exception is not necessarily a programming defect. A missing optional configuration file may be an expected condition that the application can handle. Conversely, catching every exception and continuing may conceal a serious defect. Good design therefore asks two questions:

• Can the program recover meaningfully?
• If it cannot recover, can it fail clearly and safely?

1.1 Exception propagation

When an exception is raised inside a function, the function may handle it or allow it to propagate to its caller. Propagation is useful when the current function lacks sufficient context to decide what to do.

For example, a low-level file-reading function may report that a file could not be opened, while a user-interface layer decides whether to ask the user for another filename. The lower layer should not silently replace the file with an empty file, because that changes the meaning of the data.

1.2 Exception handling constructs

Python uses try, except, else, and finally. Java uses try, catch, finally, and try-with-resources. C++ uses try, catch, and automatic resource management through destructors and RAII.

Python example:

def read_age():
    while True:
        text = input("Age: ").strip()
        try:
            age = int(text)
            if age < 0 or age > 130:
                raise ValueError("age is outside the valid range")
            return age
        except ValueError as error:
            print(f"Invalid age: {error}")

The try block should be narrow. It contains only operations expected to fail in the relevant way. A broad block can accidentally classify programming defects as input errors.

Java example:

try {
    int age = Integer.parseInt(text.trim());
    if (age < 0 || age > 130) {
        throw new IllegalArgumentException("Age is outside the valid range");
    }
} catch (NumberFormatException | IllegalArgumentException ex) {
    System.err.println("Invalid age: " + ex.getMessage());
}

In Java, NumberFormatException is a subclass of IllegalArgumentException; therefore, listing both in the same multi-catch is illegal in actual Java because one alternative is a subclass of another. A correct version catches NumberFormatException separately or catches only IllegalArgumentException:

try {
    int age = Integer.parseInt(text.trim());
    if (age < 0 || age > 130) {
        throw new IllegalArgumentException("Age is outside the valid range");
    }
} catch (IllegalArgumentException ex) {
    System.err.println("Invalid age: " + ex.getMessage());
}

C++ example:

try {
    int age = std::stoi(text);
    if (age < 0 || age > 130) {
        throw std::invalid_argument("age is outside the valid range");
    }
} catch (const std::invalid_argument& ex) {
    std::cerr << "Invalid age: " << ex.what() << '\n';
} catch (const std::out_of_range& ex) {
    std::cerr << "Age is too large: " << ex.what() << '\n';
}

Catch exceptions by reference in C++ to avoid unnecessary copying and object slicing. Catch more specific exceptions before more general ones.

1.3 Cleanup and resource safety

Cleanup must occur whether an operation succeeds or fails. A file should be closed, a lock released, and a temporary resource removed. Python's with statement, Java's try-with-resources, and C++ RAII address this requirement.

Python:

with open("records.txt", "r", encoding="utf-8") as file:
    contents = file.read()

Java:

try (BufferedReader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8)) {
    String line;
    while ((line = reader.readLine()) != null) {
        System.out.println(line);
    }
}

C++:

std::ifstream file("records.txt");
if (!file) {
    throw std::runtime_error("Cannot open records.txt");
}
std::string line;
while (std::getline(file, line)) {
    // Process line
}

The C++ stream object closes its file when it leaves scope. This is an example of Resource Acquisition Is Initialization (RAII).

1.4 Designing useful exceptions

A useful exception message states what operation failed and identifies relevant context without exposing secrets. For example, "Cannot load configuration file: permissions denied" is more useful than "Error". Do not include passwords, access tokens, private keys, or personal data in messages or logs.

Custom exceptions are appropriate when callers need to distinguish a domain failure from a low-level failure. For example, a banking program might use InsufficientFundsError, while a file library might preserve the original file-system exception as its cause.

2. Validation and Defensive Programming

Exception handling is not a replacement for validation. Validation checks whether data satisfies expected rules before the program performs an operation. It may be syntactic, semantic, or contextual.

• Syntactic validation checks form, such as whether an email contains an expected structure.
• Semantic validation checks meaning, such as whether an age is within a permitted range.
• Contextual validation checks relationships, such as whether an end date follows a start date.

Validation should occur at trust boundaries: user interfaces, API endpoints, files, databases, and message queues. However, important business rules should also be enforced in the domain layer so they cannot be bypassed by an alternative interface.

A reliable validation sequence is:

1. Read the input as text or a safe neutral representation.
2. Check presence, length, format, and permitted characters.
3. Convert it to the required type.
4. Check range and business constraints.
5. Use the validated value.

Validation must be fail-closed: if the program cannot establish that input is safe and valid, it should reject it or request clarification. Never construct a filename, SQL statement, shell command, or deserialised object directly from untrusted input without appropriate controls.

3. Text and Binary Files

A text file stores characters encoded into bytes. UTF-8 is a common choice because it represents Unicode text and is widely supported. Programs should specify the encoding rather than relying on an operating-system default.

A binary file stores bytes according to an application-specific format. Images, compiled data, compressed archives, and many scientific formats are binary. Binary data should not be decoded as text unless the format explicitly requires it.

3.1 Reading and writing text

Python:

from pathlib import Path

path = Path("notes.txt")
path.write_text("First line\nSecond line\n", encoding="utf-8")
text = path.read_text(encoding="utf-8")

Java:

Path path = Path.of("notes.txt");
Files.writeString(path, "First line\nSecond line\n", StandardCharsets.UTF_8);
String text = Files.readString(path, StandardCharsets.UTF_8);

C++:

std::ofstream output("notes.txt");
output << "First line\nSecond line\n";
output.close();

For large files, process incrementally rather than loading the entire file into memory. A line-by-line strategy reduces peak memory use and permits streaming validation.

3.2 File modes and atomic updates

Opening a file for writing may truncate existing content. Append mode adds data, while exclusive-create mode fails if the target already exists. An application should choose deliberately.

For important configuration or transaction files, write to a temporary file, flush and close it, then replace the original using an atomic file-system operation where supported. This reduces the risk of leaving a half-written file after a crash. Atomic replacement does not automatically provide a complete backup or guarantee durability against every hardware failure; those requirements may require a journal, backup, or explicit synchronization.

3.3 Paths and portability

A path should be constructed with the language's path library rather than manual string concatenation. This handles platform-specific separators and reduces mistakes. Programs should also consider relative versus absolute paths, current working directories, permissions, symbolic links, and path traversal attacks such as ../private.txt.

4. CSV and JSON Data

4.1 CSV

Comma-separated values are simple but not trivial. Correct CSV may contain quoted fields, embedded commas, escaped quotation marks, and line breaks. Splitting a line with string.split(",") is therefore unsafe for general CSV.

Python example:

import csv

with open("students.csv", newline="", encoding="utf-8") as file:
    reader = csv.DictReader(file)
    for row in reader:
        name = row["name"].strip()
        mark = float(row["mark"])
        if not 0 <= mark <= 100:
            raise ValueError("mark outside permitted range")
        print(name, mark)

CSV has no universal type system. A value such as 0017 may represent an identifier rather than the number seventeen. Applications should define column names, types, missing-value rules, and acceptable ranges.

4.2 JSON

JSON represents objects, arrays, strings, numbers, Boolean values, and null. It is human-readable and common in web APIs and configuration files. JSON itself does not distinguish all language-specific types, such as dates, sets, or arbitrary class instances. Such values need an explicit representation, often an ISO 8601 date string or a documented object structure.

Python:

import json
from pathlib import Path

configuration = {"host": "localhost", "port": 8080, "debug": False}
Path("config.json").write_text(
    json.dumps(configuration, indent=2), encoding="utf-8"
)

data = json.loads(Path("config.json").read_text(encoding="utf-8"))
if not isinstance(data.get("port"), int):
    raise ValueError("port must be an integer")

Parsing JSON confirms that the document is syntactically valid; it does not confirm that required fields, types, ranges, and permissions are correct. Schema validation or explicit application checks are still necessary.

5. Serialization and Deserialization

Serialization converts an in-memory value into a representation that can be stored or transmitted. Deserialization reconstructs an in-memory value from that representation.

Serialization formats include JSON, XML, CSV, Protocol Buffers, MessagePack, Java object serialization, Python pickle, and custom binary formats. The choice depends on interoperability, size, speed, schema evolution, human readability, and security.

5.1 Versioning and compatibility

A serialized representation is an interface. It should have a documented schema, versioning policy, required and optional fields, default values, and migration strategy. Robust readers should tolerate safely ignorable new fields where the format permits it. Writers should avoid removing or changing the meaning of fields without a migration plan.

A useful design principle is to serialize data, not accidental implementation details. Store a student's identifier, name, and mark rather than the memory layout of a language object. Explicit data transfer objects are often more stable than direct native serialization.

5.2 Security

Deserializing untrusted native objects can cause code execution or other severe vulnerabilities in some ecosystems. Python pickle and Java native object serialization must not be used on attacker-controlled data without a carefully controlled, authenticated, and reviewed design. JSON parsing is generally safer, but unsafe use can still cause denial of service, excessive nesting, or injection into later operations.

Security controls include:

• use allow-listed types and fields;
• limit input size and nesting depth;
• authenticate data where integrity matters;
• validate after parsing;
• keep libraries updated;
• separate data parsing from object construction;
• never treat serialized input as trusted merely because it came from a file.

6. Debugging and Logging

Debugging is a systematic process of locating, explaining, and correcting a defect. Effective debugging begins with a reproducible failure, not random code changes.

A practical debugging cycle is:

1. Reproduce the problem with the smallest reliable input.
2. Observe the actual result and record expected behaviour.
3. Form a specific hypothesis.
4. Inspect state using logs, a debugger, or targeted tests.
5. Change one relevant factor.
6. Re-run the reproduction and regression tests.
7. Document the cause and the fix.

A debugger commonly provides breakpoints, step-over, step-into, step-out, variable inspection, call-stack inspection, watch expressions, and conditional breakpoints. A breakpoint should be placed near the boundary between a known-correct and an incorrect state.

Assertions express conditions that should always be true for a correct program. They are appropriate for internal invariants, such as a sorted collection or a non-null object after an internal constructor. They are not a substitute for validating user input, because assertions may be disabled in some environments.

Logging differs from printing. A logging system supports severity levels, timestamps, component names, structured fields, output routing, and configuration. Common levels are DEBUG, INFO, WARNING, ERROR, and CRITICAL. Production logs should be useful without revealing confidential information. Structured logs, often JSON objects, support searching and correlation across services.

Example Python logging:

import logging

logger = logging.getLogger(__name__)

try:
    result = process_record(record)
except ValueError:
    logger.warning("Rejected record", extra={"record_id": record.get("id")})
    raise
except OSError:
    logger.exception("Storage operation failed")
    raise

logger.exception includes the current exception traceback and should be used inside an exception handler. Logging and handling are separate decisions: an exception should not be logged repeatedly at every layer, which creates duplicate noise.

7. Integrated Worked Example: A Safe Student-Record Importer

Consider an illustrative application that imports student marks from a CSV file and writes a JSON summary. The input contains student_id, name, and mark.

Background and problem

The college previously used manual spreadsheet copying. Errors included missing files, invalid marks, duplicate identifiers, malformed CSV quoting, and partial output when the program stopped midway.

Analysis

The importer crosses several trust boundaries. The file path is external input; CSV fields are untrusted; the output must not replace a valid report with incomplete data; and diagnostic information must help an administrator correct the source file.

Approach

The program should:

1. Validate that the input path is permitted and exists.
2. Read CSV using a library rather than splitting lines.
3. Require the three expected columns.
4. Validate each row, including identifier format, non-empty name, and mark range.
5. Reject duplicate identifiers.
6. Accumulate valid records only if the whole import succeeds.
7. Write JSON to a temporary file and replace the target after successful completion.
8. Log row numbers and failure categories, but not unnecessary personal data.

A simplified Python core is:

import csv
import json
from pathlib import Path

class ImportError(Exception):
    pass

def import_records(source: Path, target: Path):
    records = []
    seen = set()

    try:
        with source.open(newline="", encoding="utf-8") as file:
            reader = csv.DictReader(file)
            required = {"student_id", "name", "mark"}
            if reader.fieldnames is None or not required <= set(reader.fieldnames):
                raise ImportError("required columns are missing")

            for row_number, row in enumerate(reader, start=2):
                student_id = row["student_id"].strip()
                name = row["name"].strip()
                if not student_id or not name:
                    raise ImportError(f"row {row_number}: missing required value")
                if student_id in seen:
                    raise ImportError(f"row {row_number}: duplicate student ID")
                try:
                    mark = float(row["mark"])
                except ValueError as error:
                    raise ImportError(f"row {row_number}: mark is not numeric") from error
                if not 0 <= mark <= 100:
                    raise ImportError(f"row {row_number}: mark outside 0–100")
                seen.add(student_id)
                records.append({"student_id": student_id, "name": name, "mark": mark})

        temporary = target.with_suffix(target.suffix + ".tmp")
        temporary.write_text(json.dumps(records, indent=2), encoding="utf-8")
        temporary.replace(target)
    except OSError as error:
        raise ImportError(f"file operation failed: {error}") from error

Outcome

The importer either produces a complete validated report or leaves the previous report unchanged. The error identifies the row and category, allowing correction without inspecting a traceback. In a production system, the application would also impose limits on file size and row count, define an authentication and authorisation policy, and use a formal schema where appropriate.

Learning points

The example combines validation, narrow exception handling, CSV parsing, JSON serialization, resource safety, and atomic replacement. It also illustrates a transaction-like principle: do not publish partial results as if they were complete.

8. Common Mistakes and Professional Practice

Common mistakes include catching Exception or ... without a clear recovery action, using an empty catch block, logging sensitive values, assuming JSON validation guarantees business validity, splitting CSV manually, relying on default character encodings, and deserializing native objects from untrusted sources.

Professional practice includes:

• handle the most specific expected failure;
• preserve the original cause when translating exceptions;
• keep exception scopes narrow;
• use resource-management constructs;
• validate at trust boundaries and enforce domain rules centrally;
• make file formats and encodings explicit;
• define schemas and compatibility policies;
• test malformed, missing, oversized, and hostile inputs;
• use atomic or transactional output strategies;
• record enough diagnostic context to reproduce failures;
• remove secrets and unnecessary personal data from logs;
• include regression tests for every corrected defect.

9. Testing Failure Behaviour

A robust test suite tests both successful and unsuccessful paths. Unit tests should cover invalid types, boundary values, missing fields, duplicate records, empty files, malformed quoting, invalid encodings, inaccessible paths, and interrupted output. Integration tests should verify real file permissions, platform path behaviour, and compatibility with representative external files.

Boundary-value tests are especially valuable. If a mark is valid from 0 through 100 inclusive, test −1, 0, 100, and 101. The requirement is not a universal mathematical law; it is an explicit domain rule that must be documented and tested.

Fault injection can simulate disk-full conditions, unavailable services, corrupted files, and timeouts. Such tests reveal whether the program leaves consistent state and whether its messages help operators recover.

10. Contemporary Considerations: Cloud Systems and AI-Assisted Development

Modern applications often process files in object storage and exchange serialized messages through queues or APIs. These environments add concerns such as eventual consistency, retry duplication, message ordering, access policies, checksums, and idempotency. A retry may repeat an operation, so processing should use unique event identifiers or transactional safeguards where duplicate effects would be harmful.

AI coding assistants can suggest exception handling, parsers, and tests, but generated code must be reviewed. An assistant may recommend unsafe deserialization, catch overly broad exceptions, omit encoding declarations, or produce tests that verify only the happy path. Developers remain responsible for threat modelling, licensing compliance, privacy protection, and correctness.

Chapter Summary

Exceptions represent abnormal conditions and may be handled locally or propagated to a layer with enough context to recover. Validation prevents invalid or unsafe data from reaching later operations, but parsing alone does not establish semantic correctness. Text files require explicit encoding; binary files require format-aware processing; CSV should be parsed with a standards-aware library; and JSON should be checked against application rules after parsing.

Serialization is an interface between memory and storage or transport. Stable systems document schemas, plan for version changes, and avoid treating untrusted serialized data as executable objects. Resource-management constructs ensure cleanup during both success and failure. Debuggers, assertions, tests, and structured logs complement one another: they help reproduce, locate, explain, and prevent defects. The integrated importer demonstrates a central professional principle: validate before use, preserve useful evidence, and publish only complete, trustworthy results.

Key Terms

Exception: An event that interrupts normal program execution because an abnormal condition occurred.

Exception propagation: Passing an unhandled exception to the calling function or a higher layer.

Validation: Checking that input satisfies syntactic, semantic, and contextual rules.

Trust boundary: A point where data enters from a user, file, service, or other potentially untrusted source.

RAII: A C++ resource-management technique in which resource lifetime follows object lifetime.

Encoding: A rule for representing characters as bytes.

Text file: A file containing encoded characters intended to be interpreted as text.

Binary file: A file containing bytes organised according to a non-text format.

CSV: A tabular text format with rules for delimiters, quoting, and escaped characters.

JSON: A text-based notation for objects, arrays, strings, numbers, Boolean values, and null.

Serialization: Conversion of an in-memory value into a storable or transferable representation.

Deserialization: Reconstruction of a value from serialized data.

Schema: A documented structure specifying fields, types, and constraints.

Atomic replacement: Updating a file so that readers observe either the old complete version or the new complete version, rather than a partial write.

Assertion: A check of an internal condition expected to hold in a correct program.

Breakpoint: A debugger instruction that pauses execution at a selected location.

Logging: Recording structured runtime information for diagnosis, monitoring, or auditing.

Regression test: A test that ensures a previously corrected defect does not return.

Idempotency: A property in which repeating an operation produces the same intended effect as performing it once.

Review Questions

5 Short Answer Questions

1. What is the difference between validation and exception handling?
2. Why should a try block generally be narrow?
3. State one advantage of JSON and one limitation of CSV.
4. What purpose does a debugger breakpoint serve?
5. Why should native object deserialization from untrusted input be avoided?

5 Long Answer Questions

1. Compare exception handling in Python, Java, and C++, including resource cleanup techniques.
2. Explain how a program should safely read, validate, and write a CSV-derived JSON report.
3. Discuss schema evolution and backward compatibility in serialized data.
4. Describe a systematic debugging process for a program that intermittently fails during file processing.
5. Explain how logging, assertions, automated tests, and exception handling work together in dependable software.

3 Analytical / Application Questions

1. An importer accepts marks from 0 to 100 but sometimes produces a report containing only half the rows after a crash. Identify the design weaknesses and propose a corrected processing strategy.
2. A Java application reads serialized objects uploaded by users. Analyse the security risks and recommend a safer data-exchange design.
3. A Python service catches every exception, logs only "failed", and continues processing. Evaluate the consequences for correctness, diagnosis, and security, then redesign the policy.

Multiple Choice Questions with Answers

Question 1

Which practice most directly ensures that a file is closed when an operation raises an exception in Python?

A. Using a global variable for the file

B. Using a with statement

C. Printing the filename after reading

D. Catching every exception and ignoring it

Correct Answer: B. Using a with statement

Question 2

Why is splitting a CSV line using split(",") unsafe for general CSV data?

A. CSV cannot contain numbers

B. CSV files are always binary

C. Fields may contain quoted commas and line breaks

D. CSV requires JSON schemas

Correct Answer: C. Fields may contain quoted commas and line breaks

Question 3

What does deserialization do?

A. Deletes a stored object

B. Converts an in-memory value into bytes only

C. Reconstructs a value from serialized data

D. Compiles source code

Correct Answer: C. Reconstructs a value from serialized data

Question 4

Which item should generally not be written to a production log?

A. An operation name

B. A correlation identifier

C. A processing duration

D. A user's password

Correct Answer: D. A user's password

Question 5

What is the main purpose of input validation at a trust boundary?

A. To make all programs run faster

B. To ensure external data satisfies permitted rules before use

C. To replace all automated tests

D. To guarantee that a disk cannot fail

Correct Answer: B. To ensure external data satisfies permitted rules before use

Question 6

Which strategy best reduces the risk of publishing a partially written report?

A. Write directly to the final file throughout processing

B. Write a temporary complete file and replace the target after success

C. Ignore write errors

D. Store the report only in a local variable

Correct Answer: B. Write a temporary complete file and replace the target after success

Question 7

What is a suitable use of an assertion?

A. Checking an internal invariant that should always hold

B. Validating a user's password

C. Replacing permission checks

D. Parsing arbitrary untrusted objects

Correct Answer: A. Checking an internal invariant that should always hold

Question 8

A function cannot decide whether a missing file is recoverable because it lacks user-interface context. What is usually appropriate?

A. Silently return unrelated data

B. Allow a meaningful exception to propagate

C. Delete the file path

D. Disable all logging

Correct Answer: B. Allow a meaningful exception to propagate

Question 9

Which statement about JSON parsing is correct?

A. Successful parsing proves all business rules are satisfied

B. JSON can represent every language-specific object exactly

C. Successful parsing establishes syntax, but application validation is still required

D. JSON is always safer than every other format

Correct Answer: C. Successful parsing establishes syntax, but application validation is still required

Question 10

Which debugging action should normally occur before making repeated code changes?

A. Reproduce and characterise the failure

B. Remove all exception handlers

C. Disable tests

D. Change several unrelated modules at once

Correct Answer: A. Reproduce and characterise the failure