Exception Handling Fundamentals

Introduction

A running program encounters two broad kinds of failure. A syntax error prevents Python from constructing a valid program, whereas an exception occurs while valid Python code is executing. Exceptions may result from invalid user input, missing files, unavailable network services, incorrect data, programming defects, or unexpected environmental conditions.

Exception handling is the disciplined process of detecting, responding to, and, where possible, recovering from exceptional conditions. It is not a substitute for correct design. A robust program should validate expected input, handle foreseeable failures, preserve useful diagnostic information, and allow genuinely unexpected defects to remain visible during development.

Python represents an exception as an object. When an exception is raised, normal sequential execution is interrupted and Python searches for a suitable handler. Understanding this process is essential for developing reliable file-processing, modular, and production-quality applications.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish errors from exceptions;
• explain Python’s exception hierarchy and inheritance relationships;
• describe how exceptions propagate through nested calls;
• use try, except, else, and finally correctly;
• raise built-in and custom exceptions;
• preserve the original cause of an error through exception chaining;
• avoid overly broad, silent, and misleading error handling;
• design exception handling appropriate to a professional application.

1. Errors, Exceptions, and Exceptional Conditions

An exception is an event indicating that an operation cannot proceed normally. Consider:

number = int("abc")

The syntax is valid, but conversion fails at runtime and Python raises ValueError. Similarly, opening a nonexistent file commonly raises FileNotFoundError, and dividing by zero raises ZeroDivisionError.

A useful distinction is:

Not every failure should be handled locally. A low-level function may lack enough context to decide what to do. It may therefore propagate the exception to a higher-level component that can report the problem to a user, retry an operation, or terminate safely.

2. Python’s Exception Hierarchy

Python exceptions form a class hierarchy. At the top of the ordinary exception hierarchy is BaseException. Most application exceptions should derive from Exception, not directly from BaseException.

A simplified hierarchy is:

BaseException
├── KeyboardInterrupt
├── SystemExit
├── GeneratorExit
└── Exception
    ├── ArithmeticError
    │   ├── ZeroDivisionError
    │   └── OverflowError
    ├── LookupError
    │   ├── IndexError
    │   └── KeyError
    ├── OSError
    │   ├── FileNotFoundError
    │   ├── PermissionError
    │   └── TimeoutError
    ├── TypeError
    ├── ValueError
    └── RuntimeError

The hierarchy supports polymorphic handling. A handler for OSError can handle FileNotFoundError and PermissionError, because both are subclasses of OSError. A handler for Exception can handle most ordinary application exceptions, but it is usually too broad for routine local handling.

KeyboardInterrupt, SystemExit, and GeneratorExit inherit directly from BaseException so that a routine except Exception does not accidentally suppress user interruption or program termination.

Common Built-in Exceptions

The most specific applicable exception should generally be handled first. For example:

try:
    value = int(input("Enter an integer: "))
except ValueError:
    print("Please enter a valid integer.")

A TypeError should not be caught here merely because it is possible. Handling an exception that the operation is not expected to produce can conceal programming defects.

3. The Propagation Model

When Python raises an exception, it searches for a matching except clause in the current function. If no matching clause exists, Python returns to the function that called it and repeats the search. This process is called stack unwinding or exception propagation.

def read_score(text):
    return int(text)

def process_score(text):
    return read_score(text)

try:
    score = process_score("unknown")
except ValueError as error:
    print(f"Invalid score: {error}")

read_score raises ValueError. It has no handler, so the exception propagates through process_score to the surrounding try statement. Once a matching handler is found, Python executes that handler and resumes after the complete try statement.

If no handler is found, the exception reaches the program’s top level. Python then displays a traceback containing the exception type, message, and call path. A traceback is valuable diagnostic evidence: it identifies where the exception was raised and how execution reached that point.

Propagation has important design consequences:

1. A function should handle an exception only when it can take a meaningful action.
2. A function may add context and re-raise the exception.
3. A public application boundary should normally convert technical failures into appropriate user-facing or service-level responses.
4. Logging should occur at a level where the event can be interpreted, while avoiding duplicate logs at every layer.

4. The try Statement

The basic structure is:

try:
    risky_operation()
except SpecificError:
    recovery_action()

The try block should be narrow. It should contain only operations expected to raise the exceptions being handled. This is preferable:

try:
    quantity = int(text)
except ValueError:
    quantity = 0

to a large block that includes unrelated calculations, file operations, and output. A large block makes it difficult to identify which operation failed and may cause an unrelated bug to be misinterpreted as invalid input.

Multiple Handlers

try:
    result = items[index]
except IndexError:
    print("The position does not exist.")
except TypeError:
    print("The position must be an integer.")

Handlers are tested from top to bottom. Put specific exception classes before their base classes:

try:
    open(path)
except FileNotFoundError:
    print("The file is missing.")
except OSError:
    print("Another operating-system error occurred.")

The reverse order would make the FileNotFoundError handler unreachable in practice because OSError would match first.

Binding the Exception Object

try:
    result = int(user_text)
except ValueError as error:
    print(f"Conversion failed: {error}")

The bound object can provide a message or, for selected exceptions, attributes such as a filename or error number. Error messages are useful for diagnostics but should not automatically be shown to users, because they may reveal internal paths or implementation details.

5. else and finally

The optional else block executes only when the try block completes without raising an exception. It is useful for code that depends on successful completion but should not itself be treated as part of the risky operation.

try:
    number = int(text)
except ValueError:
    print("Invalid number.")
else:
    print(f"Accepted value: {number}")

The execution possibilities are:

The finally block runs whether the operation succeeds, fails, or exits through a return statement. It is intended for cleanup, such as releasing a resource or restoring program state.

file = None
try:
    file = open("records.txt", encoding="utf-8")
    data = file.read()
finally:
    if file is not None:
        file.close()

In modern Python, a context manager is usually clearer for files:

with open("records.txt", encoding="utf-8") as file:
    data = file.read()

The with statement arranges cleanup even if an exception occurs. It is not an alternative exception handler; it is a resource-management mechanism that works especially well with try/except.

Caution with finally

A return or new exception inside finally can replace an exception or return value from the try block. This is usually confusing and should be avoided unless the behavior is deliberate. Cleanup code should be short and unlikely to fail.

6. Raising Exceptions Explicitly

The raise statement allows a program to signal that a condition violates its contract.

def withdraw(balance, amount):
    if amount < 0:
        raise ValueError("amount cannot be negative")
    if amount > balance:
        raise ValueError("insufficient balance")
    return balance - amount

Raising an exception is appropriate when a function cannot produce a valid result under the supplied conditions. The exception should communicate the nature of the failure without exposing unnecessary internal details.

A bare raise inside an active handler re-raises the current exception while preserving its traceback:

try:
    process_file()
except OSError:
    record_failure()
    raise

This pattern is useful when a layer performs logging or cleanup but does not own the recovery decision. By contrast, raise error can alter traceback information and is generally less appropriate when simply re-raising the current exception.

7. Exception Chaining

Sometimes a low-level exception must be translated into a higher-level exception. Python supports explicit exception chaining with from:

class ConfigurationError(Exception):
    pass

def load_configuration(path):
    try:
        with open(path, encoding="utf-8") as file:
            return file.read()
    except OSError as error:
        raise ConfigurationError(
            f"Unable to load configuration from {path}"
        ) from error

The caller sees a domain-specific ConfigurationError, while the original OSError remains available as its cause. This preserves diagnostic information and expresses abstraction clearly: the file system failure has become a configuration failure for the caller.

Python may also display implicit context when one exception occurs while handling another. Explicit from is preferable when the relationship is intentional. The form raise NewError(...) from None suppresses display of the chained context, but it should be used sparingly because it removes useful debugging evidence.

8. Custom Exceptions

Custom exceptions communicate domain concepts more precisely than generic built-ins.

class InvalidRecordError(ValueError):
    """A record does not satisfy the input format."""


def parse_record(line):
    fields = line.rstrip("\n").split(",")
    if len(fields) != 3:
        raise InvalidRecordError("A record must contain three fields")
    return fields

Custom exceptions should normally inherit from Exception or an appropriate built-in subclass. A hierarchy can represent related failure categories:

class ApplicationError(Exception):
    pass

class DataError(ApplicationError):
    pass

class ConfigurationError(ApplicationError):
    pass

Callers can handle a specific class when they know how, or the common base class when they want a general application-level response.

9. Avoiding Harmful Exception Practices

Overly broad handling

This code is usually poor practice:

try:
    result = perform_task()
except Exception:
    print("Something went wrong")

It may catch programming defects, hide valuable tracebacks, and make the program appear successful when it is not. If broad handling is required at an application boundary, it should log diagnostic details, avoid exposing sensitive information, and terminate or respond in a clearly defined way.

Catching BaseException is almost never appropriate in application code because it also catches KeyboardInterrupt and SystemExit.

Silent handling

try:
    optional_operation()
except OSError:
    pass

Silence is justified only when failure is explicitly harmless and documented. Otherwise, it creates unreliable behavior and makes troubleshooting difficult. Better alternatives include a fallback, a warning, a structured log, or propagation.

Catching too much work

Keep the protected region small. If parsing, database updates, and report generation all occur in one try block, an exception from any stage may be incorrectly reported as a parsing problem.

Using exceptions for ordinary control flow

Exceptions are appropriate for exceptional or invalid conditions, not for routine choices that can be expressed clearly with a test. For example, use if key in mapping when absence is an expected, frequent decision; use try/except KeyError when attempting the lookup directly is clearer and race-free in the relevant context.

Logging and re-raising repeatedly

Logging the same exception at every layer creates duplicate records. A lower layer should usually add context and propagate; an outer boundary should log once with the complete context, unless separate audit requirements justify additional records.

10. A Professional File-Processing Example

The following function reads comma-separated records and distinguishes recoverable input problems from environmental failures.

class RecordFormatError(ValueError):
    pass


def parse_record(line, line_number):
    fields = line.rstrip("\n").split(",")
    if len(fields) != 3:
        raise RecordFormatError(
            f"line {line_number}: expected 3 fields"
        )
    return fields


def load_records(path):
    records = []
    try:
        with open(path, encoding="utf-8") as file:
            for line_number, line in enumerate(file, start=1):
                try:
                    records.append(parse_record(line, line_number))
                except RecordFormatError as error:
                    print(f"Skipping invalid record: {error}")
    except FileNotFoundError as error:
        raise ConfigurationError(
            f"Input file does not exist: {path}"
        ) from error
    except PermissionError as error:
        raise ConfigurationError(
            f"Input file cannot be read: {path}"
        ) from error
    return records

The inner handler treats an invalid record as a data-quality problem and continues with later records. The outer handlers treat missing or inaccessible files as application-level configuration failures. The context manager closes the file automatically. Other unexpected exceptions propagate rather than being misclassified.

11. Illustrative Case Study: Importing Student Results

Background

A college application imports student results from a text file. Each row should contain an identifier, a course code, and a numeric mark. The import must continue when one row is malformed, but it must not pretend that a missing file was successfully imported.

Problem

An early implementation used:

try:
    import_results(path)
except Exception:
    return []

This design silently converted every failure into an empty result set. A misspelled path, permission failure, programming defect, or malformed row could therefore appear to be a legitimate file containing no records.

Analysis

The failures have different meanings and require different responses:

• a malformed row is a data-quality issue and can be reported while processing continues;
• a missing file is an operational or configuration issue and should stop the import;
• a permission failure requires administrative action;
• an unexpected programming exception should remain visible to developers.

Approach and Solution

The revised design uses a custom RecordFormatError for row validation, specific handlers for file-system exceptions, and exception chaining when converting OSError into ConfigurationError. The user interface reports the number of accepted and rejected rows, while technical details are logged for support staff.

Outcome

The application no longer confuses failure with an empty dataset. Valid rows are imported, invalid rows are identified, missing files produce an actionable message, and unexpected defects generate tracebacks during testing.

Learning Points

Exception classes communicate policy. Recovery should occur only where recovery is meaningful. A handler should preserve diagnostic information and should not conceal an unsuccessful operation behind a plausible default value.

12. Testing Exception Behavior

Exception handling requires deliberate tests, not only successful-path tests. Test at least:

• valid input;
• invalid values and malformed records;
• missing files;
• permission or access failures where practical;
• empty input;
• unexpected exceptions that should propagate;
• cleanup after failure;
• custom exception messages and chained causes.

With pytest, an expected exception can be tested as follows:

import pytest


def test_negative_amount_is_rejected():
    with pytest.raises(ValueError, match="negative"):
        withdraw(100, -5)

Tests should verify observable behavior rather than implementation details. For exception chains, inspect error.__cause__ when preserving the underlying cause is part of the function’s contract.

13. Best-Practice Framework

A practical decision process is:

1. Can the condition be prevented by validation? Validate early when that is clearer.
2. What exact exception can occur? Identify and handle the narrowest relevant class.
3. Can this layer recover? If not, add useful context and propagate.
4. Does the handler preserve evidence? Avoid silent suppression and unnecessary replacement.
5. Is cleanup guaranteed? Use with or finally for resources and state restoration.
6. Is the user-facing response safe? Do not expose secrets, internal paths, or stack traces in production interfaces.
7. Can the behavior be tested? Include both handled and propagated failures.

In production systems, logs should contain enough context to diagnose the event, but should not include passwords, access tokens, or unnecessary personal data. Exception handling is therefore also a matter of security, privacy, and operational governance.

Chapter Summary

Python exceptions are objects arranged in an inheritance hierarchy. A raised exception interrupts normal execution and propagates through the call stack until a matching handler is found. try identifies risky operations, except handles selected failures, else runs after successful completion, and finally performs essential cleanup. The raise statement signals contract violations or re-raises failures, while exception chaining preserves the relationship between low-level causes and higher-level domain errors.

Professional exception handling is specific, purposeful, and observable. It avoids catching BaseException or unrelated errors, avoids silent pass statements, keeps protected regions narrow, and uses context managers for resource management. Custom exceptions can express application concepts, and testing should cover both recovery and propagation. Good exception design improves reliability without concealing defects.

Key Terms

Exception: An object representing an abnormal condition during execution.

Exception hierarchy: The inheritance structure connecting Python exception classes.

BaseException: The root class of Python’s built-in exception hierarchy.

Exception: The usual base class for application-level exceptions.

Handler: An except clause that responds to a matching exception.

Propagation: Movement of an unhandled exception through calling functions.

Stack unwinding: The process of leaving function frames while searching for a handler.

Traceback: Diagnostic record showing the exception and call path.

try: Statement block containing operations that may raise exceptions.

except: Clause that handles selected exception types.

else: Block executed when the try block completes without exception.

finally: Block executed regardless of whether an exception occurs.

raise: Statement used to signal or re-signal an exception.

Exception chaining: Preservation of a relationship between a new exception and its cause.

Custom exception: Programmer-defined exception class for domain-specific failures.

Context manager: Object supporting controlled setup and cleanup, commonly used with with.

Broad exception handling: Catching a very general class such as Exception.

Silent failure: Suppressing an error without reporting or compensating for it.

Recovery: A meaningful action that allows a program to continue safely.

Review Questions

Short Answer Questions

1. What is the difference between a syntax error and a runtime exception?
2. Why should specific exception handlers generally precede general handlers?
3. What is the purpose of the else clause in a try statement?
4. Why is BaseException rarely caught in application code?
5. What information does exception chaining preserve?

Long Answer Questions

1. Explain Python’s exception propagation model, including stack unwinding and tracebacks.
2. Compare the roles of try, except, else, and finally, using a file-processing example.
3. Explain how custom exceptions improve the design of a modular application.
4. Discuss the risks of broad and silent exception handling and propose alternatives.
5. Describe how context managers and exception handling cooperate to guarantee resource cleanup.

Analytical / Application Questions

1. Design an exception strategy for a program that reads thousands of data records, where malformed rows should be skipped but an unavailable file should stop processing.
2. A function catches OSError and raises DataImportError without preserving the original exception. Analyse the diagnostic consequences and show how exception chaining should be applied.
3. Review a large try block containing input conversion, database updates, and report generation. Explain how you would restructure it to distinguish failures and avoid misclassification.

Multiple Choice Questions with Answers

Question 1

Which class is the usual base class for application-defined exceptions?

A. BaseException

B. Exception

C. SystemExit

D. KeyboardInterrupt

Correct Answer: B. Exception

Question 2

What happens when an exception has no matching handler in the current function?

A. Python silently ignores it

B. The program always retries the statement

C. The exception propagates to the calling function

D. Python converts it into a syntax error

Correct Answer: C. The exception propagates to the calling function

Question 3

When does a try statement’s else block execute?

A. Whenever the except block executes

B. Only when the try block completes without an exception

C. Before the try block begins

D. Only when finally is absent

Correct Answer: B. Only when the try block completes without an exception

Question 4

Which block is intended for cleanup that must occur whether or not an exception is raised?

A. try

B. except

C. else

D. finally

Correct Answer: D. finally

Question 5

Which exception is most appropriate when a string has the correct general type for conversion but contains an invalid value?

A. ValueError

B. IndexError

C. PermissionError

D. NameError

Correct Answer: A. ValueError

Question 6

What does raise NewError("message") from error do?

A. Suppresses all exceptions permanently

B. Creates an explicit cause relationship between exceptions

C. Reruns the failed statement automatically

D. Converts the error into a warning

Correct Answer: B. Creates an explicit cause relationship between exceptions

Question 7

Why should a try block generally be kept narrow?

A. To prevent Python from creating exception objects

B. To ensure every exception becomes a syntax error

C. To avoid misclassifying failures from unrelated operations

D. To make finally execute twice

Correct Answer: C. To avoid misclassifying failures from unrelated operations

Question 8

Which statement is generally best for opening a file and ensuring it is closed?

A. with open(path) as file:

B. except open(path):

C. else open(path) as file:

D. raise open(path) as file:

Correct Answer: A. with open(path) as file:

Question 9

What is the usual effect of a bare raise inside an active exception handler?

A. It raises a new SyntaxError

B. It re-raises the current exception

C. It suppresses the current exception

D. It terminates Python without traceback information

Correct Answer: B. It re-raises the current exception

Question 10

Why is except Exception: pass often poor practice?

A. It catches no ordinary exceptions

B. It prevents all cleanup mechanisms from working

C. It may hide defects and provide no diagnostic or recovery action

D. It is valid only inside a class definition

Correct Answer: C. It may hide defects and provide no diagnostic or recovery action