Object-Oriented and Pythonic Programming

Introduction

Object-oriented programming (OOP) is a way of designing software around objects that combine data with the operations that use that data. Python supports OOP while also retaining procedural, functional, and scripting styles. Effective Python programmers therefore do more than create classes: they choose the simplest design that makes a program understandable, reusable, testable, and maintainable.

For data science, OOP is useful when a program must represent entities such as datasets, models, experiments, pipelines, users, or reports. Pythonic programming complements OOP by encouraging readable code, iteration over indexing, composition over unnecessary inheritance, explicit interfaces, and appropriate use of the standard library.

Learning Objectives

By the end of this chapter, you should be able to:

• define classes and create objects;
• distinguish class attributes, instance attributes, methods, and properties;
• apply encapsulation and controlled access to state;
• use inheritance, composition, and polymorphism appropriately;
• create and use data classes;
• implement iterators and generators;
• write and apply decorators;
• manage resources with context managers;
• evaluate designs using Pythonic principles and professional coding practices.

1. Objects, Classes, and State

A class is a blueprint describing the data and behavior associated with a type of object. An object, also called an instance, is a concrete value created from that class. Attributes represent state; methods represent behavior.

class Student:
    university = "North Valley University"  # class attribute

    def __init__(self, name, student_id):
        self.name = name                    # instance attributes
        self.student_id = student_id

    def describe(self):
        return f"{self.name} ({self.student_id})"

s1 = Student("Amina", "DS101")
print(s1.describe())

The parameter self refers to the object on which a method operates. It is supplied automatically when the method is called through an instance. The constructor-like method __init__ initializes an already-created object; it does not itself create the object.

A class attribute is shared by instances unless an instance receives an attribute with the same name. An instance attribute normally belongs to one object. Mutable class attributes require special caution:

class Incorrect:
    records = []                 # shared by every instance

class Correct:
    def __init__(self):
        self.records = []        # separate list for each instance

Identity, equality, and representation

Object identity answers whether two references point to the same object and is tested with is. Equality asks whether two objects have equivalent values and is tested with ==. These concepts must not be confused.

first = [1, 2]
second = [1, 2]
third = first

first == second   # True: equal contents
first is second   # False: different objects
first is third    # True: same object

Special methods, often called dunder methods, allow objects to cooperate with Python syntax. Common examples include __repr__, __str__, __len__, __iter__, and __eq__.

class Measurement:
    def __init__(self, value, unit):
        self.value = value
        self.unit = unit

    def __repr__(self):
        return f"Measurement({self.value!r}, {self.unit!r})"

    def __str__(self):
        return f"{self.value} {self.unit}"

__repr__ should be useful to developers, while __str__ should be suitable for ordinary display.

2. Encapsulation and Interfaces

Encapsulation means keeping an object's implementation details controlled behind a clear interface. Python does not enforce private fields in the same way as some languages. Instead, it uses conventions and mechanisms:

• _name means “internal use” by convention;
• __name triggers name mangling, mainly to prevent accidental clashes in subclasses;
• properties provide controlled access to an attribute;
• methods can enforce validation and invariants.

An invariant is a condition that should remain true for a valid object. For example, a temperature in Kelvin should not be negative.

class Temperature:
    def __init__(self, kelvin):
        self.kelvin = kelvin

    @property
    def kelvin(self):
        return self._kelvin

    @kelvin.setter
    def kelvin(self, value):
        if value < 0:
            raise ValueError("Kelvin temperature cannot be negative")
        self._kelvin = value

    @property
    def celsius(self):
        return self._kelvin - 273.15

The property makes temperature.kelvin appear like ordinary data while allowing validation. A property is appropriate when access has behavior, validation, conversion, or a stable public interface. It should not be used merely to imitate another language's getters and setters.

Python's @property is a built-in decorator. A decorator is a callable that receives a function or class and returns a modified function or class. Decorators are discussed later in this chapter.

3. Inheritance, Composition, and Polymorphism

Inheritance

Inheritance creates a relationship in which a subclass receives behavior from a base class and may extend or specialize it.

class Classifier:
    def predict(self, features):
        raise NotImplementedError

class RuleClassifier(Classifier):
    def predict(self, features):
        return [1 if value >= 0.5 else 0 for value in features]

The subclass satisfies the interface established by Classifier. super() can call behavior from a parent class:

class LoggedClassifier(RuleClassifier):
    def predict(self, features):
        result = super().predict(features)
        print(f"Generated {len(result)} predictions")
        return result

Inheritance is most suitable for a genuine “is-a” relationship and a stable shared abstraction. It becomes harmful when subclasses inherit irrelevant behavior, when the hierarchy is deep, or when classes are changed merely to reuse a small piece of code.

Composition

Composition builds a class from other objects. It expresses a “has-a” relationship and often produces more flexible designs.

class DataLoader:
    def load(self, source):
        return source

class Pipeline:
    def __init__(self, loader):
        self.loader = loader

    def run(self, source):
        data = self.loader.load(source)
        return [x * 2 for x in data]

pipeline = Pipeline(DataLoader())

The Pipeline does not need to inherit from DataLoader; it uses a loader supplied from outside. This is an example of dependency injection. A different loader—perhaps one for CSV files, databases, or test data—can be substituted without rewriting the pipeline.

Polymorphism

Polymorphism means that different objects can be used through a common operation. In Python this commonly arises through duck typing: an object is suitable if it provides the required behavior, regardless of its declared class.

class CsvExporter:
    def export(self, rows):
        return "csv data"

class JsonExporter:
    def export(self, rows):
        return "json data"

def save_report(exporter, rows):
    return exporter.export(rows)

save_report depends on the export operation, not on a particular inheritance tree. This reduces coupling. If an object supplies the expected method, it can participate in the operation.

Abstract interfaces

The abc module supports explicit abstract base classes:

from abc import ABC, abstractmethod

class Storage(ABC):
    @abstractmethod
    def save(self, record):
        pass

A subclass must implement save before it can be instantiated. Abstract base classes are useful when a project benefits from a formally documented contract. They are not required for every polymorphic design.

4. Data Classes

Many classes primarily store structured data. A data class reduces repetitive code for initialization, representation, and comparison.

from dataclasses import dataclass, field

@dataclass
class Experiment:
    name: str
    learning_rate: float
    tags: list[str] = field(default_factory=list)

experiment = Experiment("baseline", 0.01)

default_factory=list is essential: it creates a new list for each instance. Using tags: list[str] = [] would incorrectly share one mutable list.

Useful options include:

• frozen=True, which makes ordinary field assignment unavailable after creation;
• order=True, which generates ordering methods where appropriate;
• kw_only=True, which requires keyword arguments in supported Python versions;
• slots=True, which can reduce per-instance memory use and prevent arbitrary attributes in supported versions.

A frozen data class prevents reassignment but does not automatically make every contained object immutable. A frozen object containing a list can still contain a mutable list. Data classes are ideal for records, configurations, value objects, and results; they are less suitable when a class has complex lifecycle behavior or many invariants.

Type annotations improve documentation and tooling but are not, by themselves, runtime validation. For untrusted input, explicit validation is still required.

5. Iterables, Iterators, and Generators

An iterable is an object that can produce an iterator, usually through __iter__. An iterator maintains its current position and supplies values through __next__. When no values remain, __next__ raises StopIteration.

class CountUp:
    def __init__(self, stop):
        self.current = 0
        self.stop = stop

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.stop:
            raise StopIteration
        value = self.current
        self.current += 1
        return value

A for loop obtains an iterator and repeatedly calls next() until StopIteration. An iterator is generally stateful and is usually exhausted after one pass.

A generator is a simpler way to create an iterator. A function containing yield returns a generator object. Execution pauses at each yield and resumes later.

def squares(limit):
    for number in range(limit):
        yield number * number

for value in squares(4):
    print(value)

Generators are valuable for large files, streams, and pipelines because they produce values lazily rather than storing all results at once. If a process handles n values and stores them all, storage may grow in proportion to n; a generator can often maintain storage proportional to the size of its current working state. This is a design observation rather than a universal guarantee: downstream operations such as list(generator) materialize all values.

Generator expressions provide compact syntax:

positive = (x for x in values if x > 0)

Use a list when repeated traversal or random indexing is required. Use an iterator or generator when sequential, one-pass, lazy processing is preferable.

6. Decorators

A decorator wraps a function, method, or class to add behavior without editing its original body. The following decorator measures execution time:

from functools import wraps
from time import perf_counter

def timed(function):
    @wraps(function)
    def wrapper(*args, **kwargs):
        start = perf_counter()
        result = function(*args, **kwargs)
        elapsed = perf_counter() - start
        print(f"{function.__name__}: {elapsed:.6f} seconds")
        return result
    return wrapper

@timed
def process(data):
    return sum(data)

@timed is equivalent to assigning process = timed(process). wraps preserves metadata such as the original function name and documentation.

Decorators are appropriate for cross-cutting concerns such as logging, authorization, caching, retries, validation, and measurement. They should be used carefully because several nested decorators can obscure control flow. A decorator should preserve the wrapped function's return value and should document changed behavior.

7. Context Managers and Resource Safety

A context manager defines setup and cleanup around a block of code. The with statement ensures cleanup even when an exception occurs.

with open("observations.txt", encoding="utf-8") as file:
    text = file.read()

The file is closed after the block. This is safer than manually calling open and close on separate lines.

A custom context manager can implement __enter__ and __exit__:

class TemporaryMessage:
    def __enter__(self):
        print("Start")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("Finish")
        return False

Returning False from __exit__ allows an exception to propagate. Returning True suppresses it and should be done only deliberately, because suppression can hide failures.

For simple generator-based context managers, use contextlib.contextmanager:

from contextlib import contextmanager

@contextmanager
def managed_message():
    print("Start")
    try:
        yield
    finally:
        print("Finish")

Context managers are useful for files, database transactions, locks, temporary directories, network connections, and resource limits. Their central principle is deterministic cleanup.

8. A Data-Science Application: Lazy Experiment Processing

The following illustrative case shows how several ideas can work together.

Background and problem

A research team receives experiment records from a large text file. Each record contains an experiment name and a score. The team needs to validate records, retain only valid scores, compute a summary, and export results. Loading the entire file into memory is undesirable, and the processing logic should be testable.

Approach

A data class models a record. A generator reads records lazily. A validation decorator records invalid input, while composition allows the processor to receive any compatible source.

from dataclasses import dataclass
from functools import wraps

@dataclass(frozen=True)
class Result:
    name: str
    score: float

def require_score(function):
    @wraps(function)
    def wrapper(name, score_text):
        try:
            score = float(score_text)
        except ValueError:
            return None
        if not 0 <= score <= 100:
            return None
        return function(name, score)
    return wrapper

@require_score
def make_result(name, score):
    return Result(name.strip(), score)

def read_results(lines):
    for line in lines:
        name, score_text = line.rstrip("\\n").split(",", maxsplit=1)
        result = make_result(name, score_text)
        if result is not None:
            yield result

class ResultProcessor:
    def __init__(self, source):
        self.source = source

    def average(self):
        total = 0.0
        count = 0
        for result in self.source:
            total += result.score
            count += 1
        return total / count if count else None

Analysis

The data class supplies a clear value representation. frozen=True prevents accidental reassignment of name and score. The generator avoids materializing every record. The decorator isolates validation, and composition allows ResultProcessor to work with a generator, list, database cursor, or another iterable.

For a small input, a list may be simpler. For a large input, the lazy design is preferable, but a generator is single-use: calling average() consumes it. If multiple summaries are needed, the source must be recreated or materialized deliberately. The implementation should also handle malformed lines, missing fields, encoding problems, and audit requirements in production.

Outcome and learning points

The design separates representation, validation, input traversal, and analysis. This improves testing: each component can be tested independently. It also demonstrates that OOP does not require every operation to be a method. A generator function and a decorator can be clearer than a large class hierarchy.

9. Pythonic Design Principles

Pythonic programming emphasizes clarity and appropriate use of the language. Important practices include:

• Prefer readable names and small functions.
• Follow the principle that there should be one obvious way to perform a task, while recognizing that context matters.
• Use for item in collection rather than manually managing indexes when indexes are unnecessary.
• Use enumerate, zip, comprehensions, and standard-library tools appropriately.
• Prefer composition and dependency injection when they reduce coupling.
• Use exceptions for exceptional conditions, not ordinary control flow.
• Validate at boundaries, especially when handling files, APIs, or user input.
• Avoid mutable default arguments and accidental shared state.
• Keep public interfaces stable and implementation details internal.
• Write docstrings and tests for public classes and functions.
• Use if __name__ == "__main__": to separate reusable definitions from script execution.

The Zen of Python, displayed by running import this, is a useful cultural guide rather than a substitute for judgment. Readability, explicitness, and simplicity must be balanced against performance, reliability, and domain requirements.

10. Testing, Performance, and Professional Practice

Classes should be tested through their public behavior. Tests should cover valid inputs, invalid inputs, boundary conditions, exceptions, and resource cleanup. Dependency injection makes testing easier because a real database or network service can be replaced with a controlled test double.

Object-oriented abstraction has costs. Classes introduce names, state, and indirection. Inheritance can create tight coupling. Decorators can hide execution order. Generators complicate debugging because computation is delayed. Context managers can suppress exceptions if implemented incorrectly.

Measure performance rather than assuming that a class, generator, or decorator is faster. In data science, vectorized operations in libraries such as NumPy or pandas may be more efficient than Python-level loops for numerical arrays. Nevertheless, generators remain valuable for input/output streams and pipelines. Performance decisions should consider time, memory, maintainability, and correctness together.

Security and governance are also relevant. Do not deserialize untrusted data with unsafe mechanisms. Do not allow decorators, plugins, or dynamic imports to execute untrusted code without controls. Protect credentials and personal data, log access appropriately, and preserve reproducibility by recording software versions, configuration, and random seeds where relevant. A data-processing class should make assumptions about missing values, validation, and transformations explicit so that results can be audited.

Contemporary Developments

Modern Python development increasingly combines object-oriented design with type checking, automated testing, packaging, and data validation tools. Structural protocols can express behavior-based interfaces without requiring inheritance. Asynchronous context managers and asynchronous iterators support network and concurrent applications. Data classes and typed models improve clarity in pipelines, but external validation remains necessary for untrusted data.

Artificial intelligence systems also benefit from these patterns. A model wrapper can provide a common predict interface, a generator can stream records or responses, a context manager can manage temporary resources, and decorators can add logging or retry policies. However, abstraction must not conceal model limitations, data provenance, fairness risks, privacy obligations, or uncertainty. Good design makes these concerns visible and testable.

Chapter Summary

Object-oriented Python combines state and behavior in classes and objects. Instance attributes belong to individual objects, whereas class attributes are shared. Encapsulation uses conventions, properties, and validation to protect invariants. Inheritance supports genuine subtype relationships, while composition often provides greater flexibility. Polymorphism allows different implementations to satisfy a common operation, frequently through duck typing or explicit abstract interfaces.

Data classes efficiently represent structured values, but mutable defaults require default_factory, and type annotations do not automatically validate input. Iterables provide values for iteration; iterators maintain traversal state; generators provide lazy computation. Decorators add reusable cross-cutting behavior, and context managers guarantee setup and cleanup around resource-sensitive operations. Pythonic design favors readable, simple, loosely coupled solutions and uses OOP where it clarifies the domain rather than applying classes everywhere.

Key Terms

Class: A definition describing attributes and behavior for a type of object.

Object: A concrete instance of a class.

Instance attribute: State stored on an individual object.

Class attribute: An attribute associated with the class and commonly shared by instances.

Method: A function defined within a class.

Encapsulation: Controlled organization and access to an object's state and behavior.

Property: Attribute-like access implemented through methods.

Inheritance: Creation of a specialized class from a base class.

Composition: Building an object from other objects.

Polymorphism: Use of different objects through a common operation or interface.

Duck typing: Behavioral compatibility based on available operations rather than declared type.

Data class: A class designed primarily to store structured data.

Iterable: An object capable of producing an iterator.

Iterator: A stateful object that supplies values through __next__.

Generator: A lazy iterator-producing function or expression using yield.

Decorator: A callable that transforms or extends a function, method, or class.

Context manager: An object controlling setup and cleanup around a with block.

Invariant: A condition that must remain true for a valid object.

Dependency injection: Supplying a dependency from outside rather than constructing it internally.

Review Questions

Short Answer Questions

1. What is the difference between a class attribute and an instance attribute?
2. Why should mutable data classes commonly use default_factory?
3. What event causes an iterator to raise StopIteration?
4. Why is functools.wraps useful in a decorator?
5. What is the principal resource-management advantage of a context manager?

Long Answer Questions

1. Explain encapsulation in Python and demonstrate how a property can preserve an object invariant.
2. Compare inheritance and composition, including one situation in which each is appropriate.
3. Explain polymorphism and duck typing using a data-science example.
4. Describe the differences among an iterable, an iterator, and a generator.
5. Discuss the benefits and limitations of data classes in professional Python applications.

Analytical / Application Questions

1. Design a small processing pipeline that reads records lazily, validates them, and produces a summary. Explain where you would use a generator, data class, and composition.
2. A class hierarchy has six levels and subclasses override many unrelated methods. Analyze the design problems and propose a more Pythonic alternative.
3. A file-processing program sometimes leaves files open after an exception. Identify the likely design weakness and redesign the program using a context manager. Explain how your design handles exceptions.

Multiple Choice Questions with Answers

Question 1

Which statement best describes an instance attribute?

A. It is shared automatically by all classes in a program

B. It stores state associated with a particular object

C. It can only contain immutable values

D. It is created only by a decorator

Correct Answer: B. It stores state associated with a particular object

Question 2

What does a property primarily provide?

A. Controlled attribute-like access implemented with methods

B. Automatic conversion of every object into a data class

C. Multiple inheritance without a base class

D. A guarantee that all values are immutable

Correct Answer: A. Controlled attribute-like access implemented with methods

Question 3

Which design most directly represents composition?

A. A subclass overriding a parent method

B. A class receiving and storing a separate loader object

C. A function raising StopIteration

D. A decorator preserving function metadata

Correct Answer: B. A class receiving and storing a separate loader object

Question 4

What happens when a generator reaches a yield statement?

A. It terminates permanently before producing a value

B. It pauses and can resume from that point later

C. It converts all remaining values into a list

D. It automatically creates a new thread

Correct Answer: B. It pauses and can resume from that point later

Question 5

Why is field(default_factory=list) used in a data class?

A. To create a separate list for each instance

B. To make the list globally shared

C. To sort the list automatically

D. To prevent the list from being modified

Correct Answer: A. To create a separate list for each instance

Question 6

What is the purpose of functools.wraps in a decorator?

A. It converts a function into an iterator

B. It suppresses all exceptions from the function

C. It preserves useful metadata of the wrapped function

D. It makes every function asynchronous

Correct Answer: C. It preserves useful metadata of the wrapped function

Question 7

Which statement about an iterator is correct?

A. It must support random indexing

B. It supplies values through __next__ and eventually raises StopIteration

C. It always stores every value in memory

D. It can never be used in a for loop

Correct Answer: B. It supplies values through __next__ and eventually raises StopIteration

Question 8

What is the principal purpose of a context manager?

A. To guarantee setup and cleanup around a block

B. To enforce inheritance between two classes

C. To replace all exception handling

D. To make a function run faster

Correct Answer: A. To guarantee setup and cleanup around a block

Question 9

Which approach best supports polymorphism through duck typing?

A. A function requiring one exact concrete class

B. A function using an object because it provides the required method

C. A function accepting only dictionaries

D. A function avoiding all interfaces

Correct Answer: B. A function using an object because it provides the required method

Question 10

Which practice is most Pythonic when an operation only needs sequential access to a large data stream?

A. Materialize the entire stream before processing

B. Use a lazy iterator or generator

C. Use a six-level inheritance hierarchy

D. Store the stream in a mutable class attribute

Correct Answer: B. Use a lazy iterator or generator