Functions, Modules and Object-Oriented Programming

Introduction

Data science programs are rarely built as a single sequence of statements. A professional Python application is divided into reusable functions, organized into modules and packages, and designed with classes when data and behaviour naturally belong together. These structures improve readability, testing, collaboration, maintenance and reproducibility.

A data scientist may begin with a short exploratory script, but production work commonly requires a stronger architecture. A data-cleaning operation may be used in several notebooks; a model-evaluation routine may be shared across projects; and a data-ingestion system may need clearly defined objects representing files, datasets or database connections. Python provides language features for all of these needs.

This chapter examines functions, namespaces and scope, modules and packages, classes and objects, inheritance, iterators, generators and type hints. The emphasis is not merely on syntax, but on design decisions: when abstraction is useful, how interfaces should be defined, and how reusable code can remain reliable in a data-science environment.

Learning Objectives

By the end of this chapter, you should be able to:

• define and call functions using positional, keyword, default and variable-length parameters;
• explain namespaces, local and global scope, closures and the LEGB name-resolution rule;
• design functions that are cohesive, testable and safe to reuse;
• organize code into modules and packages and import it appropriately;
• define classes, instance attributes, methods and properties;
• apply inheritance and composition appropriately;
• explain and implement iterators and generators;
• use type hints to communicate intended interfaces and improve tooling;
• evaluate design choices in a practical data-science application.

1. Functions as Units of Abstraction

A function is a named, reusable block of code that receives zero or more inputs, performs an operation, and may return a result. Functions reduce duplication and give a program a meaningful vocabulary. Instead of repeatedly writing the steps for standardizing a column, a program can call standardize_column(data, column).

A basic function has the form:

def function_name(parameter1, parameter2):
    """Describe the operation."""
    result = parameter1 + parameter2
    return result

The def statement creates a function object and binds it to a name. The parameters are local names that receive values when the function is called. return terminates the function and sends a value to the caller. If no explicit return is executed, Python returns None.

def mean(values):
    if not values:
        raise ValueError("values must not be empty")
    return sum(values) / len(values)

average = mean([10, 20, 30])
print(average)       # 20.0

The expression for an arithmetic mean is:

Here, xᵢ is the i-th observed value, n is the number of observations, and x̄ is the mean. The formula assumes that the observations are numeric and that n > 0; the function therefore rejects an empty collection rather than causing a less informative division-by-zero error.

1.1 Parameters and Arguments

A parameter is a variable in a function definition; an argument is a value supplied during a call.

def describe(value, label="measurement"):
    return f"{label}: {value}"

print(describe(12.5))
print(describe(12.5, label="temperature"))
print(describe(label="temperature", value=12.5))

Python supports several parameter forms:

Positional-only and keyword-only parameters make an interface explicit:

def normalize(value, /, *, minimum=0.0, maximum=1.0):
    if maximum <= minimum:
        raise ValueError("maximum must exceed minimum")
    return (value - minimum) / (maximum - minimum)

The function must receive value positionally, while minimum and maximum must be named. This can prevent confusing calls and allows the interface to evolve more safely.

1.2 Default Arguments and Mutable Objects

Default values are evaluated once, when the function is defined. A mutable default such as a list can therefore retain data between calls, often unexpectedly.

# Avoid this design
def add_item(item, items=[]):
    items.append(item)
    return items

The safer pattern is to use None as a sentinel:

def add_item(item, items=None):
    if items is None:
        items = []
    items.append(item)
    return items

This creates a fresh list for each call in which no list is supplied.

1.3 Return Values and Multiple Results

Python functions can return a tuple containing several values:

def min_max(values):
    if not values:
        raise ValueError("values must not be empty")
    return min(values), max(values)

lowest, highest = min_max([4, 8, 2])

Returning a structured result is usually clearer than modifying global variables. A function should generally communicate through parameters and return values, with side effects—such as writing a file or changing an external object—made explicit.

1.4 Functions as First-Class Objects

Functions can be assigned to variables, stored in collections and passed to other functions. This supports higher-order operations such as sorting with a key function.

def absolute(value):
    return abs(value)

measurements = [-3, 1, -2]
sorted_values = sorted(measurements, key=absolute)

A lambda expression creates a small anonymous function:

records = [("A", 82), ("B", 74), ("C", 91)]
by_score = sorted(records, key=lambda record: record[1], reverse=True)

Named functions are generally preferable when logic is complex, reused, or important enough to document and test.

2. Scope, Namespaces and Closures

A namespace is a mapping between names and objects. Modules, classes and functions create namespaces, helping prevent unrelated names from colliding. Scope describes where a name can be accessed.

Python’s commonly used name-resolution rule is LEGB:

1. Local: names defined inside the current function;
2. Enclosing: names in surrounding function scopes;
3. Global: names defined at module level;
4. Built-in: names supplied by Python, such as len and sum.

rate = 0.15              # global scope

def calculate_tax(amount):
    adjustment = 1.0     # local scope
    return amount * rate * adjustment

Reading a global value is possible, but changing global state inside functions makes programs difficult to test. If assignment occurs inside a function, Python normally treats that name as local. The global and nonlocal statements can alter this behaviour, but they should be used sparingly.

A closure is an inner function that remembers values from its enclosing scope after the enclosing function has finished.

def make_multiplier(factor):
    def multiply(value):
        return value * factor
    return multiply

triple = make_multiplier(3)
print(triple(5))        # 15

Closures are useful for configurable transformations, but a small class may be clearer when state and behaviour become substantial.

2.1 Side Effects and Purity

A pure function produces the same output for the same inputs and does not modify external state. For example:

def square(value):
    return value * value

A function that writes to a database, changes a list supplied by the caller, or uses the current time has side effects. Side effects are not inherently wrong; data pipelines must read files and save results. However, isolating side effects at system boundaries makes computational logic easier to test and reuse.

Good function design commonly follows these principles:

• one clear responsibility;
• meaningful names and a precise docstring;
• explicit assumptions and validation;
• limited dependence on global state;
• predictable return types;
• errors reported with appropriate exceptions;
• no unnecessary mutation of caller-owned data.

3. Modules and Packages

A module is a Python file containing definitions and executable statements. It can hold functions, classes, constants and configuration. A package is a directory structure used to organize related modules.

Suppose a project contains:

project/
    analysis/
        __init__.py
        cleaning.py
        metrics.py
    run_analysis.py

A function in cleaning.py can be imported as follows:

from analysis.cleaning import remove_missing

Alternatively:

import analysis.cleaning as cleaning
cleaned = cleaning.remove_missing(data)

The second style makes the source of the function visible and reduces the risk of name collisions. Wildcard imports such as from module import * are generally discouraged because they obscure which names enter the current namespace.

3.1 The Module Execution Context

Every module has a special variable called __name__. When a file is run directly, its value is usually "__main__"; when imported, it is the module’s import name. This supports a useful pattern:

def main():
    print("Running analysis")

if __name__ == "__main__":
    main()

The function is available for import without automatically running the command-line behaviour.

3.2 Packages and Import Design

An __init__.py file can mark a directory as a package and define the package’s public interface. Modern Python also supports namespace packages without that file, but including an explicit package structure remains common for teaching and many applications.

A package should have a coherent responsibility. A data-science project might separate:

• io.py for reading and writing data;
• cleaning.py for validation and transformation;
• features.py for feature construction;
• models.py for model-related abstractions;
• reporting.py for summaries and visual outputs.

Circular imports occur when module A imports module B while module B imports module A. They often indicate that responsibilities are mixed. Moving shared definitions into a lower-level module or redesigning dependencies usually provides a better solution than relying on import order.

3.3 Reproducible Environments

Reusable modules should declare their dependencies and avoid relying on a developer’s unrecorded environment. Professional projects commonly use a virtual environment, a dependency specification, version control and automated tests. Imports should be explicit, and public functions should have stable, documented interfaces.

4. Classes and Objects

Object-oriented programming (OOP) models a system as objects that combine state and behaviour. A class is a blueprint; an object, or instance, is a concrete value created from that class.

class DatasetSummary:
    def __init__(self, name, rows):
        self.name = name
        self.rows = rows

    def describe(self):
        return f"{self.name}: {self.rows} rows"

summary = DatasetSummary("sales", 1200)
print(summary.describe())

__init__ initializes an instance after it is created. The first parameter of an instance method is conventionally named self and refers to the current instance. Instance attributes belong to individual objects.

4.1 Encapsulation and Invariants

Encapsulation means placing data and operations that belong together behind a coherent interface. Python does not enforce private fields in the same way as some languages. A leading underscore communicates that an attribute is intended for internal use.

An invariant is a condition that should remain true for a valid object. For a dataset summary, the number of rows should not be negative. Validation in the constructor protects this invariant:

class DatasetSummary:
    def __init__(self, name: str, rows: int):
        if rows < 0:
            raise ValueError("rows cannot be negative")
        self.name = name
        self.rows = rows

A property provides method-like control with attribute-like syntax:

class Temperature:
    def __init__(self, celsius: float):
        self.celsius = celsius

    @property
    def fahrenheit(self) -> float:
        return self.celsius * 9 / 5 + 32

The conversion formula assumes temperatures are measured in degrees Celsius and Fahrenheit. It follows the affine relationship F = (9/5)C + 32, where C is the Celsius value and F is the Fahrenheit value.

4.2 Class Attributes, Methods and Special Methods

A class attribute is shared by the class and, unless shadowed, its instances. Instance methods operate on one object; class methods receive the class as cls; static methods receive neither automatically.

Special methods, surrounded by double underscores, integrate objects with Python syntax. For example, __repr__ controls a useful representation, __len__ supports len(object), and __iter__ supports iteration.

For simple data-bearing classes, dataclass can generate common methods:

from dataclasses import dataclass

@dataclass
class Observation:
    identifier: str
    value: float

A dataclass improves clarity when the primary purpose of a class is to store structured data. It does not remove the need to validate data or select suitable types.

5. Inheritance and Composition

Inheritance allows a subclass to reuse or specialize a parent class. The relationship should represent a genuine “is-a” relationship.

class DataSource:
    def read(self):
        raise NotImplementedError

class MemorySource(DataSource):
    def __init__(self, records):
        self.records = records

    def read(self):
        return self.records

MemorySource is a kind of DataSource. A subclass can override a method, and super() can call a parent implementation. Polymorphism allows code to work with any object supporting the expected interface:

def count_records(source: DataSource) -> int:
    return len(source.read())

Composition means building a class from other objects. A Report may contain a data source, validator and formatter. Composition is often more flexible than deep inheritance because components can be replaced independently. In data science, composition is particularly useful for pipelines in which cleaning, feature generation and prediction are separate stages.

Inheritance should not be used merely to avoid writing a few lines of code. Deep class hierarchies can make behaviour difficult to trace. Prefer a small, stable interface and composition where the relationship is “has-a” rather than “is-a”.

6. Iterators and Generators

An iterable is an object that can produce its members one at a time. Lists, tuples, dictionaries, files and strings are iterable. An iterator maintains its current position and implements __next__(), which returns the next value or raises StopIteration when exhausted.

The iter() function obtains an iterator:

values = iter([10, 20, 30])
print(next(values))
print(next(values))

A for loop automatically obtains an iterator and handles StopIteration.

A generator is a convenient way to create an iterator. A function containing yield suspends at each yield point and resumes when the next value is requested.

def positive_values(values):
    for value in values:
        if value > 0:
            yield value

for value in positive_values([-2, 4, 0, 7]):
    print(value)

Generators are valuable for large data sets because they support lazy evaluation: values are produced when needed rather than all being stored in memory. This does not make the underlying computation free; it changes when values are computed and how much intermediate memory is required.

A generator pipeline can process records incrementally:

def read_records(lines):
    for line in lines:
        yield line.strip()

def nonempty(records):
    for record in records:
        if record:
            yield record

records = nonempty(read_records(lines))

The pipeline can process a stream without constructing a second complete list. However, a generator is single-use: after exhaustion, it must be recreated to iterate again.

Generator expressions provide a compact form:

squares = (value * value for value in range(1_000_000))

By contrast, a list comprehension creates all results immediately:

squares = [value * value for value in range(1_000_000)]

The choice depends on whether repeated access, indexing and immediate materialization are required.

7. Type Hints and Contracts

Type hints annotate intended types without changing the basic runtime semantics of Python. They improve readability, editor support, documentation and static analysis.

def median(values: list[float]) -> float:
    ordered = sorted(values)
    n = len(ordered)
    if n == 0:
        raise ValueError("values must not be empty")
    middle = n // 2
    if n % 2 == 1:
        return ordered[middle]
    return (ordered[middle - 1] + ordered[middle]) / 2

The function accepts a list of floating-point values and returns a float. The type hint does not itself guarantee that every caller supplies valid data; runtime validation may still be necessary.

For mappings and sequences, modern Python supports generic notation such as list[str] and dict[str, float]. Optional values can be represented with str | None. A Protocol can describe behaviour rather than inheritance:

from typing import Protocol

class Reader(Protocol):
    def read(self) -> list[dict[str, object]]: ...

Any object with a compatible read method can satisfy this structural interface. This supports flexible, testable designs.

Type hints should communicate meaningful contracts rather than create excessive complexity. They are especially valuable at module boundaries, public functions and data-transformation stages.

8. Applied Case: A Reusable Sales-Data Processing Component

Background

The following is an illustrative case. A small retail organization receives daily sales records from different sources. Analysts need to remove invalid records, calculate revenue and process files that may be too large to load entirely into memory.

Problem

An initial script contains duplicated logic in three notebooks. It uses global variables, silently accepts negative quantities and loads every file into a list. Changes in one notebook are not reflected in the others.

Analysis

The problem has four design dimensions:

1. validation should be a reusable function;
2. source-specific reading should be separated from transformation;
3. records should be processed lazily where possible;
4. a class can represent a validated sale and preserve its invariants.

Approach and Solution

from dataclasses import dataclass
from typing import Iterable, Iterator

@dataclass(frozen=True)
class Sale:
    product: str
    quantity: int
    unit_price: float

    def __post_init__(self):
        if not self.product:
            raise ValueError("product is required")
        if self.quantity <= 0:
            raise ValueError("quantity must be positive")
        if self.unit_price < 0:
            raise ValueError("unit_price cannot be negative")

    @property
    def revenue(self) -> float:
        return self.quantity * self.unit_price


def valid_sales(rows: Iterable[dict[str, object]]) -> Iterator[Sale]:
    for row in rows:
        try:
            yield Sale(
                product=str(row["product"]),
                quantity=int(row["quantity"]),
                unit_price=float(row["unit_price"]),
            )
        except (KeyError, TypeError, ValueError):
            continue


def total_revenue(sales: Iterable[Sale]) -> float:
    return sum(sale.revenue for sale in sales)

The Sale class owns the rules for a valid sale. valid_sales is a generator, so it can consume an iterable of rows incrementally. total_revenue accepts any iterable of Sale objects, making it compatible with a list, generator or custom data source.

If three valid sales are:

rows = [
    {"product": "Pen", "quantity": 2, "unit_price": 1.50},
    {"product": "Book", "quantity": 1, "unit_price": 12.00},
    {"product": "Pen", "quantity": 3, "unit_price": 1.50},
]

their revenue is calculated as:

where R is total revenue, qᵢ is the quantity in sale i, pᵢ is its unit price, and n is the number of valid sales. Thus, R = 2×1.50 + 1×12.00 + 3×1.50 = 19.50. The units are currency units because quantity is measured in items and unit price in currency per item.

sales = valid_sales(rows)
print(total_revenue(sales))  # 19.5

Outcome

The design removes duplicated validation, prevents invalid objects from being created, supports streaming input and allows the revenue function to be tested independently. A production implementation would also log rejected rows rather than silently discarding them, define a policy for malformed data and use an appropriate decimal representation when financial rounding requirements demand it.

Learning Points

• Functions should separate transformations from side effects.
• Classes are useful when data must preserve invariants.
• Generators help process large inputs incrementally.
• Type hints document the expected interface but do not replace validation.
• Business rules, especially financial rules, must be explicit and tested.

9. Testing, Reliability and Professional Practice

Reusable code requires verification. Unit tests should test normal cases, boundary cases and invalid inputs. For the Sale class, useful tests include zero quantity, negative price, missing product and correct revenue calculation.

A function’s docstring should state its purpose, parameters, return value and exceptions when these are not obvious. Naming should reflect domain meaning: total_revenue is more informative than calculate.

Avoid excessive abstraction. A five-line operation does not always need a class hierarchy. Conversely, duplicated validation and hidden state are warning signs that an abstraction may be justified. Refactoring should be guided by changing requirements, repeated patterns and the cost of defects.

Security and governance also matter. Do not place passwords or API keys in modules committed to version control. Validate external input, restrict file access where appropriate, preserve data lineage and record how rejected or transformed records are handled. In regulated contexts, reproducibility requires versioned code, documented dependencies and traceable transformations.

10. Contemporary Developments

Modern Python data-science systems increasingly combine type checking, data validation libraries, workflow orchestration and distributed processing. The underlying principles remain the same: define clear interfaces, separate responsibilities and make assumptions observable. Data classes, protocols and static-analysis tools support maintainability, while generators and iterators remain important for streaming and memory-efficient processing.

Artificial intelligence systems also benefit from these principles. A model-serving application may use a class for a predictor, a generator for incoming records, a module for preprocessing and type hints for the prediction interface. Good software design does not guarantee model validity, fairness or privacy; those concerns require separate evaluation, documentation and governance.

Chapter Summary

Functions provide reusable units of computation and should have clear responsibilities, explicit inputs and predictable outputs. Parameters may be positional, keyword-only, defaulted or variable-length. Scope and namespaces determine how Python resolves names, while minimizing global state improves testability.

Modules and packages organize code into maintainable units. Explicit imports, sensible dependency structure and reproducible environments support professional development. Classes combine state and behaviour, and constructors can enforce invariants. Properties, dataclasses and special methods provide convenient object interfaces.

Inheritance supports genuine “is-a” relationships and polymorphism, but composition is often more flexible. Iterators and generators produce values incrementally, which is valuable for large data sets. Type hints document intended contracts and enable tooling, although they do not replace runtime validation. In practical data-science systems, these features work together to create code that is reusable, testable, scalable and governable.

Key Terms

Function: A named, reusable block of code that may accept inputs and return a result.

Parameter: A name in a function definition that receives an argument.

Argument: A value supplied to a function call.

Namespace: A mapping from names to objects.

Scope: The region in which a name can be accessed.

Closure: An inner function that retains access to enclosing-scope values.

Module: A Python file containing code definitions and statements.

Package: An organized collection of related Python modules.

Class: A blueprint defining the structure and behaviour of objects.

Object: A concrete instance of a class.

Method: A function associated with a class or object.

Encapsulation: Combining data and related operations behind an interface.

Invariant: A condition that must remain true for a valid object.

Inheritance: A mechanism by which a subclass derives behaviour from a parent class.

Composition: Constructing an object from other collaborating objects.

Iterator: An object that returns successive values and eventually raises StopIteration.

Generator: A function or expression that produces values lazily using iteration.

Type hint: An annotation describing an intended type or interface.

Polymorphism: The ability to use different objects through a common interface.

Side effect: A change to state outside a function’s returned value.

Review Questions

Short Answer Questions

1. Distinguish between a parameter and an argument.
2. What does the LEGB rule describe?
3. Why is a mutable default argument usually avoided?
4. State one difference between an iterator and an iterable.
5. What is the purpose of a type hint?

Long Answer Questions

1. Explain how functions, scope and namespaces contribute to maintainable Python programs. Include examples of poor and improved design.
2. Describe how modules and packages should be organized in a data-science project. Discuss import practices and circular dependencies.
3. Explain encapsulation, invariants, properties and dataclasses in object-oriented Python.
4. Compare inheritance and composition. Give a data-science example in which composition is preferable.
5. Explain how generators support memory-efficient data processing and identify their limitations.

Analytical / Application Questions

1. Design a package structure for a project that reads raw customer data, validates it, creates features and produces a report. Justify the responsibility assigned to each module.
2. A function modifies a global list, returns inconsistent types and silently ignores malformed records. Analyze the risks and propose a more reusable design.
3. Design a streaming sales-processing solution using a class, a generator and type hints. Explain how you would test validation, revenue calculation and malformed input handling.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a function parameter?

A. A value supplied when a function is called

B. A name in a function definition that receives a value

C. A module imported from another package

D. An object created only by a class

Correct Answer: B. A name in a function definition that receives a value

Question 2

What does the LEGB rule determine?

A. The order in which Python searches for a name

B. The order in which files are saved

C. The memory size of a list

D. The inheritance depth of a class

Correct Answer: A. The order in which Python searches for a name

Question 3

Why should a list usually not be used as a default function argument?

A. Lists cannot contain numbers

B. Lists are always immutable

C. The same list can be reused across calls

D. Python cannot pass lists to functions

Correct Answer: C. The same list can be reused across calls

Question 4

Which keyword makes a function produce values lazily?

A. return

B. yield

C. async

D. continue

Correct Answer: B. yield

Question 5

What is the main purpose of a module?

A. To organize reusable Python definitions in a file

B. To force every function to be global

C. To replace all classes with lists

D. To prevent code from being tested

Correct Answer: A. To organize reusable Python definitions in a file

Question 6

In object-oriented programming, what is an object?

A. A comment describing a class

B. A concrete instance of a class

C. A package directory only

D. A function parameter

Correct Answer: B. A concrete instance of a class

Question 7

Which design best represents composition?

A. A subclass is a specialized form of its parent

B. A class contains and collaborates with other objects

C. A function changes a global variable

D. A module imports itself repeatedly

Correct Answer: B. A class contains and collaborates with other objects

Question 8

What happens when an exhausted iterator is advanced with next()?

A. It restarts automatically

B. It returns None

C. It raises StopIteration

D. It converts into a list

Correct Answer: C. It raises StopIteration

Question 9

What do type hints primarily provide?

A. Automatic runtime validation of every value

B. Documentation and support for static analysis tools

C. Guaranteed faster execution

D. Automatic database connection management

Correct Answer: B. Documentation and support for static analysis tools

Question 10

If three valid sales have revenues 2×1.50, 1×12.00 and 3×1.50, what is total revenue?

A. 15.00 currency units

B. 18.00 currency units

C. 19.50 currency units

D. 21.00 currency units

Correct Answer: C. 19.50 currency units