3. Functions, Modularity and Object-Oriented Programming

Introduction and Context

Data science programs rarely remain small for long. A script that begins as a few lines for loading a dataset may eventually include validation, transformation, statistical analysis, model training, evaluation, visualisation, reporting, and deployment. When all of these operations are written sequentially in one file, the code becomes difficult to understand, test, reuse, and modify. Functions, modules, and object-oriented programming provide the structural foundations needed to manage this complexity.

A function gives a name to a coherent computation. Modularity divides a program into manageable and relatively independent components. Object-oriented programming organises data and behaviour into objects that represent entities in a problem domain. These ideas are not merely stylistic preferences. They influence reliability, collaboration, testing, performance, maintainability, and the ability to convert exploratory data analysis into dependable software.

Python provides extensive support for all three approaches. R also supports functions, lexical scoping, environments, and object-oriented systems such as S3, S4, and R6. The examples in this chapter use Python because its function and class syntax is widely used in data-science workflows, while the underlying design principles apply to both Python and R.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the role of functions, modules, and objects in data-science software.

2. Define and call functions using positional arguments, keyword arguments, default values, and variable-length arguments.

3. Distinguish local, global, and enclosing scope and explain closures.

4. Apply recursion and functional programming constructs appropriately.

5. Design classes using attributes, methods, constructors, inheritance, and composition.

6. Select between procedural, functional, inheritance-based, and composition-based designs.

7. Organise code into modules and packages using readable conventions and documentation.

8. Identify common design errors, ethical risks, and maintainability problems in analytical software.

Functions as Units of Reasoning

A function is a named, reusable unit of computation. It receives zero or more inputs, performs a defined operation, and may return a result. A well-designed function generally has a clear purpose, a limited responsibility, and an interface that states what callers must provide and what they may expect in return.

```python

def mean(values):

"""Return the arithmetic mean of a non-empty sequence."""

if not values:

raise ValueError("values must not be empty")

return sum(values) / len(values)

```

The function has a name, a parameter, a docstring, validation, and a return statement. The docstring is part of the function's communication interface. It helps users understand the intended input and output without reading the implementation.

A function may be called with positional or keyword arguments:

```python

def clip(value, lower=0.0, upper=1.0):

return max(lower, min(value, upper))

clip(1.4)

clip(value=0.4, lower=0, upper=1)

```

Default arguments are useful when a sensible standard behaviour exists. However, mutable objects should generally not be used as default values because the same object is created once when the function is defined:

```python

Problematic

def add_item(item, items=[]):

items.append(item)

return items

```

A safer design creates the object inside the function:

```python

def add_item(item, items=None):

if items is None:

items = []

items.append(item)

return items

```

Python also supports variable-length arguments. `args` collects additional positional arguments, while `*kwargs` collects additional keyword arguments. These facilities are useful in wrappers and configurable utilities, but excessive use can make an interface ambiguous.

```python

def weighted_total(*values, scale=1.0):

return scale * sum(values)

```

Parameters may be made keyword-only by placing them after `*`. This is often valuable in analytical code because explicit configuration reduces accidental misuse.

Return Values, Side Effects and Contracts

A function can return one value, multiple values packaged as a tuple, or no explicit value, in which case Python returns `None`. Returning a result is often preferable to modifying external state because it makes the function easier to test and reason about. A function that changes a file, database, global variable, or object supplied by the caller has a side effect.

Pure functions depend only on their inputs and produce no externally visible side effects. For example, a function that standardises a numeric vector and returns a new vector is more predictable than one that silently changes a global dataset. Pure functions are particularly suitable for testing, parallel execution, caching, and reproducible pipelines. Side effects are sometimes necessary, such as writing a model file, but they should be visible in the function's purpose and documentation.

A useful function contract specifies assumptions, outputs, errors, and side effects. Type hints can strengthen this communication:

```python

def z_score(value: float, mean: float, standard_deviation: float) -> float:

"""Standardise value; reject a zero standard deviation."""

if standard_deviation == 0:

raise ValueError("standard_deviation must be non-zero")

return (value - mean) / standard_deviation

```

Type hints are not a substitute for validation, but they improve editor support, documentation, and static analysis.

Scope, Names and Closures

Scope determines where a name can be accessed. Python commonly involves local, enclosing, global, and built-in scopes, often remembered through the LEGB rule. A name is first sought in the local scope, then in any enclosing function scopes, then in the module's global scope, and finally among built-in names.

```python

rate = 0.05

def calculate_tax(amount):

rate = 0.20

return amount * rate

```

The `rate` inside `calculate_tax` is local and therefore does not replace the global `rate`. Local variables reduce unintended interactions between functions. Global variables can be convenient for constants, but mutable global state makes programs difficult to test and may cause hidden dependencies.

An enclosing scope exists when one function is defined inside another. A closure is an inner function that retains access to variables from its enclosing function even after the outer function has finished execution:

```python

def make_multiplier(factor):

def multiply(value):

return value * factor

return multiply

triple = make_multiplier(3)

triple(5) # 15

```

The function `multiply` closes over `factor`. Closures are useful for configurable transformations, decorators, callbacks, and lightweight state. They should be used carefully when the retained state is complex, since a class may communicate such state more clearly.

The `global` and `nonlocal` keywords allow assignments to names outside the current local scope. They are sometimes necessary, but frequent use usually indicates that state should be passed explicitly or encapsulated in an object.

Recursion and Functional Techniques

Recursion occurs when a function calls itself. Every recursive design requires a base case and a recursive case that moves toward it. A simple example is traversing a nested structure:

```python

def count_leaves(item):

if not isinstance(item, list):

return 1

return sum(count_leaves(child) for child in item)

```

Recursion can express tree traversal, hierarchical data processing, and divide-and-conquer algorithms naturally. It may, however, consume considerable memory and can encounter Python's recursion-depth limit. Iteration is usually preferable for simple repetition or very deep data.

Functional programming treats functions as values. A function can be stored in a variable, passed as an argument, or returned from another function. `map`, `filter`, and `reduce` implement common transformations, although list comprehensions are often more readable in Python:

```python

values = [2, 5, 8, 11]

squares_of_even = [x * x for x in values if x % 2 == 0]

```

Lambda expressions provide short anonymous functions:

```python

records = [("A", 4), ("B", 2), ("C", 7)]

sorted(records, key=lambda record: record[1])

```

Higher-order functions such as `sorted`, `apply`, or a custom pipeline function can separate a general process from the operation applied to each item. In data science, this supports reusable feature transformations and validation rules. Nevertheless, a named function with a docstring is usually preferable when the operation is complex, reused, or domain-specific.

Modularity: From Scripts to Systems

Modularity divides a program into components with defined responsibilities and interfaces. A Python file containing definitions is a module. A package is a directory containing related modules, commonly organised around a domain or layer of the application. For example, a data-science project might separate `io`, `cleaning`, `features`, `models`, and `evaluation` components.

```python

cleaning.py

def remove_missing_rows(frame):

return frame.dropna()

analysis.py

from cleaning import remove_missing_rows

```

Imports establish dependencies. A module should expose a small, stable public interface and keep implementation details private where possible. The convention of prefixing an internal name with an underscore communicates that it is not intended as public API.

A module should not execute substantial analysis merely because it is imported. Code intended to run only when the file is executed directly belongs in a guarded entry point:

```python

def main():

print("Running analysis")

if __name__ == "__main__":

main()

```

This distinction allows functions to be imported and tested without unintentionally launching a workflow.

Good modular design aims for high cohesion and low coupling. High cohesion means that the elements of a module belong together. Low coupling means that modules depend on one another through limited, stable interfaces rather than internal details. Circular imports, duplicated utilities, excessively large modules, and functions that reach into many unrelated objects are signs of weak modularity.

Modules are also important for reproducibility. A transformation defined once in a tested module is less likely to diverge from a similar transformation copied into several notebooks. Notebook experimentation remains valuable, but stable analytical logic should eventually move into version-controlled modules with tests and documentation.

Classes and Objects in Data-Science Design

Object-oriented programming combines state and behaviour. A class defines a type; an object is an instance of that type. Attributes represent state, while methods represent operations that use or modify that state.

```python

class DataSummary:

"""Summarise a sequence of numeric observations."""

def __init__(self, values):

if not values:

raise ValueError("values must not be empty")

self.values = list(values)

def mean(self):

return sum(self.values) / len(self.values)

def count(self):

return len(self.values)

```

The `__init__` method initialises an object. The `self` parameter refers to the particular instance on which a method operates. Each `DataSummary` object has its own `values`, while all instances share the method definitions.

Encapsulation means presenting a controlled interface while protecting implementation details. Python does not enforce strict private fields in the same way as some languages, but naming conventions, properties, and carefully designed methods can discourage inappropriate access. A property can validate or compute an attribute:

```python

class ModelResult:

def __init__(self, scores):

self._scores = list(scores)

@property

def best_score(self):

return max(self._scores)

```

Inheritance allows a specialised class to reuse or extend a more general class. For example, a `ClassificationReport` might inherit common reporting behaviour from a `ModelReport`. Method overriding enables specialised behaviour, and `super()` invokes the parent implementation. Inheritance is most appropriate when the subtype genuinely satisfies the behavioural expectations of the parent type—the “is-a” relationship.

Composition builds an object from other objects—the “has-a” relationship. A `TrainingPipeline` may contain a validator, transformer, and estimator:

```python

class TrainingPipeline:

def __init__(self, transformer, estimator):

self.transformer = transformer

self.estimator = estimator

def fit(self, data, target):

transformed = self.transformer.fit_transform(data)

self.estimator.fit(transformed, target)

return self

```

Composition is often more flexible than deep inheritance because components can be replaced independently. It also supports dependency injection: the required collaborator is supplied from outside rather than constructed internally. This makes testing easier, since a real estimator can be replaced with a small test double.

Polymorphism allows different objects to be used through a common interface. A pipeline need not know the internal implementation of every transformer; it only needs the expected operations, such as `fit_transform`. Python's duck typing supports this style, although protocols and abstract base classes can make expected interfaces more explicit.

Documentation, Style and Testing

Readable code is a professional requirement in data science because analytical results may influence financial, medical, operational, or public decisions. Names should communicate intent: `training_records` is clearer than `tr`. Functions should generally be short enough to understand as a unit, but artificial fragmentation should be avoided. Consistent indentation, import organisation, spacing, and naming conventions reduce cognitive load.

Documentation should operate at several levels. A module docstring explains the module's purpose. Public functions and classes should document parameters, return values, raised exceptions, assumptions, and side effects where relevant. A project-level README should explain installation, usage, data expectations, and reproducibility steps. Comments should explain decisions or non-obvious constraints rather than restating syntax.

Testing should be designed around behaviour. Unit tests check small functions or classes; integration tests check interactions between modules; end-to-end tests check a complete workflow. Boundary cases are especially important in data science: empty data, missing values, duplicate records, unexpected types, zero variance, extreme values, and inconsistent categories.

A function that silently drops rows, changes units, or applies a different missing-value policy can produce a technically valid but substantively misleading analysis. Tests should therefore verify not only that code runs, but also that its data assumptions and outputs are correct.

An Illustrative Case: Refactoring a Customer-Retention Workflow

Background

The following fictional case concerns a subscription company whose analyst built a customer-retention model in a notebook. The notebook loaded transaction data, filled missing values, created monthly features, trained a classifier, and generated a list of customers for a retention campaign.

Situation and Problem

The notebook initially produced useful results, but three problems emerged. First, the same cleaning steps were copied into two notebooks and began to differ. Second, analysts could not easily test individual transformations. Third, the campaign team needed to compare several models without rewriting the entire workflow. A further risk was that the target variable was accidentally included in one feature-building step, creating leakage.

Analysis

The workflow was examined as a sequence of responsibilities: input validation, cleaning, feature construction, model fitting, evaluation, and prediction. These responsibilities were tightly interwoven. Configuration values were stored in global variables, and intermediate data frames were modified in place. As a result, the order in which notebook cells were executed affected the output.

Approach and Solution

The analyst extracted pure or mostly pure functions for missing-value handling and feature construction. Each function received a data frame and returned a new object with a documented schema. A validation module checked required columns and rejected data in which the target appeared among predictors.

A `FeatureBuilder` class was introduced because feature construction required configuration such as time windows and category mappings. A separate estimator object was supplied to a `TrainingPipeline` through composition. This allowed several estimators to be evaluated through the same interface. The pipeline was divided into modules and imported by both the notebook and a scheduled batch job.

The team added unit tests for boundary cases and integration tests for the complete pipeline. Configuration was moved from global variables into explicit objects or function arguments. A model report recorded the data version, feature definitions, evaluation split, and software version.

Outcome

The refactored design reduced duplicated logic, made model comparison easier, and exposed the target-leakage error before deployment. Analysts could still use notebooks for exploration, but the production workflow depended on tested modules rather than cell execution order.

Learning Points

The case illustrates that abstraction is valuable when it reflects stable responsibilities. Functions were appropriate for focused transformations, while a class was justified where state and configuration had to be retained. Composition allowed the estimator to change without redesigning the pipeline. Most importantly, modularity supported governance: assumptions became visible, testable, and reviewable.

Risks, Limitations and Professional Considerations

Abstraction has costs. Excessive small functions can make control flow difficult to follow, while deep inheritance hierarchies can conceal behaviour. A class is not automatically better than a function; it should be introduced when an entity has meaningful state, identity, or a coherent set of operations. Similarly, a closure should not hide complex state that would be clearer in a named object.

Mutable shared state is a common source of defects. It can produce order-dependent results, race conditions in concurrent programs, and irreproducible analyses. Data should not be modified in place without clear documentation. Randomness should be controlled through explicit random-number generators where reproducibility matters.

Analytical software also carries ethical and governance obligations. Functions that remove outliers or impute missing values may change the population being analysed. A model class may hide the assumptions of a particular algorithm. Documentation should record data provenance, transformation rules, known limitations, and the meaning of evaluation metrics. Access credentials and personally identifiable information should never be embedded in source code. Logging should avoid exposing sensitive records.

Automation does not remove responsibility. A reusable pipeline can reproduce an error at scale, and an object-oriented interface can make an unsafe operation appear legitimate. Code review, testing, version control, audit trails, and appropriate human oversight remain necessary, especially when outputs affect individuals.

Current Directions in Software Design for Data Science

Contemporary data-science systems increasingly use pipeline abstractions, typed interfaces, configuration management, and automated testing. Data validation frameworks check schemas and distributions before a model is trained. Workflow orchestration tools connect modular steps and record execution metadata. Containerisation helps preserve runtime dependencies, while continuous integration can run tests whenever code changes.

Machine-learning operations also encourage separation between data preparation, training, evaluation, deployment, and monitoring. This separation makes it possible to identify whether a performance change comes from new data, code, configuration, or model parameters. In both Python and R, package development tools increasingly support documentation, testing, dependency management, and reproducible builds.

These developments reinforce rather than replace the central principles of this chapter: explicit interfaces, limited responsibilities, testable components, and transparent assumptions.

Best Practices for Reusable Analytical Code

• Give each function or class a clear responsibility.
• Prefer explicit inputs and outputs over hidden global state.
• Use immutable or freshly created results when practical.
• Validate assumptions at system boundaries.
• Choose composition when components may vary independently.
• Use inheritance only for genuine subtype relationships.
• Keep public interfaces small and document them.
• Separate exploratory notebooks from reusable production modules.
• Test normal, boundary, and invalid cases.
• Record data, configuration, dependency, and model information needed for reproduction.
• Review transformations for leakage, bias, privacy risks, and unintended data loss.

Chapter Summary

Functions provide named units of computation and are the primary mechanism for reuse in procedural and functional designs. Parameters, return values, contracts, type hints, and controlled side effects determine how safely functions can be used. Scope governs name visibility, while closures retain enclosing state and support configurable behaviour. Recursion is appropriate for naturally hierarchical problems but must have a sound base case and may be less suitable than iteration for deep structures.

Modules and packages organise functions, classes, and related resources into cohesive components. Effective modularity uses stable interfaces, high cohesion, low coupling, and protected entry points. Object-oriented programming combines state and behaviour in objects. Encapsulation, polymorphism, inheritance, and composition offer different ways to structure complex systems; composition is frequently useful in configurable data-science pipelines.

Readable naming, documentation, testing, version control, and explicit data assumptions are essential professional practices. Reusable abstractions can improve reliability and governance, but poor abstractions, mutable global state, hidden transformations, and undocumented assumptions create technical and ethical risks. The appropriate design is the simplest one that makes responsibilities, dependencies, and consequences clear.

Key Terms

Function: A named, reusable unit of computation that may accept inputs and return a result.

Parameter: A named variable in a function definition that receives an argument.

Argument: A value supplied when a function is called.

Default argument: A parameter value used when the caller does not provide one.

Side effect: An observable change outside a function's returned result.

Pure function: A function whose result depends only on its inputs and that has no externally visible side effects.

Scope: The region of a program in which a name can be accessed.

Closure: An inner function that retains access to variables in an enclosing scope.

Recursion: A technique in which a function invokes itself on a smaller or simpler problem.

Higher-order function: A function that accepts functions as arguments or returns a function.

Module: A file containing related code definitions and statements.

Package: A structured collection of related modules.

Cohesion: The degree to which the elements of a component belong together.

Coupling: The degree of dependence between software components.

Class: A definition describing the attributes and behaviour of a type of object.

Object: An instance of a class containing state and behaviour.

Encapsulation: The organisation of data and operations behind a controlled interface.

Inheritance: A mechanism by which a specialised class derives behaviour from a general class.

Composition: Building an object by combining it with other objects.

Polymorphism: The ability to use different object types through a common interface.

Review Questions

Short Answer Questions

1. What distinguishes a pure function from a function with side effects?

2. Explain the LEGB rule in Python.

3. Why can mutable default arguments cause unexpected behaviour?

4. What is the difference between a module and a package?

5. Define composition in object-oriented programming.

Long Answer Questions

1. Explain how parameters, return values, validation, documentation, and type hints contribute to a reliable function interface.

2. Discuss closures, including their mechanism, uses, and limitations in data-science software.

3. Explain high cohesion and low coupling and show how they guide the organisation of a data-science project.

4. Compare inheritance and composition, including situations in which each is appropriate.

5. Describe how testing and documentation support reproducibility and governance in analytical software.

Analytical / Application Questions

1. A feature-engineering function changes a shared data frame and relies on a global date range. Analyse the risks and propose a more testable design.

2. Design the components of a modular training pipeline that can replace its estimator without changing data validation or feature construction. Justify whether you would use inheritance, composition, or both.

3. A model-development notebook gives different results depending on the order in which cells are run. Identify likely causes and propose a refactoring plan using functions, modules, explicit configuration, and tests.

22. Multiple Choice Questions with Answers

Question 1

Which design best demonstrates a pure function?

A. A function that reads a global data frame and modifies it in place

B. A function that accepts values and returns a result without changing external state

C. A function that writes a log file and returns no value

D. A function that changes a module-level configuration variable

Correct Answer: B. A function that accepts values and returns a result without changing external state

Question 2

Why is using an empty list as a Python function's default argument potentially problematic?

A. Lists cannot be passed to functions

B. The list is recreated on every call and wastes memory

C. The same mutable list may be reused across calls

D. Python converts the list into a tuple

Correct Answer: C. The same mutable list may be reused across calls

Question 3

In the LEGB lookup rule, which scope is searched immediately before the global scope?

A. Local scope

B. Built-in scope

C. Enclosing function scope

D. Class inheritance scope

Correct Answer: C. Enclosing function scope

Question 4

What is the defining feature of a closure?

A. It always calls itself recursively

B. It retains access to variables from an enclosing scope

C. It can only be defined inside a class

D. It prevents all side effects

Correct Answer: B. It retains access to variables from an enclosing scope

Question 5

Which situation is most naturally suited to recursion?

A. Adding a fixed number of numeric values

B. Traversing a tree-like nested structure

C. Renaming one variable

D. Importing a module

Correct Answer: B. Traversing a tree-like nested structure

Question 6

What is the main purpose of an `if __name__ == "__main__":` guard?

A. To prevent all functions in a module from being imported

B. To run selected code only when the file is executed directly

C. To declare a private class

D. To make global variables immutable

Correct Answer: B. To run selected code only when the file is executed directly

Question 7

Which design most clearly illustrates composition?

A. A specialised class inherits from a general reporting class

B. A pipeline contains separate transformer and estimator objects

C. A function calls itself until a base case is reached

D. A module imports a built-in function

Correct Answer: B. A pipeline contains separate transformer and estimator objects

Question 8

A class should generally be preferred to a standalone function when:

A. The operation has no state and is used once

B. The behaviour requires meaningful state and several related operations

C. The code contains a mathematical expression

D. The function has a short name

Correct Answer: B. The behaviour requires meaningful state and several related operations

Question 9

Which practice best supports modularity?

A. Duplicating transformations in every notebook

B. Using global variables for all configuration

C. Creating cohesive components with small, stable interfaces

D. Allowing every module to access internal details of every other module

Correct Answer: C. Creating cohesive components with small, stable interfaces

Question 10

Why should a data-science pipeline test for target leakage?

A. Leakage makes imports slower

B. Leakage can make evaluation appear better than performance on genuinely unseen data

C. Leakage prevents classes from being instantiated

D. Leakage occurs only in recursive functions

Correct Answer: B. Leakage can make evaluation appear better than performance on genuinely unseen data