Functions, Scope and Functional Techniques

Introduction

Functions are one of Python’s most important mechanisms for abstraction, reuse and software design. A function packages a coherent operation behind a name. Once defined, it can be called repeatedly with different data, tested independently and combined with other functions. In data science, functions are used to clean data, transform variables, calculate metrics, validate inputs, build machine-learning pipelines and automate reports.

A well-designed function has a clear purpose, a controlled interface and predictable behaviour. This chapter develops the practical foundations required to define and use functions, including parameters, return values, default and keyword arguments, scope, recursion, lambda expressions, higher-order operations, documentation and type hints.

Learning Objectives

By the end of this chapter, you should be able to:

• define and call reusable Python functions;
• distinguish parameters from arguments;
• use positional, keyword, default, variable-length and keyword-only arguments;
• return one or more results and understand the difference between returning and printing;
• explain local, enclosing, global and built-in scope;
• use global and nonlocal cautiously;
• write recursive functions and identify their limitations;
• apply lambda expressions with map(), filter(), sorted() and reduce();
• write useful docstrings and type annotations;
• design functions that are readable, testable and maintainable.

1. Functions as Abstraction

A function is a named block of code that performs a defined task. Its general structure is:

def function_name(parameters):
    """Optional documentation string."""
    statements
    return result

The def statement creates the function. The indented body is not executed when the function is defined; it executes when the function is called. A function may accept zero or more inputs and may return a value.

def celsius_to_fahrenheit(celsius):
    return (celsius * 9 / 5) + 32

answer = celsius_to_fahrenheit(25)
print(answer)       # 77.0

Here, celsius is a parameter, while 25 is an argument. The expression return ends the function and sends a value to the caller. If no return statement is executed, Python returns the special value None.

Printing versus returning

Printing displays information for a user; returning makes a result available to another part of a program.

def displayed_total(a, b):
    print(a + b)

def calculated_total(a, b):
    return a + b

x = displayed_total(2, 3)   # displays 5; x is None
y = calculated_total(2, 3)   # y is 5

Reusable data-processing functions should normally return results rather than print them. This separates computation from presentation and makes testing easier.

Parameters and arguments

Parameters are names in a function definition. Arguments are values supplied during a call.

def describe_score(score, maximum):
    return f"{score}/{maximum}"

describe_score(82, 100)

Arguments may be supplied positionally or by keyword:

describe_score(score=82, maximum=100)
describe_score(82, maximum=100)

Keyword arguments improve readability and allow callers to provide arguments in a different order, provided that required arguments are supplied correctly.

Default arguments

A default value is used when the caller omits an argument.

def greet(name, greeting="Hello"):
    return f"{greeting}, {name}!"

print(greet("Amina"))
print(greet("Amina", "Good morning"))

Defaults must follow non-default parameters. A common error is using a mutable object as a default:

# Avoid this pattern

def add_item(item, items=[]):
    items.append(item)
    return items

The list is created once, when the function is defined, so calls unexpectedly share the same list. Use None as a sentinel instead:

def add_item(item, items=None):
    if items is None:
        items = []
    items.append(item)
    return items

Positional-only and keyword-only parameters

Python can make an interface more explicit. Parameters before / are positional-only; parameters after * are keyword-only.

def divide(numerator, denominator, /, *, precision=2):
    return round(numerator / denominator, precision)

divide(10, 4, precision=3) is valid, but divide(numerator=10, denominator=4) is not. Keyword-only arguments are valuable when a parameter’s meaning is not obvious from its position.

Variable-length arguments

args collects extra positional arguments into a tuple, while *kwargs collects extra keyword arguments into a dictionary.

def average(*values):
    if not values:
        raise ValueError("At least one value is required")
    return sum(values) / len(values)


def build_profile(name, **details):
    return {"name": name, **details}

The names args and kwargs are conventions, not requirements. The asterisks perform the packing operation.

2. Return Values, Multiple Results and Validation

A function can return any Python object, including a tuple, list, dictionary or another function.

def summary(values):
    if not values:
        raise ValueError("values must not be empty")
    return min(values), max(values), sum(values) / len(values)

low, high, mean = summary([4, 7, 9])

Although three names are assigned, the function returns one tuple containing three elements. Returning structured results can be useful, but a large collection of unrelated results may indicate that a class or data structure would be clearer.

Validation protects a function’s assumptions. For example:

def percentage(part, whole):
    if whole == 0:
        raise ValueError("whole must not be zero")
    if part < 0 or whole < 0:
        raise ValueError("values must be non-negative")
    return 100 * part / whole

The function assumes numerical, non-negative inputs and a non-zero denominator. A professional function should either document such assumptions or enforce them.

3. Scope and Name Resolution

Scope describes where a name can be accessed. Python commonly explains name lookup using the LEGB sequence:

1. Local: names created inside the current function.
2. Enclosing: names in an outer function when functions are nested.
3. Global: names defined at module level.
4. Built-in: names supplied by Python, such as len and sum.

rate = 0.20          # global name

def calculate_tax(price):
    subtotal = price  # local name
    return subtotal * rate

subtotal exists only during the call to calculate_tax. rate is found in global scope. A local assignment creates a local name unless the function explicitly declares otherwise.

count = 10

def change_count():
    count = 20       # a new local name
    return count

print(change_count())  # 20
print(count)           # 10

Global and nonlocal

The global statement permits assignment to a module-level name:

total_calls = 0

def record_call():
    global total_calls
    total_calls += 1

The nonlocal statement permits assignment to a name in an enclosing function:

def make_counter():
    count = 0

    def next_value():
        nonlocal count
        count += 1
        return count

    return next_value

Although these features are valid, excessive shared state makes programs difficult to test and reason about. Prefer passing values as arguments and returning results. Closures such as make_counter are appropriate when controlled private state is genuinely useful.

Mutability and side effects

Python passes object references by assignment. A function can mutate a mutable object supplied by a caller:

def add_label(labels, label):
    labels.append(label)

labels = []
add_label(labels, "urgent")

The list changes because both variables refer to the same list. Reassignment of the parameter itself does not replace the caller’s variable:

def replace_list(values):
    values = [0]       # only the local parameter is rebound

Functions that mutate inputs should make this behaviour clear. In data science, unintended mutation can produce difficult-to-detect errors in shared datasets.

4. Recursion

Recursion occurs when a function calls itself. A correct recursive function requires:

• a base case, which stops recursion;
• a recursive case, which reduces the problem toward the base case.

def factorial(n):
    if not isinstance(n, int) or n < 0:
        raise ValueError("n must be a non-negative integer")
    if n == 0:
        return 1
    return n * factorial(n - 1)

For a non-negative integer n, the factorial is defined by:

The recursive implementation is correct because each call reduces n by one and eventually reaches zero. However, Python limits recursion depth, and each call consumes stack space. For a simple linear computation, iteration is often more efficient and safer:

def factorial_iterative(n):
    if not isinstance(n, int) or n < 0:
        raise ValueError("n must be a non-negative integer")
    result = 1
    for value in range(2, n + 1):
        result *= value
    return result

Recursion is especially natural for tree structures, nested folders and divide-and-conquer algorithms. It should not be used merely because it is shorter.

5. Functions as Objects and Higher-Order Operations

In Python, functions are first-class objects. They can be stored in variables, placed in collections, passed as arguments and returned from other functions.

def square(x):
    return x * x

operation = square
print(operation(5))

A higher-order function accepts a function, returns a function, or does both. This supports general algorithms whose specific operation is supplied by the caller.

def apply_twice(function, value):
    return function(function(value))

apply_twice(square, 2)  # 16

Lambda expressions

A lambda is a small anonymous function containing one expression.

triple = lambda x: 3 * x
print(triple(4))

For named, complex or reusable logic, def is usually clearer. Lambdas are particularly useful as short callback functions.

map, filter and sorted

map(function, iterable) applies a function to each element and returns an iterator.

values = [1, 2, 3, 4]
squares = list(map(lambda x: x * x, values))

filter(function, iterable) retains elements for which the function returns a truthy value.

positive = list(filter(lambda x: x > 0, [-2, 0, 3, 5]))

sorted can use a key function:

students = [("Lee", 72), ("Ravi", 88), ("Maya", 81)]
ordered = sorted(students, key=lambda item: item[1], reverse=True)

The key function determines the comparison value; it does not rearrange the original list. sorted returns a new list, whereas list.sort() modifies a list in place and returns None.

For straightforward transformations, list comprehensions are often more readable:

squares = [x * x for x in values]
positive = [x for x in [-2, 0, 3, 5] if x > 0]

reduce

functools.reduce repeatedly combines values into one result.

from functools import reduce

product = reduce(lambda a, b: a * b, [2, 3, 4], 1)

The initial value 1 makes the operation well-defined even when the input is empty and is appropriate for multiplication. In many situations, built-ins such as sum, any, all, min and max communicate intent more clearly than reduce.

6. Documentation and Type Hints

Documentation is part of a function’s interface. A docstring should explain purpose, parameters, return value, raised exceptions and important assumptions.

def mean(values: list[float]) -> float:
    """Return the arithmetic mean of a non-empty sequence.

    Args:
        values: Numeric observations.

    Returns:
        The sum of the observations divided by their count.

    Raises:
        ValueError: If values is empty.
    """
    if not values:
        raise ValueError("values must not be empty")
    return sum(values) / len(values)

A type hint communicates intended types but normally does not enforce them at runtime. Static analysers, integrated development environments and documentation tools can use annotations to detect mistakes and improve usability.

Modern Python supports built-in generic forms such as list[int] and dict[str, float]. For functions accepting several possible types, use Union or the pipe notation where supported:

def format_id(identifier: int | str) -> str:
    return str(identifier).strip()

Type hints should be accurate. An annotation that promises float but sometimes returns None misleads users and tools. Use float | None if both outcomes are intentional.

7. Applied Case: A Reusable Data-Quality Pipeline

Background

A fictional retail analytics team receives daily transaction records. Each record contains a product identifier, quantity and unit price. Analysts need to remove invalid rows, calculate line totals and rank products by revenue.

Problem

A single long script has duplicated validation logic, mixes printing with computation and silently accepts negative quantities. The team wants reusable, testable functions.

Approach

from typing import Any

Record = dict[str, Any]

def is_valid_record(record: Record) -> bool:
    """Return True when a transaction has valid required values."""
    return (
        isinstance(record.get("product"), str)
        and bool(record["product"].strip())
        and isinstance(record.get("quantity"), int)
        and record["quantity"] > 0
        and isinstance(record.get("unit_price"), (int, float))
        and record["unit_price"] >= 0
    )


def line_total(record: Record) -> float:
    """Calculate quantity multiplied by unit price."""
    if not is_valid_record(record):
        raise ValueError("invalid transaction record")
    return record["quantity"] * record["unit_price"]


def revenue_by_product(records: list[Record]) -> dict[str, float]:
    """Return total revenue for each valid product record."""
    totals: dict[str, float] = {}
    for record in records:
        if is_valid_record(record):
            product = record["product"].strip()
            totals[product] = totals.get(product, 0.0) + line_total(record)
    return totals

Example data:

records = [
    {"product": "Pen", "quantity": 3, "unit_price": 2.00},
    {"product": "Pen", "quantity": 2, "unit_price": 2.00},
    {"product": "Book", "quantity": 1, "unit_price": 12.50},
    {"product": "Book", "quantity": -1, "unit_price": 12.50},
]

print(revenue_by_product(records))
# {'Pen': 10.0, 'Book': 12.5}

Analysis and outcome

The invalid book transaction is excluded. The functions have separate responsibilities: validation, calculation and aggregation. Each can be tested independently, reused in a dashboard or called by a batch-processing program. Type hints document the expected structure, while explicit validation prevents questionable values from silently entering the analysis.

Learning points

1. Small functions are easier to test and replace.
2. Returning dictionaries is more reusable than printing summaries.
3. Validation should reflect the business rules of the data.
4. Type hints assist communication but do not replace runtime validation.
5. A function’s name and docstring should describe observable behaviour.

8. Professional Design Principles and Limitations

A high-quality function generally follows these principles:

• Single responsibility: perform one coherent task.
• Clear interface: use meaningful parameter names and sensible defaults.
• Predictable result: document return types and exceptions.
• Minimal side effects: avoid hidden changes to global state or caller-owned objects.
• Defensive validation: reject invalid data at an appropriate boundary.
• Testability: make outputs depend primarily on explicit inputs.
• Readability: prefer clear code over compressed cleverness.
• Appropriate abstraction: avoid both duplication and unnecessary generalisation.

Functional techniques can also create problems. Chains of map, filter and lambdas may be less readable than a loop or comprehension. Recursion can exhaust the call stack. Mutable arguments can create surprising shared state. Global variables make dependencies invisible. Type hints can become excessively complicated and distract from a simple interface. The correct technique depends on clarity, data size, performance requirements and the skills of the development team.

In data science, functions should also support reproducibility. Avoid hidden dependence on current time, random state, environment variables or mutable global datasets unless these dependencies are explicit. When randomness is required, pass a random-number generator or seed through the interface where practical.

Chapter Summary

Functions provide abstraction, reuse and modularity in Python. Parameters define a function’s interface, while arguments supply values at call time. Positional, keyword, default, variable-length and keyword-only arguments support flexible but controlled calls. A function should normally return results rather than print them, and it should validate important assumptions.

Python resolves names through local, enclosing, global and built-in scopes. global and nonlocal are useful in limited cases but should not replace explicit data flow. Recursion requires a base case and a reducing recursive case, although iteration is often more efficient for linear tasks. Because functions are first-class objects, higher-order operations such as map, filter, sorted and reduce can express reusable transformations. Lambda expressions are suitable for short callbacks, while named functions are preferable for substantial logic. Docstrings and type hints make interfaces understandable and support professional maintenance, testing and data-quality work.

Key Terms

Function: A named, reusable block of code that performs an operation.

Parameter: A name in a function definition that receives an argument.

Argument: A value supplied to a function call.

Return value: Data sent from a function to its caller.

Default argument: A value used when the caller omits a parameter.

Keyword argument: An argument supplied using a parameter name.

Scope: The region in which a name can be accessed.

LEGB: Local, Enclosing, Global and Built-in name lookup order.

Local variable: A name created within a function call.

Global variable: A name defined at module level.

Closure: A nested function that retains access to enclosing-scope values.

Recursion: A technique in which a function calls itself.

Base case: The condition that terminates recursion.

Lambda expression: A compact anonymous function expression.

Higher-order function: A function that accepts or returns another function.

Iterator: An object that supplies values one at a time.

Docstring: A string documenting a module, class or function.

Type hint: An annotation describing intended types.

Side effect: A change made outside a function’s returned result.

Mutable object: An object whose contents can be changed after creation.

Review Questions

Short Answer Questions

1. Distinguish a parameter from an argument.
2. Why is returning generally more reusable than printing inside a function?
3. State the LEGB order used in Python name lookup.
4. What two components are required for safe recursion?
5. What information does a type hint provide, and what does it normally not enforce?

Long Answer Questions

1. Explain positional, keyword, default, positional-only and keyword-only parameters, using suitable examples.
2. Describe local, enclosing, global and built-in scope. Discuss why excessive global state is undesirable.
3. Compare recursion and iteration, including their strengths and limitations.
4. Explain how map, filter, sorted and reduce support higher-order programming.
5. Discuss the characteristics of a professional function interface, including validation, documentation, side effects and testing.

Analytical / Application Questions

1. Design a function that accepts a sequence of measurements, rejects invalid input and returns the minimum, maximum and mean. Specify its assumptions and type hints.
2. A program uses a list as a default parameter and produces accumulated results across calls. Explain the cause and rewrite the function safely.
3. Refactor a long transaction-processing script into at least three functions. Identify each function’s responsibility, inputs, outputs and possible validation rules.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a parameter from an argument?

A. A parameter is supplied at runtime, while an argument is written in the definition.

B. A parameter is a name in a definition, while an argument is a value supplied in a call.

C. Parameters can only be numbers, while arguments can be any type.

D. Parameters are always returned, while arguments are always printed.

Correct Answer: B. A parameter is a name in a definition, while an argument is a value supplied in a call.

Question 2

What does a function return when it reaches the end without executing a return statement?

A. 0

B. False

C. None

D. An empty string

Correct Answer: C. None

Question 3

Which default-argument pattern avoids sharing one mutable list across calls?

A. def f(items=[]):

B. def f(items=list()):

C. def f(items=None): followed by if items is None: items = []

D. def f(items={}):

Correct Answer: C. def f(items=None): followed by if items is None: items = []

Question 4

In LEGB name resolution, which scope is searched first?

A. Global

B. Built-in

C. Enclosing

D. Local

Correct Answer: D. Local

Question 5

What is the purpose of a base case in a recursive function?

A. It converts recursion into a loop automatically.

B. It prevents the recursive calls from continuing indefinitely.

C. It stores all function arguments globally.

D. It guarantees constant memory use.

Correct Answer: B. It prevents the recursive calls from continuing indefinitely.

Question 6

Which expression returns the values in [1, 2, 3, 4] that are greater than 2?

A. list(map(lambda x: x > 2, [1, 2, 3, 4]))

B. list(filter(lambda x: x > 2, [1, 2, 3, 4]))

C. sorted(lambda x: x > 2, [1, 2, 3, 4])

D. reduce(lambda x: x > 2, [1, 2, 3, 4])

Correct Answer: B. list(filter(lambda x: x > 2, [1, 2, 3, 4]))

Question 7

What does the key argument to sorted() specify?

A. The number of elements to return

B. The function used to obtain comparison values

C. The data type of the resulting list

D. Whether the original list is deleted

Correct Answer: B. The function used to obtain comparison values

Question 8

What is the main purpose of a function docstring?

A. To execute the function twice

B. To enforce all type annotations at runtime

C. To document the function’s purpose, interface and behaviour

D. To make local variables global

Correct Answer: C. To document the function’s purpose, interface and behaviour

Question 9

What does a type hint normally do in Python?

A. It automatically converts every argument to the annotated type.

B. It communicates intended types to readers and development tools.

C. It prevents all invalid values at runtime.

D. It replaces the need for documentation.

Correct Answer: B. It communicates intended types to readers and development tools.

Question 10

For the call average(4, 6, 8), if average is defined with def average(*values), what is values inside the function?

A. The integer 18

B. The list [4, 6, 8]

C. The tuple (4, 6, 8)

D. The dictionary {4: 6: 8}

Correct Answer: C. The tuple (4, 6, 8)