Python Functions

Writing functions for analysis code, from definition to the details that cause wrong answers. A function that averages NHANES blood pressure readings; how arguments are matched to parameters, with defaults, keyword-only and positional-only parameters, *args and **kwargs; the mutable default evaluated once; pass by assignment, and why a function can change a caller's DataFrame; scope by the LEGB rule, UnboundLocalError, global and nonlocal; closures, and the late binding that makes three cutoffs act as one; functions as arguments, lambdas, docstrings and type hints that Python does not check.

NHANES measured each examined person's blood pressure up to three times. Person 93808 has systolic readings of 120, 122 and 122 mm Hg and diastolic readings of 0, 48 and 48. Person 93705 has only a third reading, 202 over 62; the first two are missing. An analysis needs one number per person, the mean of the readings that exist, and for the diastolic pressure it needs to decide what a recorded 0 means: the survey documentation notes that "Diastolic BP can be zero" (National Center for Health Statistics, 2020), and Introduction to Data Wrangling explains why and treats such zeros as missing when averaging. The rule is the same for every person, so it belongs in a function:

def mean_reading(readings, drop_zero=False):
    """Return the mean of the readings that are present (not NaN), or NaN if there are none."""
    vals = [r for r in readings if not math.isnan(r) and not (drop_zero and r == 0)]
    return sum(vals) / len(vals) if vals else math.nan

For person 93808 it gives a systolic mean of 121.33, and a diastolic mean of 32.0 with the zero counted, 48.0 with drop_zero=True. For person 93705 it gives 202.0 and 62.0, from the one reading each, and for a person with no readings it returns NaN instead of failing with a division by zero.

A function names a computation, so that it can be used many times, tested once and changed in one place. This article covers how Python functions are defined and called and the rules that decide what a call does, concentrating on the ones that lead to wrong results in analysis code: how arguments are matched, when default values are created, what a function can change in its caller's data, which variable a name refers to, and how functions are passed to other functions, as pandas and scikit-learn constantly ask. It builds on the model of names bound to objects in Python Data Types. All results are printed by tools/experiments/data-science/python-functions.py in the site's repository, run with Python 3.13.5.

Definition and return

A definition starts with def, the function's name and its parameters in parentheses; the indented body follows. def is an ordinary statement: when it executes, it creates a function object and binds the name to it, which is why a function must be defined before the line that calls it runs (though not necessarily before that line in the file: a call inside another function's body is fine as long as the definition has executed by the time the call happens).

return ends the call and gives its value. A function that finishes without return returns None, which is a frequent bug: a function whose last line is x * 2 instead of return x * 2 returns None, and the None travels on until something fails far from the cause. A function returns one object, but the object can be a tuple, and unpacking assigns its parts to several names at once:

def summary(values):
    return min(values), max(values), sum(values) / len(values)

lo, hi, avg = summary([118, 126, 131])     # 118, 131, 125.0

The string on the first line of the body is the docstring, which help(mean_reading) and editors display. By convention (Goodger and van Rossum, 2001) it starts with a one-line summary phrased as a command ("Return the mean ...") rather than as a description ("Returns the mean ..."), and goes on to describe the arguments and the return value. In analysis code it is also where assumptions belong, such as what the function does with missing values.

Matching arguments to parameters

The values given in a call are arguments; the names in the definition are parameters. Python matches them in a fixed order: positional arguments fill parameters from the left, then keyword arguments fill parameters by name, then defaults, "calculated, once, when the function is defined", fill whatever remains, and any parameter still unfilled is an error (Python Software Foundation, 2025a). With

def linear(b0, b1, x):
    return b0 + b1 * x

linear(3, 5, 2) and linear(x=2, b0=3, b1=5) both return 13. The rules forbid ambiguity rather than guess:

Call Result
linear(1, 2, b1=3) TypeError: linear() got multiple values for argument 'b1'
linear(1, 2) TypeError: linear() missing 1 required positional argument: 'x'
linear(b0=1, 2, 3) SyntaxError: positional argument follows keyword argument

The last is rejected when the code is compiled, before anything runs. Keyword arguments are the safer habit for any function with more than two or three parameters, which includes most library functions: pd.cut(ages, bins=[17, 39, 59, 80], labels=["18-39", "40-59", "60+"]) reads correctly and survives a change in the order of the parameters, while the positional version does neither.

Defaults, keyword-only and positional-only parameters

A parameter with a default, written factor=1.0, can be omitted from a call. Two markers in the parameter list restrict how the others may be passed. Parameters after a bare * are keyword-only (Talin, 2006), and parameters before a / are positional-only (Hastings, Galindo Salgado, Corchero and Vander Weele, 2018):

def scale(values, /, factor=1.0, *, center=False):
    m = sum(values) / len(values) if center else 0.0
    return [(v - m) * factor for v in values]

scale([1, 2, 3], 2) returns [2.0, 4.0, 6.0] and scale([1, 2, 3], center=True) returns [-1.0, 0.0, 1.0]. scale([1, 2, 3], 2, True) fails, because center can only be named (TypeError: scale() takes from 1 to 2 positional arguments but 3 were given), and scale(values=[1, 2, 3]) fails because values cannot be. Keyword-only parameters are how libraries keep Boolean switches readable: a call that ends in True, False says nothing, one that ends in center=True, inplace=False says what it does. Many pandas and scikit-learn parameters are keyword-only for this reason.

Variable numbers of arguments

A parameter written *args collects any extra positional arguments into a tuple, and **kwargs collects extra keyword arguments into a dictionary:

def describe(*cols, **opts):
    return cols, opts

describe("sbp", "dbp", digits=1)        # (('sbp', 'dbp'), {'digits': 1})

The same symbols in a call do the reverse, spreading a sequence into positional arguments and a dictionary into keyword arguments: describe(*args, **kwargs) with args = ("sbp", "dbp") and kwargs = {"digits": 1} gives the same result. Their main use in practice is passing options through: a plotting helper that accepts **kwargs and hands them to matplotlib lets its caller set any option matplotlib knows, without the helper listing them. That is how pyplot's own functions forward their arguments to the object-oriented interface (The matplotlib Package). The cost is that a misspelt option is passed along silently until the receiving function rejects it, or ignores it.

The mutable default

A default value is evaluated once, when the def statement runs, not on each call. The Python tutorial puts it as an "important warning: The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes" (Python Software Foundation, 2025b). A function meant to start a new list of flagged respondents on each call,

def add_flag(seqn, flagged=[]):
    flagged.append(seqn)
    return flagged

returns [93705] on the first call and [93705, 93706] on the second, since both calls append to the one list created with the function. The list is visible in add_flag.__defaults__, which after the two calls is ([93705, 93706],). The standard form uses None as a sentinel and creates the list inside the body, where it is created on every call:

def add_flag(seqn, flagged=None):
    if flagged is None:
        flagged = []
    flagged.append(seqn)
    return flagged

The rule extends beyond lists: any default that is mutable, or that is computed from something that changes (the current time, a global setting), is fixed at definition time.

What a function can change

When a function is called, each parameter name is bound to the argument object, exactly as an assignment would bind it. The tutorial calls this "call by value (where the value is always an object reference, not the value of the object)", and adds that "call by object reference would be a better description" (Python Software Foundation, 2025b); it is also called pass by assignment. Two consequences follow from Python Data Types. A function that mutates a mutable argument changes the caller's object, because there is only one object. A function that rebinds its parameter to a new object changes nothing for the caller.

def append_reading(readings):
    readings.append(200)        # mutates: the caller's list changes

def replace_readings(readings):
    readings = [0, 0]           # rebinds the local name only

After r = [202, 202], append_reading(r) leaves r as [202, 202, 200], and replace_readings(r) leaves it unchanged.

The same holds for DataFrames, which are mutable. A function that adds a column with df["sbp_kpa"] = df["sbp"] / 7.50062 adds it to the caller's table; a function that filters with df = df[df.age >= 60] creates a new table and rebinds its local name, so the caller's table still has all 5,533 rows while the function's result has the 2,018 aged 60 and over. Both styles are legitimate, but a function should do one or the other deliberately and say so. The safest default for analysis code is the second: take inputs, return outputs, and leave the arguments alone. Such a function can be called twice, in any order, without changing its answer, which is the property that makes notebook cells safe to re-run (Programming with Python and Jupyter).

Languages differ on this point, which is a source of confusion. In C, arguments are copies; in C++ and Fortran a reference parameter can rebind the caller's variable. Python does neither: the function receives the object itself, but rebinding its parameter never affects the caller's name.

Scope

A name used in a function is looked up in four scopes in turn, often abbreviated LEGB: the local scope of the function, the scopes of any enclosing functions, the module's global scope, and the built-in names such as len and sum. Which scope a name belongs to is decided when the function is compiled, not when the line runs. The language reference states the rule: "If a name is bound in a block, it is a local variable of that block, unless declared as nonlocal or global" (Python Software Foundation, 2025c), and the block is the whole function body.

That is why this function fails:

count = 0

def increment():
    count += 1

count += 1 assigns to count, so count is local to increment throughout its body, including at the moment the right-hand side reads it, before any local value exists. The call raises UnboundLocalError: cannot access local variable 'count' where it is not associated with a value. A local variable's scope is not "from its first assignment onward", a description sometimes given; it is the entire function, and reading it before assignment is an error rather than a fallback to the global.

The declaration global count inside the function makes the name refer to the module's variable, and after one call count is 1. In an enclosing function, nonlocal n does the same for the enclosing function's variable, which allows a counter that remembers its state between calls (the script's counter returns 1, 2, 3). Both are occasionally useful, but a function that changes global state has effects that cannot be seen from its arguments and return value, and the explicit alternative, passing values in and returning them, is easier to test.

Closures

A function defined inside another can use the enclosing function's variables, and keeps access to them after the enclosing function has returned. Such a function is a closure. A standardizer for systolic pressure, built from the mean (126.36) and standard deviation (20.13) of the 5,247 NHANES adults with a reading:

def make_standardizer(mean, sd):
    def z(x):
        return (x - mean) / sd
    return z

z_sbp = make_standardizer(126.36, 20.13)
z_sbp(140)       # 0.678
z_sbp(120)       # -0.316

The returned function carries the two values with it (they are visible in z_sbp.__closure__). This is the pattern behind fitted transformations in scikit-learn, whose StandardScaler stores the training mean and standard deviation and applies them to new data; a closure is the same idea without a class.

Closures capture variables, not values. A closure looks a variable up when it is called, so closures created in a loop all see the loop variable's final value:

cutoffs = [120, 130, 140]
tests = [lambda x: x >= c for c in cutoffs]
[t(125) for t in tests]          # [False, False, False]

All three functions compare against 140, the last value of c. Binding the value at definition time, with a default argument, gives the intended tests: lambda x, c=c: x >= c returns [True, False, False] for 125. This late binding is the deliberate counterpart of the early-evaluated defaults above, and the two together explain most surprises with functions created in loops.

Functions as values

A function is an object like any other. It can be bound to another name, stored in a list, passed as an argument and returned, as make_standardizer returns z. Much of the data science libraries' interface is built on passing functions:

def summarize(values, how):
    return round(how(values), 2)

summarize(vals, min)                               # 72.67
summarize(vals, max)                               # 238.0
summarize(vals, lambda v: sum(v) / len(v))         # 126.36

The argument is the function itself, min, not a call to it, min(...). A lambda is an expression that creates a small unnamed function, limited to a single expression; it suits a key or a short transformation and little else. Python's sorted takes a key function that computes the value to sort by. With the mean systolic pressure of each smoking group in a dictionary, {'current': 125.84, 'former': 130.17, 'never': 125.02}, the call sorted(by_group, key=by_group.get) orders the groups by their means, ['never', 'current', 'former'], and key=lambda g: (len(g), g) sorts by name length and then alphabetically. pandas takes functions in the same way in groupby(...).agg, apply, assign and pipe, as The pandas Package and Exploring Data with Groups and Pivot Tables show; scikit-learn takes scoring and distance functions.

Type hints

Parameters and return values can carry annotations, and PEP 484, adopted in Python 3.5, standardized their use as type hints describing the intended types (van Rossum, Lehtosalo and Langa, 2014):

def double(x: int) -> int:
    """Return twice x."""
    return x * 2

Python stores the annotations, double.__annotations__ is {'x': int, 'return': int}, but does not check them. double("ab") returns "abab" without complaint. PEP 484 is explicit that apart from storing them in the __annotations__ attribute, "no type checking happens at runtime". A static checker such as mypy, or an editor, reads the annotations and reports calls that do not match before the code runs. In analysis code, hints document intent (that a function expects a DataFrame and returns a Series, say) and catch a class of mistakes early, but a function that must reject bad input still has to check it itself.

Functions in analysis code

The rules above suggest a few habits that make analysis code easier to trust. Write a function as soon as the same computation appears twice, and give it the rule that varies as a parameter, as drop_zero above, rather than copying the code with a change. Prefer functions that take their inputs as arguments and return new results to functions that modify their arguments or global variables. Use None for mutable defaults. Put the assumptions in the docstring. And move functions that a notebook relies on into a module, where they can be imported, reused and tested; Pimentel et al. (2019), in their study of 1.4 million notebooks, recommend exactly this, observing that most notebook authors do not extract code into modules, which "hinders the reuse and test of the notebooks".

References