To standardize the systolic blood pressures of 5,232 NHANES adults, subtracting the mean and dividing by the standard deviation, a Python loop over a list takes about 1 to 1.5 milliseconds on this site's computer, depending on the run. The same computation written as one expression on a NumPy array, (a - a.mean()) / a.std(ddof=1), takes about 30 times less in every run, and agrees with the loop to within $2 \times 10^{-15}$. The array also takes a quarter of the memory: 41,856 bytes for the numbers, against 172,728 for the list and the 5,232 number objects it points to.
NumPy is the reason Python can be used for numerical work at all. It provides one data structure, the n-dimensional array, and a large library of fast operations on it, and almost every package in the data science stack, pandas, matplotlib, SciPy and scikit-learn among them, stores its numbers in NumPy arrays (Harris et al., 2020). This article explains the array from the inside, because most of NumPy's behavior, including its surprises, follows from how an array is laid out in memory: its types, its speed, broadcasting, and the difference between a view and a copy. It ends with linear algebra and NumPy's random number generators, on which every simulation in this collection depends.
The running example is the 5,232 adults examined in NHANES 2017 to 2018 who have an age, a systolic and a diastolic pressure, as a 5,232 by 3 array X. All results come from tools/experiments/data-science/numpy-package.py in the site's repository, run with Python 3.13.5 and NumPy 2.2.5 on an Intel Core i9-9900K. (The script is not called numpy.py: a file of that name would be imported in place of NumPy by every script in its folder, as Data Science Packages explains.)
The array
Memory, dtype, shape and strides
Harris et al. (2020) describe the NumPy array as "a pointer to memory, along with metadata used to interpret the data stored there, notably 'data type', 'shape' and 'strides'". Each part has a precise meaning:
- The data are one block of memory holding the elements' values in binary form, with no Python objects in between.
- The dtype says how to read each element:
float64is an 8-byte IEEE double,int64an 8-byte signed integer,boolone byte. Every element of an array has the same dtype and occupies the same number of bytes. - The shape is a tuple giving the number of elements along each axis:
X.shapeis(5232, 3). Its length is the number of dimensions,X.ndim. - The strides give, for each axis, how many bytes to move in memory to reach the next element along it.
np.arange(12).reshape(4, 3), the example in Harris et al.'s first figure, has dtype int64, shape (4, 3) and strides (24, 8): the next element along a row is 8 bytes away, and the next row is $3 \times 8 = 24$ bytes away. Its transpose has strides (8, 24) and shares the same memory: transposing changed only the metadata, so it took no time and no copying. Memory order is a choice. NumPy stores row by row by default (C order), but X, extracted from a pandas table with to_numpy(), came out with strides (8, 41856), column by column (Fortran order), because pandas stores the columns of a table side by side, each column contiguous in memory. The values and every computation on them are the same in either order; what changes is which operations are fast and whether a reshape needs a copy.
The fixed layout is what makes arrays fast and compact, and it also sets their limits. An array cannot hold rows of different lengths: np.array([[1, 2], [3]]) raises ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. (Earlier versions built a one-dimensional array of list objects instead, at most with a deprecation warning; NumPy 1.24 made ragged input an error unless dtype=object is given (NumPy Developers, 2022).) And it cannot hold values of different kinds in one array, which is why tables with text and numbers need pandas.
Axes
Operations that summarize take an axis argument naming the axis that is removed. X.mean(axis=0) averages down the rows and returns one value per column, [49.83, 126.31, 72.49], the mean age, systolic and diastolic pressure; X.mean(axis=1) averages across the columns and returns one value per row, an array of shape (5232,). Harris et al. (2020) state the general rule: reducing an n-dimensional array over d axes gives an array of dimension $n - d$. Reading axis=0 as "the axis that disappears" is more reliable than "along rows" or "down columns", which mean opposite things to different readers.
Types
Fixed-width integers
A Python integer grows as needed (Python Data Types). A NumPy integer has a fixed width and wraps around when a result does not fit, following the C rules: an int8 holds $-128$ to 127, so np.array([100, 120], dtype=np.int8) + 10 is [110, -126], and a uint8 holding 200 plus 100 is 44, which is $300 - 256$. NumPy warns when this happens to a single scalar (np.int8(127) + np.int8(1) gives $-128$ with "overflow encountered in scalar add"), but not for arrays, where checking every element would cost too much. The documentation is direct about it: "The fixed size of NumPy numeric types may cause overflow errors when a value requires more memory than available in the data type" (NumPy Developers, 2025a).
A real case: the NHANES respondent numbers run from 93705 to 102956 among these adults. Their squares, up to 10,599,937,936, need more than 31 bits. Computed in int64, the sum of the 5,533 squares is 53,478,211,860,392, the same as with Python's integers. Computed in int32, every square wraps, and the sum of the wrapped squares is 5,950,103,762,856, about a ninth of the truth, with no error and no warning. Respondent numbers would not normally be squared, but sums of squares, products of counts and cumulative totals are routine, and they overflow in the same way when the dtype is too narrow.
The default integer type matters here, and it changed. In NumPy 2.0 "the default integer type on Windows is now int64 rather than int32", because "the historic 32-bit default on Windows was a common source of issues" (NumPy Developers, 2024). Code that ran correctly on Linux could overflow on Windows under NumPy 1.x, and arrays created from Python integers on Windows changed type in the upgrade.
float64 and float32
float64 is Python's float, with a machine epsilon of $2.2 \times 10^{-16}$; float32 uses 4 bytes and has an epsilon of $1.19 \times 10^{-7}$, about 7 significant digits. Halving memory is tempting for large data, and is standard in deep learning, but single precision runs out of digits quickly: np.float32(16777217) is 16777216.0, because integers above $2^{24}$ are no longer all representable, and np.float32(0.1) is 0.10000000149011612.
Where it bites is in formulas that subtract large, nearly equal numbers. The variance of the 5,533 respondent numbers is 7,211,097.70 computed in two passes, subtracting the mean before squaring. The shortcut formula $\overline{x^2} - \bar{x}^2$ gives the same in float64, but in float32 it gives 7,211,008.00: both terms are almost $10^{10}$, their float32 rounding errors are of the order of a thousand, and the difference keeps the errors. The two-pass formula in float32 gives 7,211,098.00. This is Goldberg's (1991) catastrophic cancellation, and the remedy is the one he describes: rearrange the computation so that the subtraction happens on exactly known quantities, here by centering first. NumPy's own var uses the two-pass form.
NaN and missing values
NumPy represents a missing measurement in a floating-point array as NaN, and arithmetic propagates it: np.mean of the 5,533 systolic values, of which 286 are missing, is nan, while np.nanmean ignores the NaNs and gives 126.36. As in plain Python, np.nan == np.nan is False, and the test is np.isnan. Integer arrays have no NaN: assigning np.nan into an integer array raises ValueError: cannot convert float NaN to integer. That gap is why a column of whole numbers with any missing value becomes float64 in pandas, and why pandas added nullable integer types (The pandas Package).
Vectorization
Arithmetic operators and mathematical functions apply to whole arrays element by element: a + b, a * 2, np.sqrt(a), a >= 130. NumPy calls these functions ufuncs (universal functions). A ufunc runs one compiled loop over the array's memory, knowing the dtype in advance, where a Python loop dispatches on the type of every element and creates a new object for every intermediate result (Programming with Python and Jupyter). Writing computations this way is called vectorization. Harris et al. (2020) call it "essential to array programming", since "operations that would take many tens of lines to express in languages such as C can often be implemented as a single, clear Python expression".
The speed-up depends on the operation and the machine, and absolute times on this machine varied by half from one run of the script to the next, so the script reports ratios, each timing the best of seven repetitions. Four runs gave:
| Computation | NumPy form | Loop time ÷ NumPy time |
|---|---|---|
| Standardize 5,232 systolic values | (a - a.mean()) / a.std(ddof=1) |
29 to 30 |
| Sum of squares of 1,000,000 numbers | (a * a).sum() |
12 to 14 |
| Same | np.dot(a, a) |
315 to 522 |
The last two rows compute the same number. (a * a).sum() creates a temporary array of a million squares and then sums it, so it moves 8 MB through memory twice; np.dot computes the sum of products in one pass with an optimized linear algebra routine. Vectorized code is fast because it moves the loop into compiled code, and it is fastest when the loop is also arranged to touch memory as little as possible. The timings are from one machine, and a different processor will give different ratios.
Broadcasting
Element-wise operations need operands of matching shape, but subtracting the column means from X, an operation of shape (5232, 3) minus (3,), works anyway. The mechanism is broadcasting. The documentation states the rule: comparing shapes, NumPy "starts with the trailing (i.e. rightmost) dimension and works its way left. Two dimensions are compatible when they are equal, or one of them is 1" (NumPy Developers, 2025b). A missing leading dimension counts as 1. Along every axis where one operand has size 1, it is treated as repeated to match the other, "virtually duplicated (that is, without copying any data in memory)" (Harris et al., 2020).
Standardizing every column. With mu = X.mean(axis=0) and sd = X.std(axis=0, ddof=1), both of shape (3,), the expression Z = (X - mu) / sd standardizes the three columns at once; the columns of Z have mean 0 and standard deviation 1 to twelve decimal places. The means are 49.83, 126.31 and 72.49 and the standard deviations 18.56, 20.07 and 12.01.
An outer comparison. To count the adults at or above each of three cutoffs, 120, 130 and 140 mm Hg, give the cutoffs a second axis of size 1: sbp[None, :] >= cut[:, None] compares shapes (1, 5232) and (3, 1) and returns a (3, 5232) Boolean array. Its row sums are 3,026, 1,889 and 1,104 adults, 57.8%, 36.1% and 21.1% of the 5,232. Indexing with None (or np.newaxis) inserts an axis of size 1, which is how a computation is told which axis to broadcast along.
A shape that fails. Subtracting each person's own mean from their row, X - X.mean(axis=1), raises ValueError: operands could not be broadcast together with shapes (5232,3) (5232,). The trailing sizes are 3 and 5,232. The fix is to make the row means a column, X.mean(axis=1)[:, None] or X.mean(axis=1, keepdims=True), of shape (5232, 1). The failure is the useful case: a shape mismatch that raises an error is easy to fix. The dangerous case is a square array, say 3 by 3, where row means of shape (3,) broadcast along the wrong axis without complaint. Checking .shape after every reduction is a cheap habit.
Views and copies
Indexing comes in two kinds with different results. Basic indexing, with integers and slices such as Y[:, 1] or Y[10:20], returns a view: a new array object with its own shape and strides but the same memory. The documentation explains why it can: "Views are created when elements can be addressed with offsets and strides in the original array. Hence, basic indexing always creates views" (NumPy Developers, 2025c). Changing a view changes the original. With Y a copy of X, col = Y[:, 1] has col.base is Y, and after col[0] = 0, Y[0, 1] is 0.
Advanced indexing, with a Boolean mask or an array of positions, returns a copy, because the selected elements cannot in general be described by a stride. old = Y[Y[:, 0] >= 60] selects the 1,912 rows of adults aged 60 and over into a new array (old.base is None), and setting old[:, 1] = -1 leaves Y untouched. Assignment through a mask is different again: Y[mask, 1] = -1 is a single indexing assignment on Y itself and changes those 1,912 rows of Y.
reshape returns a view when the new shape can be expressed with strides and a copy otherwise: Y.reshape(-1) of the row-ordered Y is a view, while Y.T.reshape(-1) must copy. The test for whether two arrays overlap is np.shares_memory(a, b), and a.copy() makes an independent array whenever one is needed. The same question, whether a selection is linked to its source, is answered differently by pandas, which since version 3.0 makes every selection behave as a copy (The pandas Package).
Linear algebra
The @ operator multiplies matrices (np.matmul), as distinct from *, which multiplies element by element. np.linalg holds the standard decompositions and solvers. A least-squares line of systolic pressure on age, fitted to the 5,232 adults with a column of ones for the intercept,
A = np.column_stack([np.ones(len(X)), X[:, 0]])
coef, *_ = np.linalg.lstsq(A, X[:, 1], rcond=None)
gives systolic pressure $\approx 100.17 + 0.525 \times \text{age}$, about 5 mm Hg for each decade of age. Solving the normal equations $A^\top A\, b = A^\top y$ with np.linalg.solve(A.T @ A, A.T @ y) gives the same coefficients, as the script checks. lstsq is preferred in general because it works from the singular value decomposition of $A$ instead of forming $A^\top A$, which squares the condition number and loses accuracy when predictors are nearly collinear; forming the explicit inverse np.linalg.inv(A.T @ A) is worse still. Simple Linear Regression and Multiple Linear Regression derive the normal equations and what the coefficients mean.
Random numbers
NumPy's recommended interface to random numbers is a Generator object made by np.random.default_rng(seed), whose methods draw from distributions (rng.normal, rng.integers, rng.choice). It uses the PCG64 bit generator by default (NumPy Developers, 2025d). The seed determines the whole sequence: two generators made with the same seed produce the same numbers, so a seeded analysis gives the same result on every run.
A bootstrap estimate of the standard error of the mean systolic pressure, from 2,000 resamples of the 5,232 adults (The Bootstrap Method explains the method):
rng = np.random.default_rng(742)
idx = rng.integers(0, len(sbp), size=(2000, len(sbp)))
se = sbp[idx].mean(axis=1).std(ddof=1)
gives 0.2728 with seed 742, 0.2728 again with seed 742, and 0.2812 with seed 743; the usual formula $s/\sqrt{n}$ gives 0.2774. The variation between seeds is the Monte Carlo error of using 2,000 resamples; the agreement between runs with the same seed is reproducibility. The example also uses broadcasting and advanced indexing together: idx is a 2,000 by 5,232 array of positions, and sbp[idx] is the 2,000 resamples at once.
The older interface, np.random.seed(742) followed by functions such as np.random.random(), still works and is still reproducible, but it uses one global, hidden generator shared by every part of a program. Any library call that draws from it changes the numbers every later call receives, so adding a line can change a result far away. NumPy's policy on random numbers says it plainly: "The preferred best practice for getting reproducible pseudorandom numbers is to instantiate a generator object with a seed and pass it around", because "Global state is always problematic" (Kern, 2018). For parallel or independent streams, np.random.SeedSequence(742).spawn(3) derives three child seeds whose generators do not overlap (their first draws, 0.7101, 0.1077 and 0.6156, are reproducible too).
This site's experiment scripts follow that practice: they create their generators with default_rng and a stated seed (742 in this script and in The Bootstrap Method), and experiments that are repeated report results over a fixed set of seeds, so that each published number can be regenerated exactly.
NumPy and the rest of the stack
Most readers will use NumPy less often directly than through pandas, whose columns are NumPy arrays (or extension arrays built on them) with labels attached, and through scikit-learn and matplotlib, which accept and return arrays. The ideas carry over: dtypes and NaN explain pandas' missing values, broadcasting and alignment are the two ways values are matched up, and views and copies explain when a change to one object reaches another. Harris et al. (2020) describe a further role that NumPy now plays: its API acts as a protocol that other array libraries implement, so that NumPy functions called on Dask, CuPy or sparse arrays dispatch to those libraries. Code written against NumPy's interface increasingly runs on arrays that are not NumPy arrays at all.
References
- Goldberg, D. (1991). What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 23(1), 5–48. https://doi.org/10.1145/103162.103163 record · 2 pages
- Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., Haldane, A., Fernández del Río, J., Wiebe, M., Peterson, P., ... and Oliphant, T. E. (2020). Array programming with NumPy. Nature, 585, 357–362. https://doi.org/10.1038/s41586-020-2649-2 record · 3 pages
- Kern, R. (2018). NEP 19: Random number generator policy. NumPy Enhancement Proposals. https://numpy.org/neps/nep-0019-rng-policy.html record
- NumPy Developers (2022). NumPy 1.24.0 Release Notes (section "Expired deprecations"). https://numpy.org/doc/stable/release/1.24.0-notes.html record
- NumPy Developers (2024). NumPy 2.0.0 Release Notes. https://numpy.org/doc/stable/release/2.0.0-notes.html record
- NumPy Developers (2025a). Data types (section "Overflow errors"). NumPy User Guide. https://numpy.org/doc/stable/user/basics.types.html record
- NumPy Developers (2025b). Broadcasting. NumPy User Guide. https://numpy.org/doc/stable/user/basics.broadcasting.html record
- NumPy Developers (2025c). Copies and views. NumPy User Guide. https://numpy.org/doc/stable/user/basics.copies.html record
- NumPy Developers (2025d). Random sampling. NumPy Reference. https://numpy.org/doc/stable/reference/random/index.html record