Computing average of a Python numpy array discarding NaNs

Sometimes when working with Python numpy arrays, we might end up with some elements of the array being NaN (ie Not a Number). For instance, assume that you are given an array of numbers and you would like to find the average square root of the elements in the array. So we write a program such as:

Note that this program creates a numpy array my_array containing six numbers: [1, 4, 9, 16, 25, -36] where we have conveniently included a negative number (to illustrate our idea in this blogpost). Then, it applies the square root function np.sqrt() to each element of the array, storing the results in my_array_sqrts. When calculating square roots, the program will generate a NaN (Not a Number) for the last element (-36) since the square root of a negative number is undefined in real numbers.

The output will be:

Note that the program prints the resulting array of square roots, which will show the square roots of the positive numbers (1, 2, 3, 4, 5) followed by a NaN. When it tries to calculate the mean of all values in my_array_sqrts using np.mean(), however, the resulting mean will also be NaN because NumPy's default mean function doesn’t know what to do with NaN values.

Remember that -36 is indeed a number but square root of -36 is the computation that leads to an NaN.

So not only do we get an NaN, that NaN leads to further problems down the line.

What we really need is a way to conveniently discard these values when we are computing this average.

The np.nanmean() function is specifically designed to handle arrays containing NaN (Not a Number) values by excluding them from the calculation.

Consider the updated code:

The output is:

Note that we still get the complaint in the console, the NaN in the array, but the average computation is no longer affected! The program correctly outputs 3, which is the average of all numbers sans the NaN.

Here is a second example:

This example creates an array with explicit NaN values using the “np.nan” notation, and then computes the average by ignoring the NaN values.

The output will be:

In this example we calculate row-wise averages in a 2D array using the axis=1 parameter:

The output is:

This example shows column-wise average calculation using axis=0, which is particularly useful when dealing with datasets that have missing values in different columns.

The output will be:

In all of the above examples, the np.nanmean() function automatically adjusts the denominator in the mean calculation to account for only the non-NaN values, ensuring accurate averages.

You might be wondering - how would you find the mean of these arrays if you do not have access to the nanmean() method? Here are a couple of ideas for that!

The first idea uses boolean indexing with ~np.isnan() to create a mask that filters out NaN values. Here is some example code:

This is the most straightforward approach and works well for simple calculations.

The output will be:

The second method uses NumPy's masked array functionality, which is particularly useful in cases like this where we wish to mask (ie ignore) NaN values:

The output will be:

For 2D arrays, you can use list comprehension with boolean indexing to calculate means along specific axes:

The output will be:

In summary, these methods will produce the same results as np.nanmean() but offer more explicit control over how missing values are handled.

If you liked this blogpost, checkout our other numpy related blogposts, such as numpy.isnan(),numpy.unique(), and numpy.sum().

Want to learn Python with us? Sign up for 1:1 or small group classes.

Example 1: numpy.nanmean() is numpy.mean() without NaNs

Example 2: Finding average of an 1D array with explicit NaNs

Example 3: Finding row-wise averages of a 2D array

Example 4: Finding column-wise averages of a multi-dimensional array

Finding the mean using boolean indexing

Finding the mean using masked arrays

Finding the row-wise mean in 2D arrays using boolean indexing