Pandas Diff will difference your data. This means calculating the change in your row(s)/column(s) over a set number of periods. Or simply, pandas diff will subtract 1 cell value from another cell value within the same index.
Diff is very helpful when calculating rates of change. For example: you have temperature readings per day, calculating the difference will tell you how the temperatures have changed Day-Over-Day.
You can also think of this as taking the derivative (rate of change) of the data. This is also helpful when working with time series data and calculating Week-Over-Week.
There are 1 core concept you’ll need to grasp:
- Period = How many observations do you want to difference your data by? Most of the time this will be 1 period diff, but you can select as many as you want.
1. pd.DataFrame.diff(periods=1)
Pseudo code: For a given DataFrame or Series, find the difference (or rate of change) between rows/columns.
Pandas Diff

Your first row in your resulting diff DataFrame will generally be NaN. This is because there is no other observation to difference it with. If you had periods=2
, then there would be 2 NaNs.
Diff Parameters
- Periods (Default=1): You can select how many periods you’d like to difference by via the
periods
parameter. An easier way to think about this is, ‘how many rows would you like to difference from each cell?’ In the picture above, our periods=1 so we take the difference from each neighboring cell above. - Axis (Default=0): We usually talk about differencing rows (Axis=0), but pandas also allows you to difference columns (Axis=1).
Let’s take a look at a code sample
import pandas as pd
import numpy as np
Pandas Diff¶
Pandas Diff will return the difference between rows or columns on your DataFrame. You have the option to select how many rows/columns you'd like to difference via the 'periods' parameter.
We will run through 3 examples:
- Default differencing
- Two Period Differencing
- Column Differencing
First, let's create our DataFrame
np.random.seed(seed=42)
df = pd.DataFrame(data=np.random.normal(loc=70, scale=10, size=(7,3)),
columns=('San Francisco', 'San Diego', 'Los Angeles'),
index=['Mon', 'Tues', 'Wed', 'Thurs', 'Fri', 'Sat', 'Sun']
)
df = df.round()
df
1. Default differencing¶
By default, Pandas will difference by 1 row. Let's see how this looks for our cities.
Notice how the first row in the result is NaN. This is because the first row in the original DataFrame does not have another row to difference. Pandas returns an NaN in this case. You'll always have as many NaNs as you do periods differenced.
df.diff()
2. Two Period Differencing¶
Say instead of differencing your data by 1 period, you wanted to do it by 2 periods. To do this, set your periods=2.
df.diff(periods=2)
3. Column Differencing¶
Did you know you can also do column differencing? This would be helpful if your column represent dates or other items you'd like to compare.
To do this, set axis=1
df.diff(periods=1, axis=1)
Check out more Pandas functions on our Pandas Page