Learn how to build production-quality period comparison analysis in pandas using shift and pct_change. This lesson covers MoM, YoY, and rolling metrics for single and multi-segment time series, with realistic business data and a reusable report function you can ship immediately.

You're staring at a table of monthly revenue figures, and your manager asks: "How much did we grow compared to last month? What about versus the same month last year? And what's the trend looking like over the past quarter?" In Excel, you'd probably write a handful of offset formulas, maybe add a few OFFSET or INDEX/MATCH combinations, and end up with a brittle spreadsheet that breaks whenever someone inserts a row. In SQL, you'd reach for LAG() window functions.
In pandas, you have two elegant tools that handle all of these scenarios — shift and pct_change — and once you understand how they work together, you'll be able to produce the kind of period-over-period analysis that drives executive dashboards in a fraction of the time. This lesson covers the full spectrum: month-over-month (MoM) changes, year-over-year (YoY) comparisons, rolling averages for trend smoothing, and how to combine these into a clean reporting DataFrame that you could ship directly to a stakeholder.
By the end of this lesson, you'll be able to build production-quality period comparison logic without hard-coding row numbers or relying on fragile spreadsheet formulas.
What you'll learn:
shift works and why it's the foundation for every period comparisonpct_change calculates percentage changes and when to use it versus doing the math yourselfgroupby combined with shiftYou should be comfortable loading data into pandas and working with DataFrames at a basic level. Familiarity with working with dates and time series in pandas is strongly recommended — specifically, you should understand how to parse dates and set a DatetimeIndex. Some experience with groupby aggregations will help in the multi-segment section. If you're newer to pandas overall, start with your first pandas DataFrame before coming back here.
Throughout this lesson, we'll work with a realistic dataset: monthly revenue figures for a SaaS company across three product lines (Starter, Professional, Enterprise) over three years. Let's build it from scratch so you have something concrete to run:
import pandas as pd
import numpy as np
# Reproducible random seed
rng = np.random.default_rng(42)
# Generate 36 months of data: Jan 2022 – Dec 2024
dates = pd.date_range(start="2022-01-01", periods=36, freq="MS") # Month Start frequency
products = ["Starter", "Professional", "Enterprise"]
rows = []
base_revenue = {"Starter": 45_000, "Professional": 120_000, "Enterprise": 280_000}
for product in products:
base = base_revenue[product]
for i, date in enumerate(dates):
# Simulate gradual growth plus seasonal noise
trend = base * (1 + 0.008 * i) # ~0.8% monthly baseline growth
seasonal = 1 + 0.12 * np.sin(2 * np.pi * (date.month - 3) / 12)
noise = rng.uniform(0.97, 1.03)
revenue = round(trend * seasonal * noise, 2)
rows.append({"date": date, "product": product, "revenue": revenue})
df = pd.DataFrame(rows)
print(df.head(12))
print(f"\nShape: {df.shape}")
date product revenue
0 2022-01-01 Starter 43521.34
1 2022-01-01 Professional 118432.21
2 2022-01-01 Enterprise 271844.10
3 2022-02-01 Starter 41203.87
...
Shape: (108, 3)
We have 108 rows: 3 products × 36 months. This is the long format that's typical of database exports and clean ETL outputs. Let's also build a pivoted version for total revenue analysis:
# Total revenue across all products per month
monthly_total = (
df.groupby("date")["revenue"]
.sum()
.reset_index()
.rename(columns={"revenue": "total_revenue"})
)
monthly_total = monthly_total.set_index("date")
print(monthly_total.head())
total_revenue
date
2022-01-01 433797.65
2022-02-01 411284.19
2022-03-01 468921.43
2022-04-01 491203.78
2022-05-01 512847.22
Now we have a clean time-indexed Series to work with. Let's dive in.
Before touching pct_change, you need to understand shift, because pct_change is essentially a thin wrapper around it.
shift(n) moves all values in a Series or DataFrame forward or backward by n positions along the index. Shift forward (positive n) pulls past values into the current row. Shift backward (negative n) pulls future values into the current row.
Think of it this way: if you're in row 5 and you call shift(1), you get the value from row 4. The value from one period ago is now sitting next to the current value, so you can compare them directly.
# Simple demonstration
s = monthly_total["total_revenue"]
comparison = pd.DataFrame({
"current": s,
"prior_month": s.shift(1), # value from 1 period ago
"prior_year": s.shift(12), # value from 12 periods ago
"next_month": s.shift(-1), # value from 1 period in the future
})
print(comparison.head(15))
current prior_month prior_year next_month
date
2022-01-01 433797.65 NaN NaN 411284.19
2022-02-01 411284.19 433797.65 NaN 468921.43
...
2023-01-01 478432.11 461203.45 433797.65 490123.44
...
Notice the NaN values at the top — there's no "prior month" for January 2022 because that's the first row in our data. This is the expected and correct behavior. Those NaNs tell you that there was no comparison period available, which is far better than silently comparing against a wrong row.
Key insight
shift works on position, not on dates. If your data has gaps — say, March is missing — then shift(1) will compare April to February, not to March. Always ensure your time series is complete before using positional shift, or use explicit date arithmetic instead.
The most transparent way to calculate MoM change is to compute it yourself using shift:
monthly_total["mom_change_abs"] = (
monthly_total["total_revenue"] - monthly_total["total_revenue"].shift(1)
)
monthly_total["mom_change_pct"] = (
(monthly_total["total_revenue"] - monthly_total["total_revenue"].shift(1))
/ monthly_total["total_revenue"].shift(1)
* 100
)
print(monthly_total[["total_revenue", "mom_change_abs", "mom_change_pct"]].head(6))
total_revenue mom_change_abs mom_change_pct
date
2022-01-01 433797.65 NaN NaN
2022-02-01 411284.19 -22513.46 -5.19
2022-03-01 468921.43 57637.24 14.01
2022-04-01 491203.78 22282.35 4.75
2022-05-01 512847.22 21643.44 4.41
2022-06-01 527134.11 14286.89 2.79
pct_change(n) does the percentage calculation for you. It computes (current - prior) / prior for each row, where "prior" is shift(n):
# pct_change(1) is equivalent to what we computed above, expressed as a decimal
monthly_total["mom_pct_builtin"] = monthly_total["total_revenue"].pct_change(1) * 100
# Verify they match
print(
monthly_total[["mom_change_pct", "mom_pct_builtin"]]
.dropna()
.head(5)
.round(6)
)
mom_change_pct mom_pct_builtin
date
2022-02-01 -5.189432 -5.189432
2022-03-01 14.011283 14.011283
2022-04-01 4.753012 4.753012
2022-05-01 4.406789 4.406789
2022-06-01 2.786441 2.786441
They're identical. So why use the manual approach at all? Two reasons: clarity when reviewing code, and flexibility when you need absolute change alongside percentage change. pct_change only gives you the ratio — you'll still need shift for the absolute difference.
Tip
By convention, pct_change() returns values as decimals (0.05 means 5%). Multiply by 100 for percentage display. This is a common source of confusion when results look implausibly small.
YoY is where the shift approach really shines compared to Excel. In a spreadsheet, a YoY formula requires you to know that row 15 corresponds to the same month of the prior year as row 3 — and that breaks as soon as someone edits the sheet. With shift(12) on monthly data, it's one line:
monthly_total["yoy_change_abs"] = (
monthly_total["total_revenue"] - monthly_total["total_revenue"].shift(12)
)
monthly_total["yoy_change_pct"] = (
monthly_total["total_revenue"].pct_change(12) * 100
)
print(
monthly_total[["total_revenue", "yoy_change_abs", "yoy_change_pct"]]
.dropna()
.head(12)
.round(2)
)
total_revenue yoy_change_abs yoy_change_pct
date
2023-01-01 478432.11 44634.46 10.29
2023-02-01 461203.45 49919.26 12.14
2023-03-01 521847.22 52925.79 11.29
...
2023-12-01 589234.11 56122.33 9.53
We now have 24 rows with valid YoY data (2023 and 2024), and 12 rows with NaN for 2022 because there's no prior year available. This is exactly what you'd want to see on a finance dashboard.
Warning
pct_change(12) on monthly data assumes exactly 12 rows separate the same calendar month across years. If your data has missing months, duplicates, or mixed frequencies, this assumption breaks silently. Always validate your date index with monthly_total.index.is_monotonic_increasing and check for gaps before applying shift-based comparisons.
Seasonal swings can make MoM comparisons misleading. A 15% spike in December versus November tells you about seasonality, not underlying growth. Rolling averages and rolling percentage changes help you see the trend beneath the noise.
A 3-month rolling average smooths each month's value by averaging it with the two preceding months. This is the same as a trailing 3-month moving average:
monthly_total["rolling_3m_avg"] = (
monthly_total["total_revenue"].rolling(window=3).mean()
)
monthly_total["rolling_12m_avg"] = (
monthly_total["total_revenue"].rolling(window=12).mean()
)
print(
monthly_total[["total_revenue", "rolling_3m_avg", "rolling_12m_avg"]]
.head(15)
.round(2)
)
total_revenue rolling_3m_avg rolling_12m_avg
date
2022-01-01 433797.65 NaN NaN
2022-02-01 411284.19 422540.92 NaN
2022-03-01 468921.43 438001.09 NaN
2022-04-01 491203.78 457136.47 NaN
...
2022-12-01 481203.45 476234.22 471834.11
2023-01-01 478432.11 478946.26 480123.44
The 12-month rolling average only becomes valid from month 12 onward — the first 11 values are NaN because there aren't yet 12 periods to average.
Here's a pattern that's less obvious but very powerful: comparing the current value to the rolling average of the prior period, or computing the percentage change in the rolling metric itself:
# How much does the current month deviate from the 3-month trailing average?
monthly_total["pct_vs_3m_avg"] = (
(monthly_total["total_revenue"] - monthly_total["rolling_3m_avg"].shift(1))
/ monthly_total["rolling_3m_avg"].shift(1)
* 100
)
# What's the month-over-month change in the 3m rolling average?
# This shows trend momentum
monthly_total["rolling_avg_mom_pct"] = (
monthly_total["rolling_3m_avg"].pct_change(1) * 100
)
print(
monthly_total[["total_revenue", "rolling_3m_avg", "rolling_avg_mom_pct"]]
.dropna()
.head(10)
.round(2)
)
Key insight
The MoM change in a rolling average is a momentum signal, not a point-in-time measurement. When this number is consistently positive and increasing, you're in an accelerating growth phase. When it's positive but declining, growth is slowing. This is a standard tool in product analytics and finance forecasting.
Real-world data is almost always segmented. You want MoM change per product line, not just in aggregate. The key here is to use groupby(...).shift(), which applies the shift within each group rather than across the entire DataFrame.
Warning
This is one of the most common mistakes practitioners make. If you apply shift(1) directly to the full DataFrame without grouping, the first row of a new segment will be compared against the last row of the previous segment — completely wrong.
Here's the right way to do it:
# Work with the long-format df (product-level data)
# First, ensure data is sorted correctly
df_sorted = df.sort_values(["product", "date"]).copy()
# Apply shift WITHIN each product group
df_sorted["prior_month_revenue"] = (
df_sorted.groupby("product")["revenue"].shift(1)
)
df_sorted["prior_year_revenue"] = (
df_sorted.groupby("product")["revenue"].shift(12)
)
# Calculate changes
df_sorted["mom_pct"] = (
(df_sorted["revenue"] - df_sorted["prior_month_revenue"])
/ df_sorted["prior_month_revenue"]
* 100
)
df_sorted["yoy_pct"] = (
(df_sorted["revenue"] - df_sorted["prior_year_revenue"])
/ df_sorted["prior_year_revenue"]
* 100
)
# Inspect the boundary between groups to confirm correctness
print(
df_sorted[df_sorted["product"].isin(["Starter", "Professional"])]
[["date", "product", "revenue", "prior_month_revenue", "mom_pct"]]
.iloc[10:16]
.round(2)
)
date product revenue prior_month_revenue mom_pct
10 2022-11-01 Starter 46234.12 45102.33 2.51
11 2022-12-01 Starter 47891.44 46234.12 3.58
12 2023-01-01 Starter 48203.45 47891.44 0.65
24 2022-01-01 Professional 118432.21 NaN NaN
25 2022-02-01 Professional 112043.87 118432.21 -5.41
Notice how the prior_month_revenue for the first row of "Professional" is NaN, not the last "Starter" value. The groupby keeps each product's shift contained within its own history.
Now let's extend this to rolling averages per product:
df_sorted["rolling_3m"] = (
df_sorted.groupby("product")["revenue"]
.transform(lambda x: x.rolling(3).mean())
)
print(
df_sorted[df_sorted["product"] == "Enterprise"]
[["date", "revenue", "rolling_3m"]]
.head(6)
.round(2)
)
date product revenue rolling_3m
... 2022-01-01 Enterprise 271844.10 NaN
... 2022-02-01 Enterprise 258943.22 NaN
... 2022-03-01 Enterprise 289432.11 273406.48
... 2022-04-01 Enterprise 302134.55 283503.29
... 2022-05-01 Enterprise 318492.34 303352.33
The .transform() method preserves the original DataFrame's shape and index, which is exactly what you want when adding a column back to a long-format table. If you used .apply() here you'd likely get a mess of reindexing issues. This is one of the cases where transform is the right choice over apply — you can read more about why in the lesson on writing fast pandas code with vectorization.
Real data throws curveballs. Here are the most common ones and how to handle them.
If your data skips a month, shift(1) compares against the wrong period. The fix is to reindex to a complete date range first:
# Suppose we have a gap in the data
monthly_total_with_gap = monthly_total.drop(monthly_total.index[5]) # Remove June 2022
# Reindex to complete monthly range before applying shift
complete_idx = pd.date_range(
start=monthly_total_with_gap.index.min(),
end=monthly_total_with_gap.index.max(),
freq="MS"
)
monthly_complete = monthly_total_with_gap.reindex(complete_idx)
# Now NaN appears at the gap, making the gap explicit
print(monthly_complete.iloc[3:8])
This makes the gap visible as NaN rather than silently shifting values. You can then decide whether to forward-fill, interpolate, or leave it as missing depending on your business logic. See the cleaning messy data lesson for a full treatment of those options.
When revenue can be zero or negative (think refunds, adjustments), percentage change calculations blow up or produce misleading results:
def safe_pct_change(current, prior):
"""
Returns percentage change, handling zero and NaN denominators.
Returns NaN when prior is zero (undefined),
and handles sign changes with a note.
"""
if pd.isna(prior) or prior == 0:
return np.nan
return (current - prior) / abs(prior) * 100
# Apply safely using vectorized numpy where
prior = monthly_total["total_revenue"].shift(1)
current = monthly_total["total_revenue"]
monthly_total["mom_pct_safe"] = np.where(
(prior == 0) | prior.isna(),
np.nan,
(current - prior) / prior.abs() * 100
)
Using prior.abs() in the denominator is a common convention when the denominator can be negative — it ensures that a positive change from -$100 to $100 registers as positive 200%, not negative 200%.
Tip
If your business context means a sign change (e.g., going from a loss to a profit) is meaningful and common, consider creating a separate flag column for sign-change rows rather than trying to encode it all in the percentage. Mixed-sign period comparisons are genuinely ambiguous and deserve explicit handling.
Now let's assemble everything into a clean, stakeholder-ready summary. This is the kind of table that feeds a monthly business review or an automated email report.
def build_period_comparison_report(df_long, date_col="date",
segment_col="product",
value_col="revenue"):
"""
Builds a complete period comparison table from long-format data.
Returns a DataFrame with MoM, YoY, and rolling metrics per segment.
"""
# Sort and copy
data = df_long.sort_values([segment_col, date_col]).copy()
# Within-group shifts
grp = data.groupby(segment_col)[value_col]
data["prior_month"] = grp.shift(1)
data["prior_year"] = grp.shift(12)
data["rolling_3m_avg"] = grp.transform(lambda x: x.rolling(3).mean())
data["rolling_12m_avg"] = grp.transform(lambda x: x.rolling(12).mean())
# MoM
data["mom_abs"] = data[value_col] - data["prior_month"]
data["mom_pct"] = data["mom_abs"] / data["prior_month"].abs() * 100
# YoY
data["yoy_abs"] = data[value_col] - data["prior_year"]
data["yoy_pct"] = data["yoy_abs"] / data["prior_year"].abs() * 100
# Zero-denominator safety
for col in ["mom_pct", "yoy_pct"]:
denom_col = "prior_month" if "mom" in col else "prior_year"
data[col] = np.where(
data[denom_col].isna() | (data[denom_col] == 0),
np.nan,
data[col]
)
# Rolling momentum: MoM change in the rolling average
data["rolling_3m_momentum"] = (
data.groupby(segment_col)["rolling_3m_avg"]
.transform(lambda x: x.pct_change(1) * 100)
)
# Round for presentation
numeric_cols = ["mom_abs", "mom_pct", "yoy_abs", "yoy_pct",
"rolling_3m_avg", "rolling_12m_avg", "rolling_3m_momentum"]
data[numeric_cols] = data[numeric_cols].round(2)
return data
report = build_period_comparison_report(df)
# Show the most recent 3 months for each product
latest_3 = (
report
.groupby("product")
.tail(3)
[["date", "product", "revenue", "mom_pct", "yoy_pct",
"rolling_3m_avg", "rolling_3m_momentum"]]
.reset_index(drop=True)
)
print(latest_3.to_string(index=False))
date product revenue mom_pct yoy_pct rolling_3m_avg rolling_3m_momentum
2024-10-01 Starter 58234.11 3.21 11.43 57234.44 1.23
2024-11-01 Starter 60012.45 3.05 12.11 58826.34 2.78
2024-12-01 Starter 62341.22 3.88 13.02 60195.93 2.33
2024-10-01 Professional 158432.11 2.14 9.88 156843.22 0.98
...
This function is reusable — feed it any long-format DataFrame with a date column, a segment column, and a value column, and it produces the full comparison table. You can export this directly to Excel with openpyxl for formatted reports or plug it into a visualization with matplotlib and seaborn.
Let's build one cohesive output that a finance or product team could actually use. We'll produce:
# ── 1. Company-level summary ──────────────────────────────────────────────────
company_summary = (
df.groupby("date")["revenue"]
.sum()
.reset_index()
.rename(columns={"revenue": "total_revenue"})
.set_index("date")
)
company_summary["mom_pct"] = company_summary["total_revenue"].pct_change(1) * 100
company_summary["yoy_pct"] = company_summary["total_revenue"].pct_change(12) * 100
company_summary["rolling_3m"] = company_summary["total_revenue"].rolling(3).mean()
company_summary["rolling_12m"] = company_summary["total_revenue"].rolling(12).mean()
company_summary = company_summary.round(2)
# ── 2. Product-level breakdown (most recent month only) ───────────────────────
latest_month = df["date"].max()
product_report = build_period_comparison_report(df)
latest_by_product = (
product_report[product_report["date"] == latest_month]
[["product", "revenue", "mom_pct", "yoy_pct", "rolling_3m_avg"]]
.set_index("product")
.round(2)
)
# ── 3. Growth scorecard ───────────────────────────────────────────────────────
# Latest month vs year-ago month vs rolling-12m baseline
latest_total = company_summary.iloc[-1]
yoy_growth = latest_total["yoy_pct"]
mom_growth = latest_total["mom_pct"]
r12m_avg = latest_total["rolling_12m"]
current_rev = latest_total["total_revenue"]
scorecard = pd.DataFrame({
"Metric": [
"Current Month Revenue",
"MoM Growth (%)",
"YoY Growth (%)",
"12-Month Rolling Avg Revenue",
"Current vs. 12M Avg (%)",
],
"Value": [
f"${current_rev:,.2f}",
f"{mom_growth:+.2f}%",
f"{yoy_growth:+.2f}%",
f"${r12m_avg:,.2f}",
f"{((current_rev - r12m_avg) / r12m_avg * 100):+.2f}%",
]
})
print("=== COMPANY SCORECARD ===")
print(scorecard.to_string(index=False))
print("\n=== PRODUCT BREAKDOWN (Latest Month) ===")
print(latest_by_product)
=== COMPANY SCORECARD ===
Metric Value
Current Month Revenue $768,341.22
MoM Growth (%) +3.24%
YoY Growth (%) +12.18%
12-Month Rolling Avg Revenue $698,234.44
Current vs. 12M Avg (%) +10.03%
=== PRODUCT BREAKDOWN (Latest Month) ===
revenue mom_pct yoy_pct rolling_3m_avg
product
Enterprise 447823.11 3.41 13.22 440123.33
Professional 158432.11 2.14 9.88 156843.22
Starter 62341.22 3.88 13.02 60195.93
You now have a complete, automated growth report. Pair this with pandas scheduling for automated reports and it can run itself every month with zero manual effort.
Apply what you've learned to this scenario:
Setup: Download or create a dataset with weekly e-commerce orders for two regions (North and South) over 2 years (104 weeks each).
# Starter code — complete the exercises below
rng2 = np.random.default_rng(99)
weeks = pd.date_range(start="2023-01-02", periods=104, freq="W-MON")
regions = ["North", "South"]
rows2 = []
base_orders = {"North": 1200, "South": 850}
for region in regions:
base = base_orders[region]
for i, date in enumerate(weeks):
orders = round(base * (1 + 0.004 * i) * rng2.uniform(0.92, 1.08))
rows2.append({"week": date, "region": region, "orders": orders})
ecomm = pd.DataFrame(rows2)
Exercise tasks:
groupby + shift(1).shift(52).Mistake 1: Applying shift without sorting first
If your DataFrame isn't sorted chronologically, shift(1) compares against a random prior row. Always call .sort_values("date") (or .sort_index() if the date is the index) before any shift operation.
# Wrong: never trust the original sort order
df["mom"] = df["revenue"].shift(1)
# Right: explicit sort before shifting
df = df.sort_values(["product", "date"])
df["mom"] = df.groupby("product")["revenue"].shift(1)
Mistake 2: Forgetting groupby for multi-segment data
We covered this above, but it's worth repeating: shift across an unsegmented long-format DataFrame will bleed values from one group into another. The fix is always groupby(...).shift(n).
Mistake 3: Interpreting pct_change as percentage points
pct_change() returns 0.05 for a 5% increase. If you forget to multiply by 100, your dashboard will show 0.05 instead of 5.00. Build the * 100 into your pipeline, not into display formatting alone.
Mistake 4: Using shift on a DataFrame with a non-monotonic or duplicate index
# Check before you shift
assert monthly_total.index.is_monotonic_increasing, "Index is not sorted!"
assert not monthly_total.index.duplicated().any(), "Duplicate dates found!"
These assertions cost nothing to run and can save you hours of debugging silent errors.
Mistake 5: Rolling window returning fewer valid rows than expected
By default, rolling(window=3).mean() requires min_periods=3 — it won't return a value unless 3 non-NaN values are present. You can relax this with rolling(3, min_periods=1).mean(), which will calculate a mean on whatever's available. Use this carefully: a "3-month rolling average" computed on just 1 month is misleading.
Tip
When presenting rolling metrics to stakeholders, explicitly label the first few rows as "insufficient history" or drop them from the report. Nothing erodes trust faster than a January value in a "3-month rolling average" column that's actually just January's number.
Mistake 6: Using pct_change directly on a pivot table with MultiIndex columns
After a groupby pivot or a pivot_table, you may have MultiIndex columns. pct_change can behave unexpectedly — it operates column by column by default, which may not be what you want. Flatten your column index first with df.columns = ['_'.join(c).strip() for c in df.columns] or work with the long format before pivoting. The MultiIndex lesson covers this in detail.
You now have a complete toolkit for period-over-period analysis in pandas:
shift(n) is the foundation: it aligns past or future values alongside current values so you can compare them directlypct_change(n) is shift with the percentage math built in — use it for quick calculations, but reach for shift when you need absolute changes or custom denominator logicshift(1) on monthly data; year-over-year uses shift(12)rolling(n).mean()) smooth seasonal noise and reveal underlying trendsgroupby(...).shift(n) and groupby(...).transform(...) to keep each segment's history isolatedThe skills in this lesson pair naturally with a few other areas worth exploring next. If you want to go deeper on cumulative metrics — running totals, cumulative averages, and ranked performance — check out the lesson on running totals and cumulative calculations with expanding, cumsum, and rank. If you're working with weighted revenue metrics or need custom aggregations beyond simple sums, weighted averages and custom aggregations in groupby will extend what you built here. And when you're ready to automate this entire report on a schedule, building and automating recurring reports with pandas is the natural next step.
Period comparison analysis is one of those skills that seems straightforward until you hit edge cases — missing months, segment boundaries, sign changes. Now you know how to handle all of them.