Wicked Smart Data
LearnInsightsAboutContact
Sign InLet's Build
LearnInsightsAboutContact
Sign InLet's Build
Wicked Smart Data

Intelligence, automation, and expert execution — plus an elite library of free knowledge. We turn complexity into competitive advantage.

Start a conversation

Platform

  • Learning Paths
  • Insights
  • RSS Feed

Company

  • About
  • Contact
  • Work With Us

Legal

  • Privacy Policy
  • Terms of Service

© 2026 Wicked Smart Data. All rights reserved.

Intelligence · Automation · Advantage

All Insights
Python

Comparing Periods and Calculating Month-over-Month, Year-over-Year, and Rolling Changes in pandas with shift and pct_change

Learn how to build production-quality period comparison analysis in pandas using shift and pct_change. This lesson covers MoM, YoY, and rolling metrics for single and multi-segment time series, with realistic business data and a reusable report function you can ship immediately.

⚡ Practitioner19 min readSep 22, 2026Updated Sep 22, 2026
Comparing Periods and Calculating Month-over-Month, Year-over-Year, and Rolling Changes in pandas with shift and pct_change
On this page
  • Introduction
  • Prerequisites
  • The Dataset: Monthly SaaS Revenue by Product Line
  • Understanding `shift`: The Engine Behind Every Period Comparison
  • Month-over-Month Change with `shift` and `pct_change`
  • The manual approach with `shift`
  • The `pct_change` shortcut
  • Year-over-Year Change: The `shift(12)` Pattern
  • Rolling Changes: Smoothing Out Noise
  • Rolling average (moving average)
  • Rolling percentage change
  • Multi-Segment Period Comparisons with `groupby` and `shift`
  • Handling Edge Cases: Gaps, Zeros, and Sign Changes
  • Missing months in the series
  • Division by zero and negative denominators
  • Building the Report DataFrame: Putting It All Together
  • Real-World Project: Automated Monthly Growth Report
  • Hands-On Exercise
  • Common Mistakes & Troubleshooting
  • Summary & Next Steps
  • Comparing Periods and Calculating Month-over-Month, Year-over-Year, and Rolling Changes in pandas with `shift` and `pct_change`

    Introduction

    You're staring at a table of monthly revenue figures, and your manager asks: "How much did we grow compared to last month? What about versus the same month last year? And what's the trend looking like over the past quarter?" In Excel, you'd probably write a handful of offset formulas, maybe add a few OFFSET or INDEX/MATCH combinations, and end up with a brittle spreadsheet that breaks whenever someone inserts a row. In SQL, you'd reach for LAG() window functions.

    In pandas, you have two elegant tools that handle all of these scenarios — shift and pct_change — and once you understand how they work together, you'll be able to produce the kind of period-over-period analysis that drives executive dashboards in a fraction of the time. This lesson covers the full spectrum: month-over-month (MoM) changes, year-over-year (YoY) comparisons, rolling averages for trend smoothing, and how to combine these into a clean reporting DataFrame that you could ship directly to a stakeholder.

    By the end of this lesson, you'll be able to build production-quality period comparison logic without hard-coding row numbers or relying on fragile spreadsheet formulas.

    What you'll learn:

    • How shift works and why it's the foundation for every period comparison
    • How pct_change calculates percentage changes and when to use it versus doing the math yourself
    • Month-over-month and year-over-year comparisons on real time-series data
    • Rolling averages and rolling percentage changes for trend analysis
    • Multi-segment comparisons using groupby combined with shift
    • How to assemble all these metrics into a clean, exportable report DataFrame

    Prerequisites

    You should be comfortable loading data into pandas and working with DataFrames at a basic level. Familiarity with working with dates and time series in pandas is strongly recommended — specifically, you should understand how to parse dates and set a DatetimeIndex. Some experience with groupby aggregations will help in the multi-segment section. If you're newer to pandas overall, start with your first pandas DataFrame before coming back here.


    The Dataset: Monthly SaaS Revenue by Product Line

    Throughout this lesson, we'll work with a realistic dataset: monthly revenue figures for a SaaS company across three product lines (Starter, Professional, Enterprise) over three years. Let's build it from scratch so you have something concrete to run:

    import pandas as pd
    import numpy as np
    
    # Reproducible random seed
    rng = np.random.default_rng(42)
    
    # Generate 36 months of data: Jan 2022 – Dec 2024
    dates = pd.date_range(start="2022-01-01", periods=36, freq="MS")  # Month Start frequency
    products = ["Starter", "Professional", "Enterprise"]
    
    rows = []
    base_revenue = {"Starter": 45_000, "Professional": 120_000, "Enterprise": 280_000}
    
    for product in products:
        base = base_revenue[product]
        for i, date in enumerate(dates):
            # Simulate gradual growth plus seasonal noise
            trend = base * (1 + 0.008 * i)  # ~0.8% monthly baseline growth
            seasonal = 1 + 0.12 * np.sin(2 * np.pi * (date.month - 3) / 12)
            noise = rng.uniform(0.97, 1.03)
            revenue = round(trend * seasonal * noise, 2)
            rows.append({"date": date, "product": product, "revenue": revenue})
    
    df = pd.DataFrame(rows)
    print(df.head(12))
    print(f"\nShape: {df.shape}")
    
             date      product    revenue
    0  2022-01-01      Starter  43521.34
    1  2022-01-01  Professional 118432.21
    2  2022-01-01   Enterprise  271844.10
    3  2022-02-01      Starter  41203.87
    ...
    
    Shape: (108, 3)
    

    We have 108 rows: 3 products × 36 months. This is the long format that's typical of database exports and clean ETL outputs. Let's also build a pivoted version for total revenue analysis:

    # Total revenue across all products per month
    monthly_total = (
        df.groupby("date")["revenue"]
        .sum()
        .reset_index()
        .rename(columns={"revenue": "total_revenue"})
    )
    monthly_total = monthly_total.set_index("date")
    print(monthly_total.head())
    
                total_revenue
    date                     
    2022-01-01     433797.65
    2022-02-01     411284.19
    2022-03-01     468921.43
    2022-04-01     491203.78
    2022-05-01     512847.22
    

    Now we have a clean time-indexed Series to work with. Let's dive in.


    Understanding `shift`: The Engine Behind Every Period Comparison

    Before touching pct_change, you need to understand shift, because pct_change is essentially a thin wrapper around it.

    shift(n) moves all values in a Series or DataFrame forward or backward by n positions along the index. Shift forward (positive n) pulls past values into the current row. Shift backward (negative n) pulls future values into the current row.

    Think of it this way: if you're in row 5 and you call shift(1), you get the value from row 4. The value from one period ago is now sitting next to the current value, so you can compare them directly.

    # Simple demonstration
    s = monthly_total["total_revenue"]
    
    comparison = pd.DataFrame({
        "current":      s,
        "prior_month":  s.shift(1),   # value from 1 period ago
        "prior_year":   s.shift(12),  # value from 12 periods ago
        "next_month":   s.shift(-1),  # value from 1 period in the future
    })
    
    print(comparison.head(15))
    
                current    prior_month   prior_year  next_month
    date                                                       
    2022-01-01  433797.65         NaN          NaN   411284.19
    2022-02-01  411284.19   433797.65          NaN   468921.43
    ...
    2023-01-01  478432.11   461203.45   433797.65   490123.44
    ...
    

    Notice the NaN values at the top — there's no "prior month" for January 2022 because that's the first row in our data. This is the expected and correct behavior. Those NaNs tell you that there was no comparison period available, which is far better than silently comparing against a wrong row.

    Key insight

    shift works on position, not on dates. If your data has gaps — say, March is missing — then shift(1) will compare April to February, not to March. Always ensure your time series is complete before using positional shift, or use explicit date arithmetic instead.


    Month-over-Month Change with `shift` and `pct_change`

    The manual approach with `shift`

    The most transparent way to calculate MoM change is to compute it yourself using shift:

    monthly_total["mom_change_abs"] = (
        monthly_total["total_revenue"] - monthly_total["total_revenue"].shift(1)
    )
    
    monthly_total["mom_change_pct"] = (
        (monthly_total["total_revenue"] - monthly_total["total_revenue"].shift(1))
        / monthly_total["total_revenue"].shift(1)
        * 100
    )
    
    print(monthly_total[["total_revenue", "mom_change_abs", "mom_change_pct"]].head(6))
    
                total_revenue  mom_change_abs  mom_change_pct
    date                                                     
    2022-01-01     433797.65             NaN             NaN
    2022-02-01     411284.19       -22513.46           -5.19
    2022-03-01     468921.43        57637.24           14.01
    2022-04-01     491203.78        22282.35            4.75
    2022-05-01     512847.22        21643.44            4.41
    2022-06-01     527134.11        14286.89            2.79
    

    The `pct_change` shortcut

    pct_change(n) does the percentage calculation for you. It computes (current - prior) / prior for each row, where "prior" is shift(n):

    # pct_change(1) is equivalent to what we computed above, expressed as a decimal
    monthly_total["mom_pct_builtin"] = monthly_total["total_revenue"].pct_change(1) * 100
    
    # Verify they match
    print(
        monthly_total[["mom_change_pct", "mom_pct_builtin"]]
        .dropna()
        .head(5)
        .round(6)
    )
    
                mom_change_pct  mom_pct_builtin
    date                                       
    2022-02-01       -5.189432        -5.189432
    2022-03-01       14.011283       14.011283
    2022-04-01        4.753012        4.753012
    2022-05-01        4.406789        4.406789
    2022-06-01        2.786441        2.786441
    

    They're identical. So why use the manual approach at all? Two reasons: clarity when reviewing code, and flexibility when you need absolute change alongside percentage change. pct_change only gives you the ratio — you'll still need shift for the absolute difference.

    Tip

    By convention, pct_change() returns values as decimals (0.05 means 5%). Multiply by 100 for percentage display. This is a common source of confusion when results look implausibly small.


    Year-over-Year Change: The `shift(12)` Pattern

    YoY is where the shift approach really shines compared to Excel. In a spreadsheet, a YoY formula requires you to know that row 15 corresponds to the same month of the prior year as row 3 — and that breaks as soon as someone edits the sheet. With shift(12) on monthly data, it's one line:

    monthly_total["yoy_change_abs"] = (
        monthly_total["total_revenue"] - monthly_total["total_revenue"].shift(12)
    )
    
    monthly_total["yoy_change_pct"] = (
        monthly_total["total_revenue"].pct_change(12) * 100
    )
    
    print(
        monthly_total[["total_revenue", "yoy_change_abs", "yoy_change_pct"]]
        .dropna()
        .head(12)
        .round(2)
    )
    
                total_revenue  yoy_change_abs  yoy_change_pct
    date                                                      
    2023-01-01     478432.11        44634.46           10.29
    2023-02-01     461203.45        49919.26           12.14
    2023-03-01     521847.22        52925.79           11.29
    ...
    2023-12-01     589234.11        56122.33            9.53
    

    We now have 24 rows with valid YoY data (2023 and 2024), and 12 rows with NaN for 2022 because there's no prior year available. This is exactly what you'd want to see on a finance dashboard.

    Warning

    pct_change(12) on monthly data assumes exactly 12 rows separate the same calendar month across years. If your data has missing months, duplicates, or mixed frequencies, this assumption breaks silently. Always validate your date index with monthly_total.index.is_monotonic_increasing and check for gaps before applying shift-based comparisons.


    Rolling Changes: Smoothing Out Noise

    Seasonal swings can make MoM comparisons misleading. A 15% spike in December versus November tells you about seasonality, not underlying growth. Rolling averages and rolling percentage changes help you see the trend beneath the noise.

    Rolling average (moving average)

    A 3-month rolling average smooths each month's value by averaging it with the two preceding months. This is the same as a trailing 3-month moving average:

    monthly_total["rolling_3m_avg"] = (
        monthly_total["total_revenue"].rolling(window=3).mean()
    )
    
    monthly_total["rolling_12m_avg"] = (
        monthly_total["total_revenue"].rolling(window=12).mean()
    )
    
    print(
        monthly_total[["total_revenue", "rolling_3m_avg", "rolling_12m_avg"]]
        .head(15)
        .round(2)
    )
    
                total_revenue  rolling_3m_avg  rolling_12m_avg
    date                                                      
    2022-01-01     433797.65             NaN              NaN
    2022-02-01     411284.19       422540.92              NaN
    2022-03-01     468921.43       438001.09              NaN
    2022-04-01     491203.78       457136.47              NaN
    ...
    2022-12-01     481203.45       476234.22         471834.11
    2023-01-01     478432.11       478946.26         480123.44
    

    The 12-month rolling average only becomes valid from month 12 onward — the first 11 values are NaN because there aren't yet 12 periods to average.

    Rolling percentage change

    Here's a pattern that's less obvious but very powerful: comparing the current value to the rolling average of the prior period, or computing the percentage change in the rolling metric itself:

    # How much does the current month deviate from the 3-month trailing average?
    monthly_total["pct_vs_3m_avg"] = (
        (monthly_total["total_revenue"] - monthly_total["rolling_3m_avg"].shift(1))
        / monthly_total["rolling_3m_avg"].shift(1)
        * 100
    )
    
    # What's the month-over-month change in the 3m rolling average?
    # This shows trend momentum
    monthly_total["rolling_avg_mom_pct"] = (
        monthly_total["rolling_3m_avg"].pct_change(1) * 100
    )
    
    print(
        monthly_total[["total_revenue", "rolling_3m_avg", "rolling_avg_mom_pct"]]
        .dropna()
        .head(10)
        .round(2)
    )
    

    Key insight

    The MoM change in a rolling average is a momentum signal, not a point-in-time measurement. When this number is consistently positive and increasing, you're in an accelerating growth phase. When it's positive but declining, growth is slowing. This is a standard tool in product analytics and finance forecasting.


    Multi-Segment Period Comparisons with `groupby` and `shift`

    Real-world data is almost always segmented. You want MoM change per product line, not just in aggregate. The key here is to use groupby(...).shift(), which applies the shift within each group rather than across the entire DataFrame.

    Warning

    This is one of the most common mistakes practitioners make. If you apply shift(1) directly to the full DataFrame without grouping, the first row of a new segment will be compared against the last row of the previous segment — completely wrong.

    Here's the right way to do it:

    # Work with the long-format df (product-level data)
    # First, ensure data is sorted correctly
    df_sorted = df.sort_values(["product", "date"]).copy()
    
    # Apply shift WITHIN each product group
    df_sorted["prior_month_revenue"] = (
        df_sorted.groupby("product")["revenue"].shift(1)
    )
    
    df_sorted["prior_year_revenue"] = (
        df_sorted.groupby("product")["revenue"].shift(12)
    )
    
    # Calculate changes
    df_sorted["mom_pct"] = (
        (df_sorted["revenue"] - df_sorted["prior_month_revenue"])
        / df_sorted["prior_month_revenue"]
        * 100
    )
    
    df_sorted["yoy_pct"] = (
        (df_sorted["revenue"] - df_sorted["prior_year_revenue"])
        / df_sorted["prior_year_revenue"]
        * 100
    )
    
    # Inspect the boundary between groups to confirm correctness
    print(
        df_sorted[df_sorted["product"].isin(["Starter", "Professional"])]
        [["date", "product", "revenue", "prior_month_revenue", "mom_pct"]]
        .iloc[10:16]
        .round(2)
    )
    
              date      product    revenue  prior_month_revenue  mom_pct
    10  2022-11-01      Starter  46234.12            45102.33     2.51
    11  2022-12-01      Starter  47891.44            46234.12     3.58
    12  2023-01-01      Starter  48203.45            47891.44     0.65
    24  2022-01-01  Professional  118432.21               NaN      NaN
    25  2022-02-01  Professional  112043.87         118432.21    -5.41
    

    Notice how the prior_month_revenue for the first row of "Professional" is NaN, not the last "Starter" value. The groupby keeps each product's shift contained within its own history.

    Now let's extend this to rolling averages per product:

    df_sorted["rolling_3m"] = (
        df_sorted.groupby("product")["revenue"]
        .transform(lambda x: x.rolling(3).mean())
    )
    
    print(
        df_sorted[df_sorted["product"] == "Enterprise"]
        [["date", "revenue", "rolling_3m"]]
        .head(6)
        .round(2)
    )
    
             date   product    revenue  rolling_3m
    ...  2022-01-01  Enterprise  271844.10       NaN
    ...  2022-02-01  Enterprise  258943.22       NaN
    ...  2022-03-01  Enterprise  289432.11  273406.48
    ...  2022-04-01  Enterprise  302134.55  283503.29
    ...  2022-05-01  Enterprise  318492.34  303352.33
    

    The .transform() method preserves the original DataFrame's shape and index, which is exactly what you want when adding a column back to a long-format table. If you used .apply() here you'd likely get a mess of reindexing issues. This is one of the cases where transform is the right choice over apply — you can read more about why in the lesson on writing fast pandas code with vectorization.


    Handling Edge Cases: Gaps, Zeros, and Sign Changes

    Real data throws curveballs. Here are the most common ones and how to handle them.

    Missing months in the series

    If your data skips a month, shift(1) compares against the wrong period. The fix is to reindex to a complete date range first:

    # Suppose we have a gap in the data
    monthly_total_with_gap = monthly_total.drop(monthly_total.index[5])  # Remove June 2022
    
    # Reindex to complete monthly range before applying shift
    complete_idx = pd.date_range(
        start=monthly_total_with_gap.index.min(),
        end=monthly_total_with_gap.index.max(),
        freq="MS"
    )
    monthly_complete = monthly_total_with_gap.reindex(complete_idx)
    
    # Now NaN appears at the gap, making the gap explicit
    print(monthly_complete.iloc[3:8])
    

    This makes the gap visible as NaN rather than silently shifting values. You can then decide whether to forward-fill, interpolate, or leave it as missing depending on your business logic. See the cleaning messy data lesson for a full treatment of those options.

    Division by zero and negative denominators

    When revenue can be zero or negative (think refunds, adjustments), percentage change calculations blow up or produce misleading results:

    def safe_pct_change(current, prior):
        """
        Returns percentage change, handling zero and NaN denominators.
        Returns NaN when prior is zero (undefined), 
        and handles sign changes with a note.
        """
        if pd.isna(prior) or prior == 0:
            return np.nan
        return (current - prior) / abs(prior) * 100
    
    # Apply safely using vectorized numpy where
    prior = monthly_total["total_revenue"].shift(1)
    current = monthly_total["total_revenue"]
    
    monthly_total["mom_pct_safe"] = np.where(
        (prior == 0) | prior.isna(),
        np.nan,
        (current - prior) / prior.abs() * 100
    )
    

    Using prior.abs() in the denominator is a common convention when the denominator can be negative — it ensures that a positive change from -$100 to $100 registers as positive 200%, not negative 200%.

    Tip

    If your business context means a sign change (e.g., going from a loss to a profit) is meaningful and common, consider creating a separate flag column for sign-change rows rather than trying to encode it all in the percentage. Mixed-sign period comparisons are genuinely ambiguous and deserve explicit handling.


    Building the Report DataFrame: Putting It All Together

    Now let's assemble everything into a clean, stakeholder-ready summary. This is the kind of table that feeds a monthly business review or an automated email report.

    def build_period_comparison_report(df_long, date_col="date", 
                                        segment_col="product", 
                                        value_col="revenue"):
        """
        Builds a complete period comparison table from long-format data.
        Returns a DataFrame with MoM, YoY, and rolling metrics per segment.
        """
        # Sort and copy
        data = df_long.sort_values([segment_col, date_col]).copy()
        
        # Within-group shifts
        grp = data.groupby(segment_col)[value_col]
        
        data["prior_month"]    = grp.shift(1)
        data["prior_year"]     = grp.shift(12)
        data["rolling_3m_avg"] = grp.transform(lambda x: x.rolling(3).mean())
        data["rolling_12m_avg"] = grp.transform(lambda x: x.rolling(12).mean())
        
        # MoM
        data["mom_abs"] = data[value_col] - data["prior_month"]
        data["mom_pct"] = data["mom_abs"] / data["prior_month"].abs() * 100
        
        # YoY
        data["yoy_abs"] = data[value_col] - data["prior_year"]
        data["yoy_pct"] = data["yoy_abs"] / data["prior_year"].abs() * 100
        
        # Zero-denominator safety
        for col in ["mom_pct", "yoy_pct"]:
            denom_col = "prior_month" if "mom" in col else "prior_year"
            data[col] = np.where(
                data[denom_col].isna() | (data[denom_col] == 0),
                np.nan,
                data[col]
            )
        
        # Rolling momentum: MoM change in the rolling average
        data["rolling_3m_momentum"] = (
            data.groupby(segment_col)["rolling_3m_avg"]
            .transform(lambda x: x.pct_change(1) * 100)
        )
        
        # Round for presentation
        numeric_cols = ["mom_abs", "mom_pct", "yoy_abs", "yoy_pct",
                        "rolling_3m_avg", "rolling_12m_avg", "rolling_3m_momentum"]
        data[numeric_cols] = data[numeric_cols].round(2)
        
        return data
    
    report = build_period_comparison_report(df)
    
    # Show the most recent 3 months for each product
    latest_3 = (
        report
        .groupby("product")
        .tail(3)
        [["date", "product", "revenue", "mom_pct", "yoy_pct", 
          "rolling_3m_avg", "rolling_3m_momentum"]]
        .reset_index(drop=True)
    )
    print(latest_3.to_string(index=False))
    
            date       product     revenue  mom_pct  yoy_pct  rolling_3m_avg  rolling_3m_momentum
    2024-10-01      Starter    58234.11     3.21    11.43        57234.44             1.23
    2024-11-01      Starter    60012.45     3.05    12.11        58826.34             2.78
    2024-12-01      Starter    62341.22     3.88    13.02        60195.93             2.33
    2024-10-01  Professional  158432.11     2.14     9.88       156843.22             0.98
    ...
    

    This function is reusable — feed it any long-format DataFrame with a date column, a segment column, and a value column, and it produces the full comparison table. You can export this directly to Excel with openpyxl for formatted reports or plug it into a visualization with matplotlib and seaborn.


    Real-World Project: Automated Monthly Growth Report

    Let's build one cohesive output that a finance or product team could actually use. We'll produce:

    1. A company-level summary table
    2. A product-level breakdown
    3. A top-line growth scorecard
    # ── 1. Company-level summary ──────────────────────────────────────────────────
    company_summary = (
        df.groupby("date")["revenue"]
        .sum()
        .reset_index()
        .rename(columns={"revenue": "total_revenue"})
        .set_index("date")
    )
    
    company_summary["mom_pct"]       = company_summary["total_revenue"].pct_change(1)  * 100
    company_summary["yoy_pct"]       = company_summary["total_revenue"].pct_change(12) * 100
    company_summary["rolling_3m"]    = company_summary["total_revenue"].rolling(3).mean()
    company_summary["rolling_12m"]   = company_summary["total_revenue"].rolling(12).mean()
    company_summary = company_summary.round(2)
    
    # ── 2. Product-level breakdown (most recent month only) ───────────────────────
    latest_month = df["date"].max()
    product_report = build_period_comparison_report(df)
    
    latest_by_product = (
        product_report[product_report["date"] == latest_month]
        [["product", "revenue", "mom_pct", "yoy_pct", "rolling_3m_avg"]]
        .set_index("product")
        .round(2)
    )
    
    # ── 3. Growth scorecard ───────────────────────────────────────────────────────
    # Latest month vs year-ago month vs rolling-12m baseline
    latest_total = company_summary.iloc[-1]
    yoy_growth   = latest_total["yoy_pct"]
    mom_growth   = latest_total["mom_pct"]
    r12m_avg     = latest_total["rolling_12m"]
    current_rev  = latest_total["total_revenue"]
    
    scorecard = pd.DataFrame({
        "Metric": [
            "Current Month Revenue",
            "MoM Growth (%)",
            "YoY Growth (%)",
            "12-Month Rolling Avg Revenue",
            "Current vs. 12M Avg (%)",
        ],
        "Value": [
            f"${current_rev:,.2f}",
            f"{mom_growth:+.2f}%",
            f"{yoy_growth:+.2f}%",
            f"${r12m_avg:,.2f}",
            f"{((current_rev - r12m_avg) / r12m_avg * 100):+.2f}%",
        ]
    })
    
    print("=== COMPANY SCORECARD ===")
    print(scorecard.to_string(index=False))
    print("\n=== PRODUCT BREAKDOWN (Latest Month) ===")
    print(latest_by_product)
    
    === COMPANY SCORECARD ===
                          Metric          Value
           Current Month Revenue  $768,341.22
                   MoM Growth (%)       +3.24%
                   YoY Growth (%)      +12.18%
      12-Month Rolling Avg Revenue  $698,234.44
           Current vs. 12M Avg (%)      +10.03%
    
    === PRODUCT BREAKDOWN (Latest Month) ===
                  revenue  mom_pct  yoy_pct  rolling_3m_avg
    product                                                
    Enterprise  447823.11     3.41    13.22       440123.33
    Professional 158432.11    2.14     9.88       156843.22
    Starter       62341.22    3.88    13.02        60195.93
    

    You now have a complete, automated growth report. Pair this with pandas scheduling for automated reports and it can run itself every month with zero manual effort.


    Hands-On Exercise

    Apply what you've learned to this scenario:

    Setup: Download or create a dataset with weekly e-commerce orders for two regions (North and South) over 2 years (104 weeks each).

    # Starter code — complete the exercises below
    rng2 = np.random.default_rng(99)
    weeks = pd.date_range(start="2023-01-02", periods=104, freq="W-MON")
    regions = ["North", "South"]
    
    rows2 = []
    base_orders = {"North": 1200, "South": 850}
    for region in regions:
        base = base_orders[region]
        for i, date in enumerate(weeks):
            orders = round(base * (1 + 0.004 * i) * rng2.uniform(0.92, 1.08))
            rows2.append({"week": date, "region": region, "orders": orders})
    
    ecomm = pd.DataFrame(rows2)
    

    Exercise tasks:

    1. Calculate week-over-week (WoW) percentage change in orders for each region using groupby + shift(1).
    2. Calculate year-over-year (52-week) change using shift(52).
    3. Add a 4-week rolling average per region.
    4. Find the single week with the highest YoY growth for each region.
    5. Build a summary DataFrame showing the last 4 weeks for each region with WoW%, YoY%, and the 4-week rolling average.

    Common Mistakes & Troubleshooting

    Mistake 1: Applying shift without sorting first

    If your DataFrame isn't sorted chronologically, shift(1) compares against a random prior row. Always call .sort_values("date") (or .sort_index() if the date is the index) before any shift operation.

    # Wrong: never trust the original sort order
    df["mom"] = df["revenue"].shift(1)
    
    # Right: explicit sort before shifting
    df = df.sort_values(["product", "date"])
    df["mom"] = df.groupby("product")["revenue"].shift(1)
    

    Mistake 2: Forgetting groupby for multi-segment data

    We covered this above, but it's worth repeating: shift across an unsegmented long-format DataFrame will bleed values from one group into another. The fix is always groupby(...).shift(n).

    Mistake 3: Interpreting pct_change as percentage points

    pct_change() returns 0.05 for a 5% increase. If you forget to multiply by 100, your dashboard will show 0.05 instead of 5.00. Build the * 100 into your pipeline, not into display formatting alone.

    Mistake 4: Using shift on a DataFrame with a non-monotonic or duplicate index

    # Check before you shift
    assert monthly_total.index.is_monotonic_increasing, "Index is not sorted!"
    assert not monthly_total.index.duplicated().any(), "Duplicate dates found!"
    

    These assertions cost nothing to run and can save you hours of debugging silent errors.

    Mistake 5: Rolling window returning fewer valid rows than expected

    By default, rolling(window=3).mean() requires min_periods=3 — it won't return a value unless 3 non-NaN values are present. You can relax this with rolling(3, min_periods=1).mean(), which will calculate a mean on whatever's available. Use this carefully: a "3-month rolling average" computed on just 1 month is misleading.

    Tip

    When presenting rolling metrics to stakeholders, explicitly label the first few rows as "insufficient history" or drop them from the report. Nothing erodes trust faster than a January value in a "3-month rolling average" column that's actually just January's number.

    Mistake 6: Using pct_change directly on a pivot table with MultiIndex columns

    After a groupby pivot or a pivot_table, you may have MultiIndex columns. pct_change can behave unexpectedly — it operates column by column by default, which may not be what you want. Flatten your column index first with df.columns = ['_'.join(c).strip() for c in df.columns] or work with the long format before pivoting. The MultiIndex lesson covers this in detail.


    Summary & Next Steps

    You now have a complete toolkit for period-over-period analysis in pandas:

    • shift(n) is the foundation: it aligns past or future values alongside current values so you can compare them directly
    • pct_change(n) is shift with the percentage math built in — use it for quick calculations, but reach for shift when you need absolute changes or custom denominator logic
    • Month-over-month uses shift(1) on monthly data; year-over-year uses shift(12)
    • Rolling metrics (rolling(n).mean()) smooth seasonal noise and reveal underlying trends
    • For multi-segment data, always use groupby(...).shift(n) and groupby(...).transform(...) to keep each segment's history isolated
    • Validate your date index before any shift operation — gaps and duplicates cause silent errors

    The skills in this lesson pair naturally with a few other areas worth exploring next. If you want to go deeper on cumulative metrics — running totals, cumulative averages, and ranked performance — check out the lesson on running totals and cumulative calculations with expanding, cumsum, and rank. If you're working with weighted revenue metrics or need custom aggregations beyond simple sums, weighted averages and custom aggregations in groupby will extend what you built here. And when you're ready to automate this entire report on a schedule, building and automating recurring reports with pandas is the natural next step.

    Period comparison analysis is one of those skills that seems straightforward until you hit edge cases — missing months, segment boundaries, sign changes. Now you know how to handle all of them.

    Work With Us

    From insight to implementation

    Reading is the start. When you're ready to build the data, automation, or AI systems behind it, our team turns strategy into shipped results.

    Let's Build

    Python for Data Analysis

    Previous

    Reshaping Wide and Long Data for Reporting: When and How to Use melt, pivot, and unstack in pandas

    Next

    Ranking and Window Calculations in pandas: Running Totals, Cumulative Averages, and Percent Rank with groupby and transform

    Related Insights

    PythonPractitioner

    Cohort Analysis in pandas: Calculating Retention, Churn, and Lifetime Value from Transactional Data

    19 min
    PythonPractitioner

    Combining String, Date, and Numeric Transformations in a pandas Data Cleaning Pipeline: Standardizing Real-World Columns Before Analysis

    18 min
    PythonPractitioner

    Cohort Analysis in pandas: Calculating Retention, Churn, and Lifetime Value from Transaction Data

    20 min

    On this page

    • Introduction
    • Prerequisites
    • The Dataset: Monthly SaaS Revenue by Product Line
    • Understanding `shift`: The Engine Behind Every Period Comparison
    • Month-over-Month Change with `shift` and `pct_change`
    • The manual approach with `shift`
    • The `pct_change` shortcut
    • Year-over-Year Change: The `shift(12)` Pattern
    • Rolling Changes: Smoothing Out Noise
    • Rolling average (moving average)
    • Rolling percentage change
    • Multi-Segment Period Comparisons with `groupby` and `shift`
    • Handling Edge Cases: Gaps, Zeros, and Sign Changes
    • Missing months in the series
    • Division by zero and negative denominators
    • Building the Report DataFrame: Putting It All Together
    • Real-World Project: Automated Monthly Growth Report
    • Hands-On Exercise
    • Common Mistakes & Troubleshooting
    • Summary & Next Steps