Wicked Smart Data
LearnInsightsAboutContact
Sign InLet's Build
LearnInsightsAboutContact
Sign InLet's Build
Wicked Smart Data

Intelligence, automation, and expert execution — plus an elite library of free knowledge. We turn complexity into competitive advantage.

Start a conversation

Platform

  • Learning Paths
  • Insights
  • RSS Feed

Company

  • About
  • Contact
  • Work With Us

Legal

  • Privacy Policy
  • Terms of Service

© 2026 Wicked Smart Data. All rights reserved.

Intelligence · Automation · Advantage

All Insights
Python

Cohort Analysis in pandas: Calculating Retention, Churn, and Lifetime Value from Transaction Data

Learn how to turn a raw transaction log into a complete cohort analysis in pandas — including a retention matrix, period-over-period churn rates, and cumulative customer lifetime value broken down by acquisition cohort. Covers everything from cohort assignment to a reusable production function.

⚡ Practitioner20 min readSep 22, 2026Updated Sep 22, 2026
Cohort Analysis in pandas: Calculating Retention, Churn, and Lifetime Value from Transaction Data
On this page
  • Introduction
  • Prerequisites
  • The Dataset: What You're Starting With
  • Step 1: Assign Each Customer to an Acquisition Cohort
  • Step 2: Build the Retention Matrix
  • Step 3: Visualize the Retention Heatmap
  • Step 4: Calculate Churn Rates
  • Step 5: Compute Revenue per Cohort and Lifetime Value
  • Step 6: Plot LTV Curves by Cohort
  • Step 7: Wrap It Into a Reusable Function
  • Hands-On Exercise
  • Common Mistakes & Troubleshooting
  • Mistake 1: Not normalizing dates to a period
  • Mistake 2: Counting transactions instead of customers for retention
  • Mistake 3: Dividing by cohort size using column 0 vs. a separate lookup
  • Mistake 4: Interpreting the NaN triangle as zero retention
  • Mistake 5: Forgetting about cohort index alignment when merging
  • Taking It Further: Advanced Extensions
  • Segmenting cohorts by acquisition channel
  • Predictive LTV with curve fitting
  • Calculating rolling retention windows
  • Summary & Next Steps
  • Cohort Analysis in pandas: Calculating Retention, Churn, and Lifetime Value from Transaction Data

    Introduction

    You've got two years of transaction data sitting in a database, and your product lead asks: "Are our newer customers sticking around as well as customers we acquired last year?" Your finance team wants to know how much a typical customer is worth over their lifetime so they can set a rational acquisition budget. And your marketing team needs to know which acquisition month produced the customers with the best 90-day retention.

    All three of these questions have the same answer: cohort analysis. A cohort is simply a group of customers who share a common starting event — usually their first purchase — and cohort analysis tracks what those groups do over time. It turns a flat transaction log into a grid that shows how customer behavior evolves month by month, relative to when each customer started.

    By the end of this lesson, you'll be able to take a raw transaction table and produce a complete cohort analysis from scratch: retention curves, churn rates, and a customer lifetime value (LTV) estimate broken out by acquisition cohort. This is one of the highest-value analyses a data practitioner can own, and once you build it in pandas, it runs in seconds on every new data export.

    What you'll learn:

    • How to assign customers to acquisition cohorts from transaction data
    • How to build a month-relative retention matrix and visualize it as a heatmap
    • How to calculate period-by-period churn rates from retention figures
    • How to compute cumulative revenue per cohort and estimate LTV
    • How to package the analysis into a reusable function that runs on any transaction dataset

    Prerequisites

    You should be comfortable with pandas fundamentals — loading data, filtering rows, and working with DataFrames. If you need a refresher on the basics, start with Your First pandas DataFrame: Loading CSV and Excel Files and Exploring Data. You should also be comfortable with groupby aggregation — we'll use it heavily. The Grouping and Aggregating in pandas: groupby as the PivotTable Replacement article covers everything you need. Basic date handling in pandas will help too; the Working with Dates and Time Series in pandas: Parsing, Resampling, and Rolling Windows lesson is the right reference.


    The Dataset: What You're Starting With

    We'll work with a realistic e-commerce transaction log. Each row represents a single order, with a customer ID, order date, and revenue amount. Here's how to generate a synthetic version that mimics real patterns — including some messiness:

    import pandas as pd
    import numpy as np
    import matplotlib.pyplot as plt
    import seaborn as sns
    
    np.random.seed(42)
    
    # Simulate ~5,000 transactions across 2,000 customers over 24 months
    n_customers = 2000
    n_transactions = 5000
    
    customer_ids = np.random.randint(1000, 3000, size=n_transactions)
    order_dates = pd.to_datetime(
        np.random.choice(
            pd.date_range("2022-01-01", "2023-12-31", freq="D"),
            size=n_transactions
        )
    )
    revenue = np.round(np.random.lognormal(mean=4.0, sigma=0.8, size=n_transactions), 2)
    
    transactions = pd.DataFrame({
        "customer_id": customer_ids,
        "order_date": order_dates,
        "revenue": revenue
    })
    
    # Introduce some realistic messiness
    transactions.loc[transactions.sample(frac=0.02).index, "revenue"] = np.nan
    transactions = transactions.sort_values("order_date").reset_index(drop=True)
    
    print(transactions.shape)
    print(transactions.dtypes)
    print(transactions.head(10))
    
    (5000, 3)
    customer_id     int64
    order_date     datetime64[ns]
    revenue        float64
    dtype: object
    
       customer_id order_date  revenue
    0         1847 2022-01-01   102.34
    1         2214 2022-01-01    45.12
    ...
    

    Before you do any cohort work, clean up the missing revenue values. Dropping them is usually correct here — a transaction with unknown revenue can't contribute to LTV:

    transactions = transactions.dropna(subset=["revenue"])
    print(f"Remaining rows: {len(transactions)}")
    

    For any dataset you're loading from CSV or a database, make sure your date column is actually parsed as datetime, not a string. See Cleaning Messy Data with pandas: Missing Values, Duplicates, and Data Types for a systematic approach to that problem.


    Step 1: Assign Each Customer to an Acquisition Cohort

    The foundation of cohort analysis is the cohort date — the period of a customer's very first transaction. Every subsequent transaction from that customer is then measured relative to that starting point.

    We'll work in monthly cohorts, which is the most common granularity for subscription and e-commerce businesses.

    # Step 1a: Find each customer's first order date
    customer_first_order = (
        transactions
        .groupby("customer_id")["order_date"]
        .min()
        .reset_index()
        .rename(columns={"order_date": "cohort_date"})
    )
    
    # Step 1b: Normalize to the first day of the month
    # This groups everyone who first purchased in, say, March 2022 into the same cohort
    customer_first_order["cohort_month"] = customer_first_order["cohort_date"].dt.to_period("M")
    
    print(customer_first_order.head())
    
       customer_id cohort_date cohort_month
    0         1000  2022-01-03      2022-01
    1         1001  2022-02-14      2022-02
    2         1002  2022-01-20      2022-01
    3         1003  2022-03-05      2022-03
    4         1004  2022-01-09      2022-01
    

    Now merge this back onto the transactions table so every transaction row knows its customer's cohort:

    # Step 1c: Join cohort info onto every transaction
    transactions = transactions.merge(
        customer_first_order[["customer_id", "cohort_month"]],
        on="customer_id",
        how="left"
    )
    
    # Step 1d: Calculate the "order period" for each transaction
    transactions["order_month"] = transactions["order_date"].dt.to_period("M")
    
    # Step 1e: Calculate cohort index — how many months after acquisition is this transaction?
    transactions["cohort_index"] = (
        transactions["order_month"] - transactions["cohort_month"]
    ).apply(lambda x: x.n)
    
    print(transactions[["customer_id", "order_date", "revenue", "cohort_month", "order_month", "cohort_index"]].head(10))
    
       customer_id order_date   revenue cohort_month order_month  cohort_index
    0         1847 2022-01-01    102.34      2022-01     2022-01             0
    1         2214 2022-01-01     45.12      2022-01     2022-01             0
    2         1231 2022-01-02     78.90      2022-01     2022-01             0
    ...
    

    cohort_index = 0 means "the same month the customer first bought." cohort_index = 3 means "three months after first purchase." This is the axis that makes cohort analysis readable — you're aligning all cohorts at their starting point regardless of when they actually acquired.

    Key insight

    The cohort index normalizes time to be relative, not absolute. Without this step, January 2022 cohort members at month 6 and July 2022 cohort members at month 6 would be at completely different calendar dates, making comparison meaningless. The cohort index puts everyone on the same ruler.


    Step 2: Build the Retention Matrix

    Retention measures how many customers from a cohort made at least one purchase in each subsequent period. It does not count transactions — it counts distinct customers.

    # Step 2a: Count distinct customers per cohort and cohort_index
    cohort_data = (
        transactions
        .groupby(["cohort_month", "cohort_index"])["customer_id"]
        .nunique()
        .reset_index()
        .rename(columns={"customer_id": "customers"})
    )
    
    # Step 2b: Pivot to create the retention matrix
    cohort_pivot = cohort_data.pivot_table(
        index="cohort_month",
        columns="cohort_index",
        values="customers"
    )
    
    print(cohort_pivot.iloc[:5, :6])
    
    cohort_index      0     1     2     3     4     5
    cohort_month                                     
    2022-01         187    89    74    61    55    48
    2022-02         163    71    58    50    42    39
    2022-03         154    68    55    47    40    35
    2022-04         171    76    62    53    45    42
    2022-05         159    69    54    46    40    36
    

    The column 0 holds the cohort size — the number of customers who made their first purchase in that period. Every other column shows how many of those original customers returned to buy in that relative month.

    Now convert raw counts to percentages, relative to the cohort size (column 0):

    # Step 2c: Divide every column by the cohort size (column 0)
    cohort_sizes = cohort_pivot[0]
    retention_matrix = cohort_pivot.divide(cohort_sizes, axis=0)
    
    # Convert to percentages and round
    retention_matrix = (retention_matrix * 100).round(1)
    
    print(retention_matrix.iloc[:5, :6])
    
    cohort_index    0.0    1.0    2.0    3.0    4.0    5.0
    cohort_month                                          
    2022-01        100.0   47.6   39.6   32.6   29.4   25.7
    2022-02        100.0   43.6   35.6   30.7   25.8   23.9
    2022-03        100.0   44.2   35.7   30.5   26.0   22.7
    2022-04        100.0   44.4   36.3   31.0   26.3   24.6
    2022-05        100.0   43.4   34.0   28.9   25.2   22.6
    

    Every cohort starts at 100% and decays over time. This is your retention curve in matrix form.

    Tip

    The triangle of NaN values in the bottom-right of this matrix is intentional and expected. Recent cohorts haven't had time to reach later cohort indices yet. When you visualize this, those cells should appear blank rather than as zeros.


    Step 3: Visualize the Retention Heatmap

    A heatmap makes the retention matrix immediately scannable. You can spot underperforming cohorts, seasonal dips in retention, and the "shape" of your churn curve at a glance.

    plt.figure(figsize=(14, 8))
    
    # Use a mask to hide NaN cells (the future periods that don't exist yet)
    mask = retention_matrix.isnull()
    
    sns.heatmap(
        retention_matrix,
        mask=mask,
        annot=True,
        fmt=".0f",
        cmap="YlOrRd_r",      # Reversed: darker = higher retention = better
        linewidths=0.5,
        linecolor="white",
        vmin=0,
        vmax=100,
        cbar_kws={"label": "Retention %"}
    )
    
    plt.title("Monthly Cohort Retention Matrix (%)", fontsize=14, fontweight="bold")
    plt.xlabel("Months Since First Purchase (Cohort Index)")
    plt.ylabel("Acquisition Cohort")
    plt.xticks(rotation=0)
    plt.yticks(rotation=0)
    plt.tight_layout()
    plt.show()
    

    The resulting heatmap has acquisition cohorts on the y-axis and months-since-acquisition on the x-axis. The color gradient goes from dark (high retention, good) to light (low retention, bad). The NaN triangle at the bottom right simply won't be rendered, which is exactly right.

    For guidance on making your charts production-ready, see Visualizing Data with matplotlib and seaborn: Charts That Explain Your Analysis.


    Step 4: Calculate Churn Rates

    Churn is the complement of retention. If 47.6% of a cohort returned in month 1, then 52.4% churned after their first purchase. But there's a nuance: there are two ways to define period-level churn.

    Simple (or marginal) churn: How many customers who were active last period dropped off this period?

    # Step 4a: Calculate period-over-period churn within the retention matrix
    # Churn at period N = (retention[N-1] - retention[N]) / retention[N-1]
    
    churn_matrix = pd.DataFrame(index=retention_matrix.index)
    
    for col in retention_matrix.columns[1:]:
        prev_col = col - 1
        if prev_col in retention_matrix.columns:
            churn_matrix[col] = (
                (retention_matrix[prev_col] - retention_matrix[col])
                / retention_matrix[prev_col]
                * 100
            ).round(1)
    
    print(churn_matrix.iloc[:5, :6])
    
    cohort_index     1.0    2.0    3.0    4.0    5.0    6.0
    cohort_month                                           
    2022-01         52.4   16.8   17.7    9.8   12.6   10.8
    2022-02         56.4   18.3   13.8   16.0    7.4    9.0
    2022-03         55.8   19.2   14.6   14.8   12.7   10.5
    2022-04         55.6   18.2   14.6   15.2    6.5    7.0
    2022-05         56.6   21.7   15.0   12.8   10.0    9.4
    

    Notice that the massive drop happens between month 0 and month 1 — more than half of first-time customers don't come back for a second purchase. This is extremely common in e-commerce and is your single biggest lever for revenue growth. After that initial drop, monthly churn stabilizes in the 10–20% range.

    Warning

    Marginal churn calculated this way measures churn among customers who were still active, not among the full original cohort. Be precise about this when presenting to stakeholders. "12% monthly churn" means 12% of remaining active customers left — not 12% of the original cohort.

    You can also look at the average churn rate by cohort index across all cohorts to get a summary curve:

    # Step 4b: Average churn rate by cohort index (across all cohorts)
    avg_churn = churn_matrix.mean(axis=0).reset_index()
    avg_churn.columns = ["cohort_index", "avg_churn_pct"]
    
    print(avg_churn.head(10))
    
       cohort_index  avg_churn_pct
    0           1.0          55.1
    1           2.0          18.4
    2           3.0          15.2
    3           4.0          13.1
    4           5.0          11.3
    ...
    

    This tells a clear story: survive the first month and churn stabilizes significantly. That's your product's "activation" problem staring you in the face.


    Step 5: Compute Revenue per Cohort and Lifetime Value

    Retention tells you about customer counts. LTV requires revenue. The approach is the same structure — but instead of counting customers, we're summing revenue.

    # Step 5a: Total revenue per cohort per cohort index
    revenue_cohort = (
        transactions
        .groupby(["cohort_month", "cohort_index"])["revenue"]
        .sum()
        .reset_index()
        .rename(columns={"revenue": "total_revenue"})
    )
    
    # Step 5b: Pivot to a matrix
    revenue_pivot = revenue_cohort.pivot_table(
        index="cohort_month",
        columns="cohort_index",
        values="total_revenue"
    )
    
    # Step 5c: Revenue per customer per period
    # Divide by cohort sizes so we're comparing apples to apples across cohorts
    revenue_per_customer = revenue_pivot.divide(cohort_sizes, axis=0).round(2)
    
    print(revenue_per_customer.iloc[:5, :6])
    
    cohort_index      0       1       2       3       4       5
    cohort_month                                               
    2022-01        52.14   24.71   20.31   16.44   14.82   12.97
    2022-02        49.83   21.45   17.89   15.21   12.40   11.44
    2022-03        51.02   22.17   18.24   14.87   12.96   11.09
    2022-04        50.74   23.16   19.82   16.02   13.21   12.44
    2022-05        48.93   20.87   16.52   13.89   12.14   10.77
    

    Now compute cumulative revenue per customer — the building block of LTV:

    # Step 5d: Cumulative revenue per customer per cohort
    # This is the "LTV up to month N" for each cohort
    cumulative_revenue = revenue_per_customer.cumsum(axis=1).round(2)
    
    print(cumulative_revenue.iloc[:5, :6])
    
    cohort_index      0       1       2       3       4       5
    cohort_month                                               
    2022-01        52.14   76.85   97.16  113.60  128.42  141.39
    2022-02        49.83   71.28   89.17  104.38  116.78  128.22
    2022-03        51.02   73.19   91.43  106.30  119.26  130.35
    2022-04        50.74   73.90   93.72  109.74  122.95  135.39
    2022-05        48.93   69.80   86.32  100.21  112.35  123.12
    

    Key insight

    cumulative_revenue.iloc[:, 11] gives you the 12-month LTV per customer for each acquisition cohort — a single number that's directly comparable to your customer acquisition cost (CAC). If CAC is $60 and your 12-month LTV is $128, your payback period is under 6 months. If LTV is $48, you have a problem.

    You can extract a clean LTV summary table:

    # Step 5e: Extract LTV at specific milestones: 1, 3, 6, 12 months
    ltv_milestones = cumulative_revenue[[0, 2, 5, 11]].copy()
    ltv_milestones.columns = ["LTV_1mo", "LTV_3mo", "LTV_6mo", "LTV_12mo"]
    ltv_milestones = ltv_milestones.dropna(how="any")  # Only cohorts with full data
    
    print(ltv_milestones)
    
                  LTV_1mo  LTV_3mo  LTV_6mo  LTV_12mo
    cohort_month                                      
    2022-01         52.14    97.16   141.39    192.47
    2022-02         49.83    89.17   128.22    175.91
    2022-03         51.02    91.43   130.35    179.22
    ...
    

    This is the table that goes into a board deck. Each row is a cohort; each column tells you how much revenue a typical customer from that cohort generated by that time in their life.

    For techniques on how to export this kind of table to Excel with proper formatting, see Building Summary Reports with pandas pivot_table and to_excel: Turning Aggregated Data into a Formatted, Multi-Sheet Workbook.


    Step 6: Plot LTV Curves by Cohort

    Visualizing cumulative LTV curves lets you see whether newer cohorts are more or less valuable than older ones — a critical signal about your business trajectory.

    fig, ax = plt.subplots(figsize=(12, 6))
    
    # Only plot cohorts that have at least 6 months of data
    min_periods = 6
    valid_cohorts = cumulative_revenue.dropna(thresh=min_periods).index
    
    for cohort in valid_cohorts:
        row = cumulative_revenue.loc[cohort].dropna()
        ax.plot(
            row.index,
            row.values,
            marker="o",
            markersize=3,
            label=str(cohort),
            alpha=0.7
        )
    
    ax.set_title("Cumulative Revenue per Customer by Acquisition Cohort", fontsize=13, fontweight="bold")
    ax.set_xlabel("Months Since First Purchase")
    ax.set_ylabel("Cumulative Revenue per Customer ($)")
    ax.legend(
        title="Cohort",
        bbox_to_anchor=(1.01, 1),
        loc="upper left",
        fontsize=8
    )
    ax.grid(True, alpha=0.3)
    plt.tight_layout()
    plt.show()
    

    What you're looking for: are the curves for recent cohorts tracking above or below older cohorts at the same cohort index? If they're tracking higher, your product improvements are working. If they're trending lower, something is deteriorating.


    Step 7: Wrap It Into a Reusable Function

    You've built the analysis piece by piece. Now package it so you can run it on any new data export with a single function call. This is how you turn a one-off analysis into infrastructure.

    def cohort_analysis(df, customer_col="customer_id", date_col="order_date", 
                        revenue_col="revenue", freq="M"):
        """
        Perform a full cohort analysis on a transaction DataFrame.
        
        Parameters
        ----------
        df : pd.DataFrame
            Transaction-level data with one row per order.
        customer_col : str
            Column containing unique customer identifiers.
        date_col : str
            Column containing order dates (must be datetime or parseable as such).
        revenue_col : str
            Column containing order revenue.
        freq : str
            Period frequency for cohorts. Default "M" for monthly.
        
        Returns
        -------
        dict with keys:
            - 'retention': Retention matrix (%)
            - 'churn': Period-over-period churn matrix (%)
            - 'revenue_per_customer': Revenue per cohort customer per period
            - 'ltv_cumulative': Cumulative revenue per customer per period
            - 'cohort_sizes': Series of cohort sizes
        """
        df = df.copy()
        
        # Ensure date column is datetime
        df[date_col] = pd.to_datetime(df[date_col])
        
        # Drop rows with missing revenue
        df = df.dropna(subset=[revenue_col])
        
        # Assign cohort month
        first_orders = (
            df.groupby(customer_col)[date_col]
            .min()
            .reset_index()
            .rename(columns={date_col: "cohort_date"})
        )
        first_orders["cohort_month"] = first_orders["cohort_date"].dt.to_period(freq)
        
        df = df.merge(first_orders[[customer_col, "cohort_month"]], on=customer_col, how="left")
        df["order_month"] = df[date_col].dt.to_period(freq)
        df["cohort_index"] = (df["order_month"] - df["cohort_month"]).apply(lambda x: x.n)
        
        # Retention matrix
        cohort_counts = (
            df.groupby(["cohort_month", "cohort_index"])[customer_col]
            .nunique()
            .reset_index()
            .rename(columns={customer_col: "customers"})
        )
        count_pivot = cohort_counts.pivot_table(
            index="cohort_month", columns="cohort_index", values="customers"
        )
        cohort_sizes = count_pivot[0]
        retention = (count_pivot.divide(cohort_sizes, axis=0) * 100).round(1)
        
        # Churn matrix
        churn = pd.DataFrame(index=retention.index)
        for col in retention.columns[1:]:
            prev = col - 1
            if prev in retention.columns:
                churn[col] = (
                    (retention[prev] - retention[col]) / retention[prev] * 100
                ).round(1)
        
        # Revenue matrices
        rev_data = (
            df.groupby(["cohort_month", "cohort_index"])[revenue_col]
            .sum()
            .reset_index()
        )
        rev_pivot = rev_data.pivot_table(
            index="cohort_month", columns="cohort_index", values=revenue_col
        )
        revenue_per_customer = rev_pivot.divide(cohort_sizes, axis=0).round(2)
        ltv_cumulative = revenue_per_customer.cumsum(axis=1).round(2)
        
        return {
            "retention": retention,
            "churn": churn,
            "revenue_per_customer": revenue_per_customer,
            "ltv_cumulative": ltv_cumulative,
            "cohort_sizes": cohort_sizes
        }
    
    
    # Usage
    results = cohort_analysis(transactions)
    print(results["retention"].iloc[:3, :5])
    print(results["ltv_cumulative"].iloc[:3, :5])
    

    This function is now a building block you can drop into any project. If you're loading from a SQL database instead of a CSV, swap in pd.read_sql() at the top — see Reading from SQL Databases into pandas with SQLAlchemy for how to set that up.


    Hands-On Exercise

    Here's a set of progressively challenging tasks to test your understanding. Work through them using the results dictionary from the function above.

    Task 1 — Identify your best cohort: Find the acquisition cohort with the highest 6-month LTV. Which month was it? How does it compare to the overall average?

    # Hint: use ltv_cumulative[5] and idxmax()
    ltv_6mo = results["ltv_cumulative"][5].dropna()
    best_cohort = ltv_6mo.idxmax()
    print(f"Best 6-month LTV cohort: {best_cohort} at ${ltv_6mo[best_cohort]:.2f}")
    print(f"Average 6-month LTV: ${ltv_6mo.mean():.2f}")
    

    Task 2 — Flag high-churn cohorts: Find all cohorts where month-1 churn (column index 1 of the churn matrix) exceeded 60%. These are cohorts where your onboarding experience likely needs work.

    # Hint: filter results["churn"][1] > 60
    high_churn = results["churn"][1][results["churn"][1] > 60]
    print("High month-1 churn cohorts:")
    print(high_churn)
    

    Task 3 — Build a 12-month LTV forecast for a new cohort: Assume a new cohort of 500 customers acquires in the current month. Using the average retention and revenue-per-customer figures across all historical cohorts, project their 12-month cumulative revenue.

    avg_revenue_per_customer = results["revenue_per_customer"].mean(axis=0)
    avg_cumulative = avg_revenue_per_customer.cumsum()
    projected_cohort_revenue = avg_cumulative * 500
    print("Projected cumulative revenue for 500-customer cohort:")
    print(projected_cohort_revenue.head(12))
    

    Task 4 — Visualize month-1 retention trend over time: Plot month-1 retention (column index 1 of the retention matrix) on the y-axis with cohort month on the x-axis. Is retention improving, declining, or flat?


    Common Mistakes & Troubleshooting

    Mistake 1: Not normalizing dates to a period

    If you use raw dates instead of periods for cohort_month, customers who bought on January 5th and January 28th end up in different "cohorts" even though they should be in the same January cohort. Always convert to dt.to_period("M") before grouping.

    # Wrong — uses full date
    df["cohort_month"] = df["first_order_date"]  # Jan 5 ≠ Jan 28
    
    # Right — normalizes to period
    df["cohort_month"] = df["first_order_date"].dt.to_period("M")
    

    Mistake 2: Counting transactions instead of customers for retention

    Retention is about whether a customer came back, not whether they placed multiple orders. Using .count() or .sum() instead of .nunique() on the customer ID will inflate your retention numbers if customers occasionally make multiple purchases in the same period.

    # Wrong
    cohort_counts = df.groupby(["cohort_month", "cohort_index"])["customer_id"].count()
    
    # Right
    cohort_counts = df.groupby(["cohort_month", "cohort_index"])["customer_id"].nunique()
    

    Mistake 3: Dividing by cohort size using column 0 vs. a separate lookup

    If your data has customers in month 0 who somehow don't appear at cohort_index 0 (data quality issue, filtering bug), your cohort sizes will be understated. Always verify:

    # Sanity check: cohort sizes should match your customer-level data
    customer_cohort_counts = (
        transactions
        .drop_duplicates(subset=["customer_id"])
        .groupby("cohort_month")["customer_id"]
        .count()
    )
    matrix_sizes = results["cohort_sizes"]
    
    discrepancies = (customer_cohort_counts - matrix_sizes).abs()
    print(discrepancies[discrepancies > 0])
    

    If you see discrepancies, investigate with Detecting and Resolving Data Quality Issues Across Merged DataFrames.

    Mistake 4: Interpreting the NaN triangle as zero retention

    The bottom-right NaN triangle in your retention and revenue matrices is not zero — it's simply data that doesn't exist yet. If you fill these with zeros before computing averages or visualizations, you'll dramatically underestimate average retention and LTV.

    # Wrong — fills NaN with 0 before computing average retention by period
    avg_retention = results["retention"].fillna(0).mean(axis=0)  # Biased low!
    
    # Right — let pandas skip NaN by default
    avg_retention = results["retention"].mean(axis=0)  # Ignores NaN cells
    

    Mistake 5: Forgetting about cohort index alignment when merging

    When you subtract two Period objects in pandas, the result is a DateOffset object. The .n attribute extracts the integer number of periods. If you skip this and try to use the raw offset as a column, you'll get an unhashable type error.

    # This will fail in pivot_table
    df["cohort_index"] = df["order_month"] - df["cohort_month"]
    
    # This works
    df["cohort_index"] = (df["order_month"] - df["cohort_month"]).apply(lambda x: x.n)
    

    Tip

    If your analysis needs to run monthly on fresh data exports, wrap the whole pipeline in a function that accepts a date cutoff parameter. Customers who first purchased in the last 30 days don't have retention data yet, so you might want to exclude them from the retention matrix to keep the averages clean. Add a min_cohort_age_months parameter to your function.


    Taking It Further: Advanced Extensions

    Segmenting cohorts by acquisition channel

    If your transaction data includes an acquisition channel (organic, paid, email), you can run separate cohort analyses per channel to compare which channel drives the most valuable long-term customers:

    for channel in transactions["channel"].unique():
        channel_df = transactions[transactions["channel"] == channel]
        channel_results = cohort_analysis(channel_df)
        # Store or print channel-level LTV
        print(f"\n{channel} — 6-month LTV:")
        print(channel_results["ltv_cumulative"][5].mean())
    

    Predictive LTV with curve fitting

    Once you have the cumulative revenue curve for a cohort, you can fit a logarithmic or power-law model to extrapolate beyond the observed data:

    from scipy.optimize import curve_fit
    
    def log_ltv_model(x, a, b):
        return a * np.log(x + 1) + b
    
    # Use the average cumulative LTV curve
    avg_ltv = results["ltv_cumulative"].mean(axis=0).dropna()
    x_obs = avg_ltv.index.astype(float).values
    y_obs = avg_ltv.values
    
    popt, _ = curve_fit(log_ltv_model, x_obs, y_obs)
    a, b = popt
    
    # Predict LTV at month 24
    predicted_24mo_ltv = log_ltv_model(24, a, b)
    print(f"Predicted 24-month LTV: ${predicted_24mo_ltv:.2f}")
    

    This is a simplification, but it's a surprisingly effective first-pass model for subscription businesses with stable churn.

    Calculating rolling retention windows

    For businesses with long purchase cycles, you might want to measure "did this customer buy within 90 days" rather than "did they buy in exactly month 3." This requires a slightly different aggregation, but the structure is the same — replace monthly period arithmetic with rolling date window checks using date comparisons.

    Note

    The techniques you've learned here pair naturally with Ranking and Window Calculations in pandas: Running Totals, Cumulative Averages, and Percent Rank with groupby and transform when you want to calculate rolling metrics or rank cohorts by LTV within a period.


    Summary & Next Steps

    Here's what you built in this lesson:

    • Cohort assignment: You extracted each customer's first purchase date and normalized it to a monthly cohort, then calculated the cohort index for every subsequent transaction.
    • Retention matrix: You counted unique customers per cohort per period, pivoted to a matrix, and divided by cohort size to express retention as a percentage.
    • Churn calculation: You derived period-over-period marginal churn rates from the retention matrix.
    • LTV computation: You built revenue-per-customer and cumulative-revenue-per-customer matrices, and extracted LTV at standardized milestones.
    • Reusable function: You wrapped the entire pipeline into a single parameterized function ready for production use.

    This kind of analysis is one of the highest-ROI things you can build as a data practitioner. Most businesses have never looked at their customer data this way, and the insights — especially the month-1 churn rate and the LTV-vs-CAC comparison — are immediately actionable.

    Where to go next:

    • Learn how to schedule this analysis to run automatically on fresh data with Building and Automating Recurring Reports with pandas: Scheduling Scripts to Run Without You
    • Learn how to export your retention and LTV tables into a formatted, multi-tab Excel workbook with Automating Excel Reports with pandas and openpyxl: Formatted Workbooks Without Manual Work
    • If you're pulling transaction data from a production database rather than a CSV export, Reading from SQL Databases into pandas with SQLAlchemy will get you set up end-to-end
    Work With Us

    From insight to implementation

    Reading is the start. When you're ready to build the data, automation, or AI systems behind it, our team turns strategy into shipped results.

    Let's Build

    Python for Data Analysis

    Previous

    Calculating Running Totals, Subtotals, and Grand Totals in pandas for Boardroom-Ready Summary Tables

    Next

    Combining String, Date, and Numeric Transformations in a pandas Data Cleaning Pipeline: Standardizing Real-World Columns Before Analysis

    Related Insights

    PythonExpert

    Segmenting Customers with RFM Analysis in pandas: Calculating Recency, Frequency, and Monetary Scores from Transaction Data

    28 min
    PythonExpert

    Building a Self-Updating Sales Dashboard in pandas and openpyxl: Automated Charts, Conditional Formatting, and Named Ranges from a Live Data Source

    24 min
    PythonExpert

    Automating pandas Data Cleaning with Custom Validation Rules, Error Logs, and a Corrected Output File

    26 min

    On this page

    • Introduction
    • Prerequisites
    • The Dataset: What You're Starting With
    • Step 1: Assign Each Customer to an Acquisition Cohort
    • Step 2: Build the Retention Matrix
    • Step 3: Visualize the Retention Heatmap
    • Step 4: Calculate Churn Rates
    • Step 5: Compute Revenue per Cohort and Lifetime Value
    • Step 6: Plot LTV Curves by Cohort
    • Step 7: Wrap It Into a Reusable Function
    • Hands-On Exercise
    • Common Mistakes & Troubleshooting
    • Mistake 1: Not normalizing dates to a period
    • Mistake 2: Counting transactions instead of customers for retention
    • Mistake 3: Dividing by cohort size using column 0 vs. a separate lookup
    • Mistake 4: Interpreting the NaN triangle as zero retention
    • Mistake 5: Forgetting about cohort index alignment when merging
    • Taking It Further: Advanced Extensions
    • Segmenting cohorts by acquisition channel
    • Predictive LTV with curve fitting
    • Calculating rolling retention windows
    • Summary & Next Steps