Before you filter, visualize, or draw any conclusions from a dataset, you need to know what you're working with. This lesson teaches you how to use pandas' mean, median, std, and describe to get a reliable statistical snapshot of your data in minutes — and how to actually interpret what the numbers are telling you.
Before you build a single chart, run a single filter, or draw a single conclusion from a dataset, you need to know what you're working with. How spread out are the values? Are there wild outliers dragging the average up? Is the "average" order value actually representative of typical behavior, or are a handful of massive enterprise deals skewing everything?
Descriptive statistics are the answer. They're the fast, structured way to get a reliable first impression of any numeric column — the equivalent of taking a step back and surveying the landscape before you start hiking. In pandas, you can calculate all of them with a single method call, or drill into each one individually depending on what question you're trying to answer.
By the end of this lesson, you'll know exactly how to use pandas to summarize numeric data, interpret what the numbers actually mean, and spot problems in your data before they cause trouble downstream.
What you'll learn:
.describe() to get a full statistical profile of your DataFrame in one shotThis lesson assumes you have pandas installed and know how to load data into a DataFrame. If you're starting fresh, Your First pandas DataFrame: Loading CSV and Excel Files and Exploring Data will get you there. Basic familiarity with Python variables and data types is helpful — Python Basics for Excel Users: Variables, Lists, Dictionaries, and Loops covers that ground if you need it.
Imagine you just received a CSV file of sales transactions — 50,000 rows, a dozen columns. Where do you even start? You could immediately start filtering, joining, or visualizing, but you'd be flying blind. You wouldn't know if revenue values are in dollars or thousands. You wouldn't know if quantity has negative values from returns. You wouldn't know if there's a single transaction for $9,000,000 that will torpedo every average you calculate.
Descriptive statistics give you a quick, reliable snapshot of the shape and health of your data. They answer questions like:
This is not just good practice — it's how professional analysts work. The five minutes you spend running .describe() before analysis can save you hours of backtracking when your results don't make sense.
Key insight
Descriptive statistics don't just summarize your data — they validate it. Unexpected minimums, maximums, or means are often the first sign of data quality issues you need to fix before analysis begins.
Throughout this lesson, we'll work with a realistic sales transaction dataset. Let's create it directly in pandas so you can follow along without downloading anything:
import pandas as pd
import numpy as np
# Simulate a sales transactions dataset
np.random.seed(42)
n = 500
data = {
"order_id": range(1001, 1001 + n),
"customer_id": np.random.randint(100, 300, n),
"region": np.random.choice(["North", "South", "East", "West"], n),
"product_category": np.random.choice(["Electronics", "Clothing", "Home", "Sports"], n),
"quantity": np.random.randint(1, 20, n),
"unit_price": np.round(np.random.uniform(9.99, 499.99, n), 2),
"discount_pct": np.random.choice([0, 5, 10, 15, 20], n),
"days_to_ship": np.random.randint(1, 15, n),
}
df = pd.DataFrame(data)
# Add a revenue column
df["revenue"] = np.round(df["quantity"] * df["unit_price"] * (1 - df["discount_pct"] / 100), 2)
# Introduce a few missing values to make it realistic
df.loc[np.random.choice(df.index, 15, replace=False), "days_to_ship"] = np.nan
print(df.shape)
print(df.head())
This gives us 500 rows with numeric columns representing quantities, prices, discounts, shipping times, and revenue. It's the kind of dataset you'd actually encounter in a business context.
Before we write any code, let's build the conceptual foundation. These four measures are the workhorses of descriptive analysis.
The mean is what most people call "the average." Add up all the values and divide by the count. In pandas:
print(df["revenue"].mean())
Output (approximate):
2643.81
The mean is useful, but it's sensitive to outliers. A single enormous value can drag the mean far from what's "typical." That's why you almost never interpret the mean in isolation.
The median is the value that sits exactly in the middle when all values are sorted. Half the values are above it, half below. In pandas:
print(df["revenue"].median())
Output (approximate):
2435.17
Notice that the median is lower than the mean here. That's a signal: the distribution is likely skewed to the right, meaning a smaller number of high-revenue transactions are pulling the mean upward.
Tip
When mean and median diverge significantly, think hard about which one better represents "typical." For salary data, house prices, or revenue — where a few large values are common — the median is usually more meaningful.
Standard deviation (often abbreviated as std or σ) measures how much values typically deviate from the mean. A small standard deviation means values cluster tightly around the mean. A large one means they're spread widely.
print(df["revenue"].std())
Output (approximate):
1891.43
On its own, standard deviation is hard to interpret without context. But compared to the mean, it's very informative. Here, a standard deviation of ~$1,891 against a mean of ~$2,644 tells you there's substantial variation — revenue orders are not all clustered around the average.
These are simple but essential:
print(df["revenue"].min())
print(df["revenue"].max())
Always check these. Negative minimums in a quantity column, or a maximum that's 100x larger than the mean, are often data quality issues worth investigating before you continue.
You can call any of these statistical methods directly on a Series (a single column) or on an entire DataFrame. Let's see both patterns.
col = df["revenue"]
print("Mean: ", col.mean())
print("Median: ", col.median())
print("Std Dev:", col.std())
print("Min: ", col.min())
print("Max: ", col.max())
print("Count: ", col.count()) # Non-null count
print("Sum: ", col.sum())
Each of these returns a single scalar value — one number. This is the right pattern when you're investigating a specific column.
Pass a list of column names to get results for several columns simultaneously:
cols = ["quantity", "unit_price", "discount_pct", "days_to_ship", "revenue"]
print(df[cols].mean())
print(df[cols].median())
This returns a pandas Series where each index label is a column name. Very useful for quickly comparing typical values across numeric fields.
Note
When you call .mean() on a DataFrame (not a Series), pandas computes the mean for every numeric column by default, skipping non-numeric ones. This is convenient but can include columns you don't want — like order_id or customer_id — so it's usually better to specify columns explicitly.
Here's where pandas really earns its keep. Instead of calling each statistical method one by one, .describe() gives you everything at once:
df.describe()
Output:
order_id customer_id quantity unit_price discount_pct days_to_ship revenue
count 500.000000 500.000000 500.000000 500.000000 500.000000 485.000000 500.000000
mean 1250.500000 199.342000 9.986000 255.198820 9.600000 7.538144 2643.812450
std 144.481385 57.744003 5.445153 141.898456 7.452893 3.911248 1891.425033
min 1001.000000 100.000000 1.000000 9.990000 0.000000 1.000000 9.99....
25% 1125.750000 150.000000 5.000000 130.502500 0.000000 5.000000 927.812500
50% 1250.500000 199.500000 10.000000 255.875000 10.000000 8.000000 2435.170000
75% 1375.250000 249.000000 15.000000 382.167500 15.000000 11.000000 4142.227500
max 1500.000000 299.000000 19.000000 499.880000 20.000000 14.000000 9423.660000
Let's walk through every row of this output, because each one is telling you something specific.
count — How many non-null values are in each column. Notice days_to_ship shows 485, not 500. That immediately tells you 15 values are missing. This is how .describe() doubles as a data quality check.
mean — The arithmetic average. We already covered this. Notice order_id and customer_id have means here even though they're IDs, not measures — this is a reminder to select only meaningful columns.
std — Standard deviation. The spread of values around the mean.
min — The smallest value in the column.
25% — The 25th percentile (also called Q1, or the first quartile). 25% of values fall at or below this point.
50% — The 50th percentile, which is exactly the median. pandas labels it 50% rather than "median" in the .describe() output.
75% — The 75th percentile (Q3). 75% of values fall at or below this point.
max — The largest value.
Key insight
The gap between 25% and 75% is called the Interquartile Range (IQR). It represents the "middle 50%" of your data and is resistant to outliers. When you see a min or max that's wildly far from the 25th or 75th percentile, that's a strong signal of outliers worth investigating.
You don't have to run .describe() on your entire DataFrame. Narrow it down to the columns that actually matter:
df[["quantity", "unit_price", "revenue"]].describe()
This gives you a cleaner, more focused view — especially useful when your DataFrame has dozens of columns.
By default, .describe() shows the 25th, 50th, and 75th percentiles. You can customize this:
df[["revenue"]].describe(percentiles=[0.1, 0.25, 0.5, 0.75, 0.9, 0.99])
Adding the 90th and 99th percentile is particularly useful when you suspect outliers at the top of the distribution. If the 99th percentile is dramatically higher than the 90th, you have a heavy tail worth understanding.
Sometimes you want different statistics for different columns, or you want to combine several statistics into a single clean table. The .agg() method (short for aggregate) is perfect for this:
df[["quantity", "unit_price", "revenue", "days_to_ship"]].agg(
["mean", "median", "std", "min", "max"]
)
This returns a tidy DataFrame where each row is a statistic and each column is a field. It's the format you'd want if you were building a summary report or exporting results.
You can also specify different aggregations for different columns using a dictionary:
df.agg({
"revenue": ["mean", "median", "std"],
"quantity": ["mean", "min", "max"],
"days_to_ship": ["mean", "median"]
})
This is the kind of flexibility that makes pandas much more powerful than running AVERAGE() and STDEV() formulas in Excel. And it connects naturally to the more advanced aggregation patterns covered in Grouping and Aggregating in pandas: groupby as the PivotTable Replacement.
A critical thing to know: by default, pandas statistical methods skip null values (NaN). That means .mean() will give you the mean of the non-null values only — which is usually the right behavior.
# days_to_ship has 15 missing values
print(df["days_to_ship"].mean()) # Computes mean of the 485 non-null values
print(df["days_to_ship"].count()) # Shows 485, not 500
If you want to know how many nulls exist before computing statistics, check it explicitly:
print(df.isnull().sum())
This gives you a count of nulls per column. Always run this before trusting your descriptive stats. If a column is 30% null, the mean you calculate is only representative of the non-missing data — which may or may not be a valid assumption depending on why values are missing.
Warning
If you use skipna=False, pandas will return NaN for any column that has even a single missing value. This is rarely what you want for exploratory analysis, but it's useful when you need to be strict about data completeness.
# This will return NaN because days_to_ship has nulls
print(df["days_to_ship"].mean(skipna=False))
Understanding and cleaning messy data before running statistics is always a good idea for production analysis.
.describe() behaves differently when applied to non-numeric columns. If you want to include them explicitly, use include="all" or include="object":
df.describe(include="object")
Output:
region product_category
count 500 500
unique 4 4
top West Electronics
freq 134 133
For categorical columns, you get:
This is useful for a quick sanity check on categorical fields, though for deeper categorical analysis, you'll want to explore Summarizing and Cross-Tabulating Categorical Data in pandas.
Descriptive statistics become even more powerful when paired with a quick chart. After running .describe(), you'll often want to visualize the distribution to see what the numbers are actually hiding.
import matplotlib.pyplot as plt
fig, axes = plt.subplots(1, 2, figsize=(12, 4))
df["revenue"].hist(bins=40, ax=axes[0], color="steelblue", edgecolor="white")
axes[0].axvline(df["revenue"].mean(), color="red", linestyle="--", label=f"Mean: {df['revenue'].mean():.0f}")
axes[0].axvline(df["revenue"].median(), color="green", linestyle="--", label=f"Median: {df['revenue'].median():.0f}")
axes[0].legend()
axes[0].set_title("Revenue Distribution")
df["days_to_ship"].hist(bins=14, ax=axes[1], color="coral", edgecolor="white")
axes[1].set_title("Days to Ship Distribution")
plt.tight_layout()
plt.show()
Plotting the mean and median on the same histogram makes the skew immediately visible. When the red line (mean) sits to the right of the green line (median), you have a right-skewed distribution. When the lines are nearly identical, the distribution is roughly symmetric.
For a deeper dive into visualization techniques, Visualizing Data with matplotlib and seaborn: Charts That Explain Your Analysis covers the full range of chart types you'll want in your toolkit.
Work through this exercise using the dataset we created at the start of the lesson.
Part 1: Run .describe() on the full DataFrame and answer these questions by reading the output:
days_to_ship?Part 2: Use .agg() to create a summary table that shows the mean, median, min, and max for quantity, unit_price, and revenue in a single DataFrame.
Part 3: Calculate the 10th and 90th percentile of revenue using .describe(percentiles=[...]). Then calculate the same values using .quantile():
print(df["revenue"].quantile(0.10))
print(df["revenue"].quantile(0.90))
Verify the outputs match.
Part 4: Add a column called high_value that is True if revenue is above the 75th percentile, then calculate the mean revenue separately for high-value and standard orders. (Hint: use .describe() to find the 75th percentile value first, then use boolean masking as covered in Selecting and Filtering Data in pandas: loc, iloc, and Boolean Masks.)
Mistake: Running .describe() on the whole DataFrame and trusting all the columns
order_id and customer_id are identifiers, not measures. Their mean and standard deviation are meaningless. Always filter to relevant numeric columns before running descriptive statistics for analytical purposes.
Mistake: Treating the mean as "typical" without checking the median
If mean and median differ significantly, the mean is being distorted by outliers. Report both, and decide which represents your question better.
Mistake: Ignoring the count row in .describe()
If count is less than the number of rows in your DataFrame, you have missing values. Know this before you interpret any other statistic from that column.
Mistake: Forgetting that .std() in pandas uses ddof=1 by default
By default, pandas calculates the sample standard deviation (dividing by n-1, not n). This is the statistically correct choice for samples from a larger population — which is almost always what you're working with. If you need the population standard deviation, pass ddof=0:
df["revenue"].std(ddof=0) # Population std
df["revenue"].std(ddof=1) # Sample std (default)
Tip
For the vast majority of data analysis work, leave ddof at its default of 1. The difference is negligible for large datasets and matters mostly in formal statistical inference.
Mistake: Not checking for nulls before comparing means across columns
If column A has 500 values and column B has 300 (due to nulls), comparing their means directly may be misleading. Always check count in .describe() or run .isnull().sum() first.
You now have a complete toolkit for summarizing numeric data before analysis:
.mean() — The arithmetic average, sensitive to outliers.median() — The middle value, resistant to outliers.std() — How spread out values are around the mean.min() / .max() — The boundaries of your data.describe() — All of the above plus percentiles, in one shot.agg() — Flexible custom summaries across multiple columnsThe workflow is simple: before you analyze anything, run .describe(). Check the count row for missing values. Compare mean and median to understand skew. Look at the min and max for outliers. You'll catch more data quality issues in those five minutes than you would after hours of analysis.
From here, the natural next step is to bring these statistics into group-level analysis — calculating the mean revenue by region, or the median days_to_ship by product category. That's exactly what Grouping and Aggregating in pandas: groupby as the PivotTable Replacement covers. And if you're preparing data for a broader analysis project, Validating and Profiling a New Dataset with pandas: Row Counts, Distributions, and Outlier Checks Before You Analyze takes this foundational work to the next level with a structured profiling approach.