Combining String, Date, and Numeric Transformations in a pandas Data Cleaning Pipeline: Standardizing Real-World Columns Before Analysis

Real data is almost never clean. If you've spent any time working with data from CRMs, ERP exports, form submissions, or legacy databases, you know the drill: phone numbers in four different formats, dates spelled out as "Jan 5th, 2023" next to ISO timestamps, currency fields with dollar signs and commas, product codes with inconsistent casing and trailing whitespace. Each of these issues is manageable on its own. The real challenge — and the real skill — is building a pipeline that handles all of them together, in a systematic, repeatable way.

This lesson is about that pipeline. We're going to work through a realistic scenario: a sales transaction export that combines all the messiness you'd encounter in production — strings that need normalization, dates in multiple formats, numeric fields encoded as text, and a few columns that need to be derived from combining all three. By the end, you'll have a reusable cleaning framework you can drop into any data project.

What you'll learn:

• How to apply string normalization (stripping, casing, regex substitution) to messy categorical and free-text columns
• How to parse dates from mixed formats and handle timezone-aware timestamps
• How to extract numeric values from currency strings, percentage fields, and coded identifiers
• How to sequence these transformations safely so each step feeds cleanly into the next
• How to encapsulate the whole pipeline into a function that's repeatable and testable

---

Prerequisites

You should already be comfortable loading DataFrames, inspecting dtypes, and performing basic column operations. If you need a refresher on loading data or initial exploration, see Your First pandas DataFrame: Loading CSV and Excel Files and Exploring Data. For foundational string cleaning concepts, Text Cleanup at Scale with pandas String Methods and Regular Expressions is a helpful companion. You should also be comfortable with the concept of missing values and type coercion — covered in Cleaning Messy Data with pandas: Missing Values, Duplicates, and Data Types.

---

The Dataset: A Realistic Mess

Let's construct a dataset that mirrors what you'd actually receive from a sales operations team or a CRM export. We'll build it programmatically so you can run this from scratch:

import pandas as pd
import numpy as np
import re

raw_data = {
    "transaction_id": ["TXN-00123", "txn-00124", "TXN00125", " TXN-00126", "TXN-00127 ", "TXN-00128"],
    "customer_name": ["Alice Fontaine", "  bob NGUYEN", "Carol O'Brien", "DAVID PARK", "eve  martinez", "Frank Li"],
    "sale_date": ["2024-01-15", "January 18, 2024", "01/22/2024", "2024-02-03T09:15:00", "Feb 7th, 2024", "2024-02-10"],
    "region": ["North-East", "north east", "NORTHEAST", "North East", "NE", " northeast"],
    "revenue": ["$14,250.00", "$8,750", "9,500.00", "$22,100.50", "11000", "$6,200.75"],
    "discount_pct": ["10%", "5.5%", "0%", "12.5%", "7%", "NaN"],
    "rep_id": ["REP-042", "rep042", "REP042", "REP-42", "Rep-042", "REP-0042"],
    "product_tier": ["Enterprise", " enterprise ", "ENTERPRISE", "Standard", "standard ", "STANDARD"],
    "days_to_close": ["32", "18", "45 days", "27", "N/A", "61 days"],
    "notes": ["  Follow-up needed. ", "Contract sent.", None, "Renewal - high priority", "  ", "New account; referred by Lee."],
}

df = pd.DataFrame(raw_data)
print(df.dtypes)

Run this and every column comes back as object. That's pandas telling you it has no idea what these values mean — everything is a string until you tell it otherwise. This is your starting point in almost every real project.

Let's walk through the cleaning systematically: strings first, then dates, then numerics, then derived columns.

---

Stage 1: String Normalization

String issues tend to cascade. A category field with "NORTHEAST", "north east", and "NE" will silently explode any groupby you run later. The good news is that pandas .str accessor handles most of these in a single, readable chain.

Stripping and Casing

# Normalize transaction_id: uppercase, strip whitespace, standardize separator
df["transaction_id"] = (
    df["transaction_id"]
    .str.strip()
    .str.upper()
    .str.replace(r"^TXN(\d+)$", r"TXN-\1", regex=True)  # handles TXN00125 → TXN-00125
)

# Normalize customer_name: strip, title case, collapse internal spaces
df["customer_name"] = (
    df["customer_name"]
    .str.strip()
    .str.title()
    .str.replace(r"\s+", " ", regex=True)  # "eve  martinez" → "Eve Martinez"
)

print(df[["transaction_id", "customer_name"]])

Notice we're chaining .str methods — each one returns a Series, so the next .str call works cleanly. The regex in the transaction_id step uses a lookahead pattern: ^TXN(\d+)$ matches the format without a hyphen and adds one back.

Normalizing Categorical Columns

The region column is a classic: six rows, at least four distinct spellings of what should be two categories. We need to standardize before we can group by it.

# Step 1: normalize the raw text
df["region_clean"] = (
    df["region"]
    .str.strip()
    .str.lower()
    .str.replace(r"[-\s]+", "", regex=True)  # remove hyphens and spaces
)

# Step 2: map abbreviated or variant forms to canonical values
region_map = {
    "northeast": "North East",
    "ne": "North East",
    "northwest": "North West",
    "nw": "North West",
    "southeast": "South East",
    "se": "South East",
}

df["region"] = df["region_clean"].map(region_map).fillna(df["region_clean"].str.title())
df.drop(columns=["region_clean"], inplace=True)

print(df["region"].value_counts())

The two-step approach here — normalize first, then map — is deliberate. You need a consistent key to map against. If you try to map against the raw messy values, you'll miss entries and fill them with NaN silently.

Normalizing the Rep ID

Rep IDs are a common headache: the same rep might appear as "REP-042", "rep042", "REP-42", or "REP-0042". We need a canonical format, which means extracting the numeric part and zero-padding it consistently:

def normalize_rep_id(val):
    """Extract numeric portion and return zero-padded canonical ID."""
    if pd.isna(val):
        return np.nan
    digits = re.sub(r"[^\d]", "", str(val))  # strip everything non-numeric
    return f"REP-{int(digits):04d}"           # zero-pad to 4 digits

df["rep_id"] = df["rep_id"].apply(normalize_rep_id)
print(df["rep_id"])

Output:

0    REP-0042
1    REP-0042
2    REP-0042
3    REP-0042
4    REP-0042
5    REP-0042

All six variants map to REP-0042. This is exactly the kind of subtle normalization that prevents join failures later — if you're planning to merge this data against a rep master table, the keys must match exactly.

Cleaning the Notes Column

Free-text notes are the wild west. At minimum, we want to strip leading/trailing whitespace and convert truly empty strings (just spaces) to NaN:

df["notes"] = df["notes"].str.strip()
df["notes"] = df["notes"].replace("", np.nan)  # catches "  " after stripping

print(df["notes"])

---

Stage 2: Date Parsing

Dates are where a lot of analysts give up and just leave things as strings — which works until you need to filter by month, compute days between events, or resample by week. The investment in proper parsing pays off every time.

Handling Mixed Date Formats

The sale_date column has at least four different formats. pd.to_datetime() with format="mixed" (available in pandas 2.0+) or infer_datetime_format=True handles most of them:

# pandas 2.0+: format="mixed" handles heterogeneous formats
df["sale_date"] = pd.to_datetime(df["sale_date"], format="mixed", dayfirst=False)

print(df["sale_date"])
print(df["sale_date"].dtype)

But wait — "Feb 7th, 2024" has an ordinal suffix ("th") that pd.to_datetime won't parse directly. Let's handle that with a preprocessing step:

def clean_date_string(val):
    """Remove ordinal suffixes like 1st, 2nd, 3rd, 4th before parsing."""
    if pd.isna(val):
        return val
    return re.sub(r"(\d+)(st|nd|rd|th)\b", r"\1", str(val))

df["sale_date_raw"] = df["sale_date_raw"] if "sale_date_raw" in df.columns else df["sale_date"].astype(str)

# Let's restart from the raw data for this column
raw_dates = pd.Series([
    "2024-01-15", "January 18, 2024", "01/22/2024",
    "2024-02-03T09:15:00", "Feb 7th, 2024", "2024-02-10"
])

cleaned_dates = raw_dates.apply(clean_date_string)
df["sale_date"] = pd.to_datetime(cleaned_dates, format="mixed", dayfirst=False)

print(df["sale_date"])

Now all six dates parse correctly to datetime64[ns].

Extracting Date Components

Once you have a proper datetime column, you can extract the pieces you need for analysis. For a sales dataset, the most common derived columns are:

df["sale_year"] = df["sale_date"].dt.year
df["sale_month"] = df["sale_date"].dt.month
df["sale_month_name"] = df["sale_date"].dt.strftime("%B")   # "January", "February", etc.
df["sale_quarter"] = df["sale_date"].dt.quarter
df["sale_weekday"] = df["sale_date"].dt.day_name()
df["days_since_sale"] = (pd.Timestamp("today").normalize() - df["sale_date"]).dt.days

print(df[["sale_date", "sale_year", "sale_quarter", "sale_month_name", "days_since_sale"]])

For a deeper treatment of time series operations including resampling and rolling windows, see Working with Dates and Time Series in pandas: Parsing, Resampling, and Rolling Windows.

---

Stage 3: Numeric Extraction and Coercion

Numeric columns stored as strings are a silent killer. You won't notice them in a .head() call, but the moment you try to sum or average them, pandas either raises an error or — worse — silently produces wrong results via string concatenation.

Parsing Currency Strings

def parse_currency(val):
    """Strip $, commas, and whitespace, then convert to float."""
    if pd.isna(val):
        return np.nan
    cleaned = re.sub(r"[$,\s]", "", str(val))
    try:
        return float(cleaned)
    except ValueError:
        return np.nan

df["revenue"] = df["revenue"].apply(parse_currency)
print(df["revenue"].dtype)   # float64
print(df["revenue"].describe())

This function is defensive: it handles missing values explicitly and catches conversion errors rather than crashing. In a pipeline that runs overnight against a live export, you want robustness, not perfection.

Parsing Percentage Fields

def parse_percentage(val):
    """Remove % sign and convert to decimal (e.g., '10%' → 0.10)."""
    if pd.isna(val) or str(val).strip().lower() in ("nan", "n/a", ""):
        return np.nan
    cleaned = str(val).replace("%", "").strip()
    try:
        return float(cleaned) / 100.0
    except ValueError:
        return np.nan

df["discount_pct"] = df["discount_pct"].apply(parse_percentage)
print(df["discount_pct"])

Notice the explicit check for "nan" and "n/a" as strings — these come through from Excel exports and CSV files constantly, and they look like real values until you check.

Extracting Numerics from Mixed Text

The days_to_close column has a mix of plain integers and strings like "45 days" and "N/A":

def parse_days(val):
    """Extract first numeric value from a mixed string."""
    if pd.isna(val):
        return np.nan
    match = re.search(r"\d+", str(val))
    if match:
        return int(match.group())
    return np.nan

df["days_to_close"] = df["days_to_close"].apply(parse_days)
print(df["days_to_close"])
print(df["days_to_close"].dtype)   # float64 (because of NaN)

The re.search(r"\d+", ...) approach is powerful because it doesn't care whether the number is at the beginning, middle, or end of the string. It just finds the first sequence of digits and returns it. For "45 days", "32", and "27", all three work correctly.

---

Stage 4: Derived Columns and Cross-Type Calculations

With clean strings, proper dates, and real numeric types in place, you can now safely create derived columns that combine all three.

Revenue After Discount

df["revenue_net"] = df["revenue"] * (1 - df["discount_pct"].fillna(0))
print(df[["transaction_id", "revenue", "discount_pct", "revenue_net"]])

Tier Labeling from Multiple Columns

# Normalize product_tier first
df["product_tier"] = df["product_tier"].str.strip().str.title()

# Create a combined segment label
df["segment_label"] = df["product_tier"] + " / " + df["region"]
print(df["segment_label"])

Flag Slow Deals

# Flag deals that took longer than 30 days to close
df["slow_close"] = df["days_to_close"].gt(30)
print(df[["transaction_id", "days_to_close", "slow_close"]])

These kinds of derived columns — segment labels, boolean flags, net values — are the payoff for doing the upstream cleaning properly. They rely on the fact that product_tier is now consistently title-cased, revenue is a float, and discount_pct is a decimal.

For more on creating conditional columns like this, see Conditional Columns and Bucketing in pandas: Creating New Fields with np.where, cut, and map.

---

Stage 5: Assembling the Full Pipeline

Now we wrap everything into a single, reusable function. This is what separates a one-time notebook hack from a production-grade cleaning script.

import pandas as pd
import numpy as np
import re


def normalize_rep_id(val):
    if pd.isna(val):
        return np.nan
    digits = re.sub(r"[^\d]", "", str(val))
    return f"REP-{int(digits):04d}" if digits else np.nan


def clean_date_string(val):
    if pd.isna(val):
        return val
    return re.sub(r"(\d+)(st|nd|rd|th)\b", r"\1", str(val))


def parse_currency(val):
    if pd.isna(val):
        return np.nan
    cleaned = re.sub(r"[$,\s]", "", str(val))
    try:
        return float(cleaned)
    except ValueError:
        return np.nan


def parse_percentage(val):
    if pd.isna(val) or str(val).strip().lower() in ("nan", "n/a", ""):
        return np.nan
    cleaned = str(val).replace("%", "").strip()
    try:
        return float(cleaned) / 100.0
    except ValueError:
        return np.nan


def parse_days(val):
    if pd.isna(val):
        return np.nan
    match = re.search(r"\d+", str(val))
    return int(match.group()) if match else np.nan


REGION_MAP = {
    "northeast": "North East",
    "ne": "North East",
    "northwest": "North West",
    "nw": "North West",
    "southeast": "South East",
    "se": "South East",
    "southwest": "South West",
    "sw": "South West",
}


def clean_sales_pipeline(df: pd.DataFrame) -> pd.DataFrame:
    """
    Full cleaning pipeline for a raw sales transaction export.
    Returns a new DataFrame with standardized, analysis-ready columns.
    """
    df = df.copy()  # never mutate the original

    # --- STRING NORMALIZATION ---
    df["transaction_id"] = (
        df["transaction_id"]
        .str.strip()
        .str.upper()
        .str.replace(r"^TXN(\d+)$", r"TXN-\1", regex=True)
    )

    df["customer_name"] = (
        df["customer_name"]
        .str.strip()
        .str.title()
        .str.replace(r"\s+", " ", regex=True)
    )

    region_normalized = (
        df["region"]
        .str.strip()
        .str.lower()
        .str.replace(r"[-\s]+", "", regex=True)
    )
    df["region"] = region_normalized.map(REGION_MAP).fillna(region_normalized.str.title())

    df["rep_id"] = df["rep_id"].apply(normalize_rep_id)
    df["product_tier"] = df["product_tier"].str.strip().str.title()

    df["notes"] = df["notes"].str.strip().replace("", np.nan)

    # --- DATE PARSING ---
    df["sale_date"] = pd.to_datetime(
        df["sale_date"].apply(clean_date_string),
        format="mixed",
        dayfirst=False,
    )
    df["sale_year"] = df["sale_date"].dt.year
    df["sale_month"] = df["sale_date"].dt.month
    df["sale_quarter"] = df["sale_date"].dt.quarter
    df["sale_month_name"] = df["sale_date"].dt.strftime("%B")

    # --- NUMERIC EXTRACTION ---
    df["revenue"] = df["revenue"].apply(parse_currency)
    df["discount_pct"] = df["discount_pct"].apply(parse_percentage)
    df["days_to_close"] = df["days_to_close"].apply(parse_days)

    # --- DERIVED COLUMNS ---
    df["revenue_net"] = df["revenue"] * (1 - df["discount_pct"].fillna(0))
    df["segment_label"] = df["product_tier"] + " / " + df["region"]
    df["slow_close"] = df["days_to_close"].gt(30)

    return df


# Run it
clean_df = clean_sales_pipeline(df)
print(clean_df.dtypes)
print(clean_df.head())

The df.copy() at the top is non-negotiable. Without it, you're modifying the original DataFrame in place, which means running the function twice produces wrong results (you're cleaning already-cleaned data).

---

Stage 6: Validating Your Output

A cleaning pipeline without validation is a black box. You need to assert that the output is what you expect before handing it off to any analysis or export step.

def validate_clean_df(df: pd.DataFrame) -> None:
    """Assert that cleaned DataFrame meets structural expectations."""
    errors = []

    # Check dtypes
    if df["revenue"].dtype != "float64":
        errors.append("revenue should be float64")
    if not pd.api.types.is_datetime64_any_dtype(df["sale_date"]):
        errors.append("sale_date should be datetime")
    if df["discount_pct"].dtype != "float64":
        errors.append("discount_pct should be float64")

    # Check no obviously invalid values
    if df["revenue"].lt(0).any():
        errors.append("revenue has negative values")
    if df["discount_pct"].dropna().gt(1).any():
        errors.append("discount_pct has values > 1.0 (check % conversion)")
    if df["transaction_id"].str.match(r"^TXN-\d+$").all() is False:
        errors.append("Some transaction_ids do not match expected pattern TXN-XXXXX")

    # Check for unexpected nulls in key fields
    for col in ["transaction_id", "sale_date", "revenue", "rep_id"]:
        if df[col].isna().any():
            errors.append(f"{col} has unexpected nulls after cleaning")

    if errors:
        raise ValueError("Cleaning validation failed:\n" + "\n".join(f"  - {e}" for e in errors))
    else:
        print("✓ All validation checks passed.")


validate_clean_df(clean_df)

This validation layer catches regressions. If someone on your team changes the source file format and the pipeline silently produces garbage, these assertions will surface the failure immediately rather than letting it propagate into a report.

For more on profiling datasets before and after transformations, see Validating and Profiling a New Dataset with pandas: Row Counts, Distributions, and Outlier Checks Before You Analyze.

---

Hands-On Exercise

Here's a messier version of the dataset for you to clean using the techniques from this lesson. Some new wrinkles have been introduced — your job is to extend the pipeline to handle them.

exercise_data = {
    "transaction_id": ["TXN-00201", "txn00202", " TXN-203 ", "TXN-00204", "TXN-205", "txn-00206"],
    "customer_name": ["  Hannah BROOKS", "ivan Petrov  ", "JULIA SANTOS", "kim CHEN-WANG", "Leo D'Angelo", "  "],
    "sale_date": ["March 3rd, 2024", "2024-03-10", "03/15/2024", "2024-03-20T14:30:00", "Mar 25, 2024", "3/30/24"],
    "region": ["SOUTH-WEST", "sw", "South West", " SW ", "southwest", "South-West"],
    "revenue": ["$31,000", "27500.00", "$18,750.50", "NaN", "$42,200", "$9,800.00"],
    "discount_pct": ["8%", "0%", "15.5%", "3%", "n/a", "20%"],
    "rep_id": ["REP-099", "rep99", "REP0099", "Rep-99", "REP-099", "rEP099"],
    "product_tier": ["Premium", " premium", "PREMIUM", "Standard", "standard", "STANDARD"],
    "days_to_close": ["14", "28 days", "N/A", "52", "33 days", "7"],
    "notes": ["  Renewal due Q3.", None, "New logo account", "  ", "Upsell opportunity - discuss in call", "Contract executed."],
}

exercise_df = pd.DataFrame(exercise_data)

Your tasks:

1. Run clean_sales_pipeline() on this DataFrame. Note which rows fail or produce unexpected results.
2. The "3/30/24" date format (two-digit year) may not parse correctly. Investigate and fix.
3. customer_name has a completely blank entry (" "). It should become NaN after cleaning — add logic for this.
4. Add a new derived column: revenue_per_day = revenue_net / days_to_close. Handle the division-by-zero case where days_to_close is 0 or NaN.
5. Run validate_clean_df() on your result. Fix any failures it surfaces.

---

Common Mistakes & Troubleshooting

Mistake 1: Mutating the Original DataFrame

# BAD: modifies df in place
def clean(df):
    df["revenue"] = df["revenue"].apply(parse_currency)
    return df

# GOOD: work on a copy
def clean(df):
    df = df.copy()
    df["revenue"] = df["revenue"].apply(parse_currency)
    return df

This is the single most common source of "why does my data look different the second time I run this?" bugs.

Mistake 2: Applying `.str` Methods to Non-String Series

If a column has been partially converted to float (e.g., pandas inferred some values as numeric), .str methods will return NaN for those rows silently. Always cast to string before applying .str methods if you're unsure of the dtype:

# Safe pattern
df["col"] = df["col"].astype(str).str.strip()
# But watch out — this converts NaN to the string "nan"!
df["col"] = df["col"].replace("nan", np.nan)

Mistake 3: The Percentage-Greater-Than-One Trap

If you store discount_pct as 10.0 instead of 0.10, then revenue * (1 - discount_pct) gives you negative revenue. Your validation check should always assert discount_pct <= 1.0 for every non-null value.

Mistake 4: Silently Dropping NaT on Date Parsing

When pd.to_datetime(..., errors="coerce") fails to parse a date, it returns NaT without any warning. If you have 10,000 rows and 200 of them silently became NaT, you won't notice until a groupby or filter produces wrong counts.

# After parsing, always check how many NaTs you created
nat_count = df["sale_date"].isna().sum()
if nat_count > 0:
    print(f"Warning: {nat_count} dates failed to parse")
    print(df[df["sale_date"].isna()]["sale_date_raw"].value_counts())

Mistake 5: Not Handling the String "NaN"

Excel and CSV exports frequently write literal "NaN", "N/A", "null", "None", "--", and "-" as text. pandas won't treat these as missing values by default. The safest approach is to handle them explicitly in your parsing functions, or pass na_values to pd.read_csv() at load time:

df = pd.read_csv(
    "sales_export.csv",
    na_values=["NaN", "N/A", "null", "None", "--", "-", ""],
    keep_default_na=True,
)

Mistake 6: Forgetting That apply() Is Slow at Scale

For small datasets (under ~100k rows), apply() with a custom function is perfectly fine. For large exports, vectorized operations are significantly faster. See Writing Fast pandas Code: Vectorization Instead of apply and Loops for how to rewrite apply() logic as vectorized operations.

For the parse_currency function, the vectorized equivalent would be:

# Vectorized version (faster for large DataFrames)
df["revenue"] = (
    df["revenue"]
    .str.replace(r"[$,\s]", "", regex=True)
    .pipe(pd.to_numeric, errors="coerce")
)

This single chain is significantly faster than calling apply(parse_currency) row by row on a 500k-row export.

---

Real-World Extension: Plugging Into a Larger Pipeline

The clean_sales_pipeline() function you built here is a self-contained transform step. In a production context, it slots naturally into a larger ETL pattern:

def run_pipeline(input_path: str, output_path: str) -> None:
    """Extract → Transform → Load for sales transaction data."""
    # Extract
    raw = pd.read_csv(
        input_path,
        na_values=["NaN", "N/A", "null", "None", "--", ""],
        keep_default_na=True,
    )
    print(f"Loaded {len(raw):,} rows from {input_path}")

    # Transform
    clean = clean_sales_pipeline(raw)
    validate_clean_df(clean)
    print(f"Cleaning complete. {clean['transaction_id'].nunique():,} unique transactions.")

    # Load
    clean.to_csv(output_path, index=False)
    print(f"Saved to {output_path}")


run_pipeline("data/sales_export_raw.csv", "data/sales_export_clean.csv")

For a full treatment of building reusable ETL patterns like this, including handling multiple source files and incremental loads, see Building a Reusable ETL Pipeline in pandas: Extract, Transform, and Load Data from Multiple Sources into a Clean, Analysis-Ready Output.

Once the data is clean, you can immediately feed it into groupby aggregations by region, quarter, or rep. For that next step, Grouping and Aggregating in pandas: groupby as the PivotTable Replacement picks up exactly where this lesson leaves off.

---

Summary & Next Steps

You've built a complete, multi-stage data cleaning pipeline that handles the three hardest categories of column messiness in real-world exports:

• String normalization: stripping, casing, regex substitution, mapping variants to canonical values
• Date parsing: handling mixed formats, ordinal suffixes, timezone-naive timestamps, and deriving analytical columns
• Numeric extraction: currency strings, percentage fields, and mixed text/numeric columns

The key architectural principle you applied throughout: clean systematically, in stages, with validation checkpoints at the end. Each stage produces a cleaner intermediate state that the next stage can rely on.

From here, the natural next steps are:

1. Aggregation and reporting: Your clean DataFrame is ready for Grouping and Aggregating in pandas: groupby as the PivotTable Replacement — slicing revenue by region, quarter, and product tier is now trivial.
2. Scaling the pipeline: If your source files grow to millions of rows, revisit the apply() calls using the vectorization patterns in Writing Fast pandas Code: Vectorization Instead of apply and Loops.
3. Encapsulating into a module: When this pipeline needs to be shared with teammates or scheduled to run automatically, structuring it as a proper Python module is the right move — covered in Structuring a Reusable Data Analysis Project: Functions, Modules, Notebooks, and Scripts.

Clean data is the foundation. Everything else — the groupbys, the pivots, the charts, the reports — depends on getting this stage right.