Selecting and Filtering Data in pandas: loc, iloc, and Boolean Masks

Introduction

Imagine you've just loaded a sales dataset with 50,000 rows into pandas — customer IDs, regions, product categories, revenue figures, dates. Your manager wants to see only the transactions from the Northeast region where revenue exceeded $10,000. Your first instinct might be to scroll through the spreadsheet looking for those rows, or to write a SQL WHERE clause. In pandas, this kind of filtering is not only possible but fast, flexible, and composable — once you understand the three core tools: loc, iloc, and boolean masks.

These three tools are the backbone of nearly every pandas workflow. Whether you're cleaning data, building reports, or feeding subsets into a machine learning model, you'll use them constantly. The challenge is that beginners often mix them up or reach for the wrong one, leading to confusing errors or — worse — silently incorrect results.

By the end of this lesson, you'll be able to confidently select any row or column (or combination of both) from a DataFrame, filter rows based on one or more conditions, and understand exactly why each tool works the way it does.

What you'll learn:

• How pandas indexes work and why they matter for selection
• How to select rows and columns by label using loc
• How to select rows and columns by position using iloc
• How to build boolean masks to filter rows by conditions
• How to combine multiple conditions with AND, OR, and NOT logic

---

Prerequisites

This lesson assumes you have pandas installed and know how to load a DataFrame. If you're starting from scratch, work through Setting Up Python for Data Analysis: Python, VS Code, Jupyter, and Virtual Environments and Your First pandas DataFrame: Loading CSV and Excel Files and Exploring Data first. You should also be comfortable with Python lists and basic syntax — if not, Python Basics for Excel Users: Variables, Lists, Dictionaries, and Loops will get you there.

---

The Setup: Our Working Dataset

Let's build a small but realistic dataset we'll use throughout this lesson — a regional sales table for a hypothetical software company.

import pandas as pd

data = {
    "order_id": [1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008],
    "region": ["Northeast", "Southeast", "Northeast", "West", "West", "Southeast", "Northeast", "West"],
    "rep": ["Diaz", "Okafor", "Chen", "Diaz", "Okafor", "Chen", "Diaz", "Okafor"],
    "product": ["Enterprise", "Starter", "Enterprise", "Pro", "Enterprise", "Pro", "Starter", "Enterprise"],
    "revenue": [12500, 3200, 18700, 9800, 22100, 7400, 4100, 15600],
    "closed": [True, False, True, True, True, False, False, True],
}

df = pd.DataFrame(data)
print(df)

   order_id     region    rep     product  revenue  closed
0      1001  Northeast   Diaz  Enterprise    12500    True
1      1002  Southeast  Okafor     Starter     3200   False
2      1003  Northeast    Chen  Enterprise    18700    True
3      1004       West    Diaz         Pro     9800    True
4      1005       West  Okafor  Enterprise    22100    True
5      1006  Southeast    Chen         Pro     7400   False
6      1007  Northeast    Diaz     Starter     4100   False
7      1008       West  Okafor  Enterprise    15600    True

Notice those numbers on the left — 0, 1, 2, 3... That's the index. By default, pandas assigns integer row labels starting at zero. The index is not a column; it's more like a row address system. This distinction matters a lot when you start using loc and iloc.

---

Understanding the Index: Your Map to the Data

Before diving into the tools, let's build the right mental model. A pandas DataFrame is essentially a grid with two coordinate systems:

• Labels: The actual names of rows (the index) and columns (column headers like "region", "revenue")
• Positions: The numeric position of a row or column — 0, 1, 2, counting from the top-left

Most of the time, the default index labels happen to look like positions (0, 1, 2...), which trips up beginners. But the index doesn't have to be integers. You might set the order_id as the index:

df_indexed = df.set_index("order_id")
print(df_indexed.head(3))

             region    rep     product  revenue  closed
order_id                                               
1001      Northeast   Diaz  Enterprise    12500    True
1002      Southeast  Okafor     Starter     3200   False
1003      Northeast    Chen  Enterprise    18700    True

Now the row labels are 1001, 1002, 1003 — not positions. loc operates on these labels. iloc always operates on positions (0, 1, 2). Keep that distinction in your head as we continue.

---

Selecting Data with `loc`: Label-Based Access

loc is how you select data by name. You give it row labels and column names, and it hands back exactly what you asked for.

The syntax is: df.loc[row_selector, column_selector]

The comma separates rows (left side) from columns (right side). Let's start simple.

Selecting a single row

# Get the row where the index label is 2
df.loc[2]

order_id    1003
region     Northeast
rep             Chen
product    Enterprise
revenue         18700
closed           True
Name: 2, dtype: object

This returns a Series — a single-dimensional object with column names as its index. Think of it as one horizontal slice of the DataFrame.

Selecting multiple rows

Pass a list of labels to get multiple rows back as a DataFrame:

df.loc[[0, 2, 4]]

   order_id     region    rep     product  revenue  closed
0      1001  Northeast   Diaz  Enterprise    12500    True
2      1003  Northeast    Chen  Enterprise    18700    True
4      1005       West  Okafor  Enterprise    22100    True

Selecting a range of rows

You can use a slice with loc. Here's an important quirk:

df.loc[1:4]  # Returns rows with labels 1, 2, 3, AND 4

Selecting specific columns

Add a column selector after the comma:

# Select rows 0–2, only the 'region' and 'revenue' columns
df.loc[0:2, ["region", "revenue"]]

      region  revenue
0  Northeast    12500
1  Southeast     3200
2  Northeast    18700

Selecting all rows, specific columns

Use a colon : alone to mean "all rows":

df.loc[:, ["rep", "product", "revenue"]]

This is equivalent to df[["rep", "product", "revenue"]] — a common shorthand you'll see in the wild.

---

Selecting Data with `iloc`: Position-Based Access

iloc works exactly like loc in terms of syntax, but the coordinates are always integer positions, regardless of what the index contains.

df.iloc[0]      # First row (position 0)
df.iloc[-1]     # Last row (Python's negative indexing works here)
df.iloc[0:3]    # Rows at positions 0, 1, 2 — NOT including 3

Selecting rows and columns by position

# First 3 rows, first 3 columns
df.iloc[0:3, 0:3]

   order_id     region    rep
0      1001  Northeast   Diaz
1      1002  Southeast  Okafor
2      1003  Northeast    Chen

When to use iloc vs loc

Use iloc when:

• You want the "first N rows" regardless of what the index says
• You're iterating through rows programmatically by position
• You've received a positional row number from some computation

Use loc when:

• You know the actual label (name, ID, date) of the row you want
• You're selecting columns by name (which is almost always)
• You're filtering with boolean masks (more on this in a moment)

---

Boolean Masks: Filtering by Condition

Here's where pandas gets really powerful. A boolean mask is a Series of True/False values — one per row — that tells pandas which rows to keep.

Let's build one:

mask = df["region"] == "Northeast"
print(mask)

0     True
1    False
2     True
3    False
4    False
5    False
6     True
7    False
Name: region, dtype: bool

This Series has the same index as df. Each value answers the question: "Is this row in the Northeast?" Now pass it into loc:

df.loc[mask]

   order_id     region   rep     product  revenue  closed
0      1001  Northeast  Diaz  Enterprise    12500    True
2      1003  Northeast  Chen  Enterprise    18700    True
6      1007  Northeast  Diaz     Starter     4100   False

Only the rows where the mask is True come back. You can write this more concisely by putting the condition directly inside loc:

df.loc[df["region"] == "Northeast"]

This is the same result. The condition df["region"] == "Northeast" evaluates to a boolean mask on the fly, and loc filters with it.

Common comparison operators for masks

df.loc[df["revenue"] > 10000]           # Greater than
df.loc[df["revenue"] <= 9800]           # Less than or equal
df.loc[df["product"] == "Enterprise"]   # Equals
df.loc[df["product"] != "Starter"]      # Not equals
df.loc[df["closed"] == True]            # Boolean column

Using `.isin()` for multiple values

What if you want rows from either Northeast or West? Don't chain multiple == conditions — use .isin():

df.loc[df["region"].isin(["Northeast", "West"])]

   order_id     region    rep     product  revenue  closed
0      1001  Northeast   Diaz  Enterprise    12500    True
2      1003  Northeast    Chen  Enterprise    18700    True
3      1004       West    Diaz         Pro     9800    True
4      1005       West  Okafor  Enterprise    22100    True
6      1007  Northeast    Diaz     Starter     4100   False
7      1008       West  Okafor  Enterprise    15600    True

Using `.str` methods in masks

For text-based filtering, pandas gives you .str accessor methods:

# Find all rows where the rep's name starts with 'D'
df.loc[df["rep"].str.startswith("D")]

# Case-insensitive contains
df.loc[df["product"].str.lower().str.contains("enter")]

---

Combining Conditions: AND, OR, NOT

Here's where most beginners hit a wall. In Python, the keywords and, or, and not don't work element-wise on Series. You need bitwise operators instead:

Logic	Python keyword	pandas operator
AND	and	&
OR	or	|
NOT	not	~

AND — both conditions must be true

Find closed deals in the Northeast:

df.loc[(df["region"] == "Northeast") & (df["closed"] == True)]

   order_id     region   rep     product  revenue  closed
0      1001  Northeast  Diaz  Enterprise    12500    True
2      1003  Northeast  Chen  Enterprise    18700    True

The parentheses around each condition are required. Python's operator precedence would otherwise try to evaluate "Northeast") & (df["closed"] as a single operation, which fails.

OR — either condition can be true

Find deals that are either high revenue OR already closed:

df.loc[(df["revenue"] > 15000) | (df["closed"] == True)]

NOT — negate a condition

Find all deals that are not closed:

df.loc[~df["closed"]]

Chaining complex conditions

Let's answer the original problem from the introduction — Northeast deals over $10,000:

northeast_big = df.loc[
    (df["region"] == "Northeast") & (df["revenue"] > 10000)
]
print(northeast_big)

   order_id     region   rep     product  revenue  closed
0      1001  Northeast  Diaz  Enterprise    12500    True
2      1003  Northeast  Chen  Enterprise    18700    True

You can keep adding conditions with more & or | operators, always wrapping each sub-condition in parentheses.

---

Selecting Columns Alongside Filtered Rows

So far we've been returning all columns. Often you want to filter rows AND narrow the columns at the same time. Just add the column selector after the comma in loc:

# Northeast deals over $10k — show only rep, product, and revenue
df.loc[
    (df["region"] == "Northeast") & (df["revenue"] > 10000),
    ["rep", "product", "revenue"]
]

     rep     product  revenue
0   Diaz  Enterprise    12500
2   Chen  Enterprise    18700

This is one of the most common patterns in real pandas work — filter rows by condition, select specific columns for output. Get comfortable writing it.

---

The Bracket Shorthand (and When Not to Use It)

You'll often see filtering written without loc:

df[df["region"] == "Northeast"]

This works for simple row filtering. But it's a shorthand with limitations:

• You can't select specific columns at the same time (without chaining, which causes other problems)
• It can trigger a SettingWithCopyWarning if you try to modify the result

---

Hands-On Exercise

Work through these tasks using the df DataFrame we built at the top of the lesson.

Task 1: Use iloc to select the last 3 rows of the DataFrame and only the first 4 columns.

Task 2: Use loc to retrieve only the rows with index labels 3, 5, and 7.

Task 3: Create a boolean mask for all rows where product is "Enterprise" and store it in a variable called enterprise_mask. Then use it to filter the DataFrame.

Task 4: Find all deals where revenue is between $5,000 and $15,000 (inclusive on both ends). Return only the order_id, region, and revenue columns.

Task 5: Find all deals that are either in the West region or have not been closed yet. How many rows does this return?

Task 6 (challenge): Filter for closed Enterprise deals with revenue above $12,000 and display only the rep name and revenue.

---

Expected output for Task 6:

     rep  revenue
0   Diaz    12500
2   Chen    18700
4  Okafor   22100
7  Okafor   15600

---

Common Mistakes & Troubleshooting

Mistake 1: Confusing loc and iloc when the index isn't 0-based

df_indexed = df.set_index("order_id")

# This will raise a KeyError — there's no label "0"
df_indexed.loc[0]

# This works — position 0 exists regardless of labels
df_indexed.iloc[0]

If you get a KeyError with loc, check whether your index actually contains the label you're requesting.

Mistake 2: Forgetting parentheses in combined conditions

# BROKEN — operator precedence causes a TypeError
df.loc[df["region"] == "Northeast" & df["revenue"] > 10000]

# CORRECT — each condition wrapped in parentheses
df.loc[(df["region"] == "Northeast") & (df["revenue"] > 10000)]

Mistake 3: Using Python's `and`/`or` instead of `&`/`|`

# BROKEN — raises ValueError
df.loc[df["region"] == "Northeast" and df["revenue"] > 10000]

# CORRECT
df.loc[(df["region"] == "Northeast") & (df["revenue"] > 10000)]

Mistake 4: Modifying a filtered DataFrame and getting a warning

northeast = df[df["region"] == "Northeast"]
northeast["revenue"] = northeast["revenue"] * 1.1  # SettingWithCopyWarning!

When you want to modify filtered data, use .copy() to make an explicit copy:

northeast = df[df["region"] == "Northeast"].copy()
northeast["revenue"] = northeast["revenue"] * 1.1  # Safe

Mistake 5: Using `==` to compare against None/NaN

# BROKEN — this doesn't work for missing values
df.loc[df["revenue"] == None]

# CORRECT — use pandas null-checking methods
df.loc[df["revenue"].isna()]
df.loc[df["revenue"].notna()]

---

Summary & Next Steps

Let's consolidate what you've learned:

• iloc selects by integer position. Slicing is exclusive on the right (like standard Python). Use it when you need positional access.
• loc selects by label. Slicing is inclusive on both ends. Use it for named rows, columns, and boolean filtering.
• Boolean masks are Series of True/False values. Build them with comparison operators, .isin(), .str methods, and more. Pass them into loc to filter rows.
• Combine conditions with & (AND), | (OR), and ~ (NOT). Always wrap each condition in parentheses.
• Select columns alongside filtered rows using the two-argument form: df.loc[row_condition, column_list].

These tools are foundational — you'll use them in every subsequent pandas task: cleaning bad data, aggregating subsets, preparing inputs for charts, and building automated reports. Once the syntax feels natural, you'll find yourself expressing in two lines of pandas what might have taken a complex SQL subquery or multiple Excel filter steps.

From here, a natural next step is learning how to reshape and aggregate your filtered data — grouping by region, summing revenue, and summarizing across dimensions. That's where pandas' groupby machinery becomes your best friend.