Automating Structured Data Extraction from Web Tables and Dynamic Grids in Power Automate Desktop: Scraping, Paginating, and Writing Results to Excel Without Manual Copying

Introduction

You've been asked to pull a month's worth of vendor transaction records from your company's procurement portal. The data lives in a paginated HTML table — 47 pages of it — and finance needs it in Excel by end of day. Your options are: spend six hours copying and pasting 940 rows by hand, ask IT to build an API integration that won't arrive until next quarter, or spend 90 minutes building a desktop flow that does it automatically every time. If you're reading this lesson, you're choosing option three.

Web table extraction in Power Automate Desktop sits at the intersection of browser automation and structured data handling. The built-in "Extract data from web page" action can pull an entire HTML table in a single gesture when the page cooperates — but real-world portals rarely cooperate. They use JavaScript-rendered grids, multi-page navigation with dynamic button states, session timeouts, loading spinners that delay rendering, and column headers that change based on filters. This lesson teaches you to handle all of that: not just the happy path, but the real production scenario where things break in interesting ways.

By the end of this lesson, you'll have a working desktop flow that launches a browser, extracts tabular data across multiple pages, handles loading delays and pagination controls, accumulates all results into a structured data table, and writes clean output to Excel — with no manual copying at any stage.

What you'll learn:

• How Power Automate Desktop's web extraction engine works and when to use it versus manual scraping with CSS selectors
• How to set up a custom extraction pattern for complex tables with non-standard HTML structure
• How to detect and navigate pagination controls reliably, including "Next" buttons, numbered pages, and load-more patterns
• How to accumulate multi-page results in a DataTable variable and write them to Excel with proper headers
• How to handle dynamic grids where rows render after JavaScript execution, including wait strategies and fallback approaches

---

Prerequisites

This lesson assumes you're already comfortable with the basics. Specifically, you should know how to:

• Launch browsers and navigate to URLs in Power Automate Desktop (covered in Web Automation in Power Automate Desktop: Browser Actions, Form Filling, and Data Extraction)
• Work with variables, lists, and DataTable types (covered in Variables, Lists, and Data Tables in Power Automate Desktop: A Complete Practitioner's Guide)
• Write extracted data to Excel workbooks (Automating Excel with Power Automate Desktop: Reading, Writing, and Running Macros)
• Build stable UI selectors that survive minor page changes (UI Elements and Selectors in Power Automate Desktop: Building Automations That Don't Break)

You should have Power Automate Desktop installed and be able to create and run desktop flows. If you're starting from scratch, see Getting Started with Power Automate Desktop: Installing, Recording, and Running Your First Desktop Flow first.

---

Understanding How PAD Extracts Web Table Data

Before writing a single action, it's worth understanding what actually happens under the hood when Power Automate Desktop extracts data from a web page. This will save you significant debugging time later.

PAD's "Extract data from web page" action works by injecting a JavaScript snippet into the currently loaded page via the browser extension. That script locates the HTML elements you've identified, reads their text content or attribute values, and returns the data as a structured object to the desktop flow runtime. The result is stored as a DataTable variable.

The important implication: the extraction happens against the current DOM state at the moment the action executes. If your table is rendered by a JavaScript framework (React, Angular, Vue), the data only exists in the DOM after the JavaScript has run and the rendering is complete. If PAD fires the extraction before rendering finishes, you'll get an empty table or partial rows. This is the single most common reason web extraction fails in production.

HTML tables with a proper <table>, <thead>, <tr>, and <td> structure are the easiest case. PAD can auto-detect these and offer to extract "the entire table" with a single click during the setup wizard. But many modern applications — including SAP Fiori, Salesforce, ServiceNow, and custom React portals — render their grid data as <div> elements styled to look like tables. PAD's auto-detection won't help there; you have to manually specify the extraction pattern.

The Two Extraction Modes

PAD gives you two ways to extract web data:

Mode 1 — Guided extraction wizard: You right-click elements in the live browser during flow design, and PAD infers the extraction pattern automatically. Best for standard HTML tables on relatively static pages.

Mode 2 — Custom extraction with JavaScript: You write your own extraction logic using the "Run JavaScript on web page" action, then parse the result. Best for complex grids, dynamic content, or when you need precise control over what gets extracted.

This lesson covers both modes, because you'll encounter both in real projects.

---

Setting Up the Extraction: Standard HTML Tables

Let's start with a concrete scenario: you're extracting purchase order data from a procurement portal that uses a standard HTML table. The table has columns for PO Number, Vendor Name, Amount, Date, and Status. There are 12 pages of results.

Step 1: Launch the Browser and Navigate

Add a "Launch new Microsoft Edge" action (or Chrome — the setup is identical). Set the URL to your portal's data page. Store the browser instance in a variable called ProcurementBrowser.

Action: Launch new Microsoft Edge
URL: https://procurement.internal.company.com/orders?status=all&daterange=thismonth
Window state: Maximized
Store in: ProcurementBrowser

After navigation, add a "Wait for web page to load" action targeting the same browser instance. This handles basic page load, but it won't wait for JavaScript-rendered content — we'll address that in a moment.

Step 2: Define the Extraction Pattern

With the flow paused at the "Extract data from web page" action setup, PAD opens the live browser and puts you in "extraction mode." You'll see a highlight overlay as you hover over elements.

Here's the procedure for a standard table:

1. Hover over any data cell in the first row. PAD highlights that single cell.
2. Right-click and choose "Extract entire HTML table." PAD detects the <table> ancestor and offers to grab the whole thing.
3. PAD shows you a preview of the extracted data. Verify that all columns appear correctly and the headers are included.
4. Click "Finish."

PAD generates the extraction configuration and stores the result in a DataTable variable — call it CurrentPageData.

Step 3: Initialize Your Accumulator DataTable

Before you start looping over pages, create an empty DataTable variable to accumulate all results. Add a "Create new data table" action — but don't use the empty template. Instead, add a "Set variable" action and assign it an empty DataTable using the expression:

%{ ^['PO Number', 'Vendor Name', 'Amount', 'Date', 'Status'] }%

This creates a DataTable with your column headers already defined, called AllOrdersData. Every page's results will be merged into this table before you move to the next page.

Step 4: Merge Page Results into the Accumulator

After extracting CurrentPageData, add a loop: "For each row in CurrentPageData." Inside the loop, add a "Insert row into data table" action that targets AllOrdersData, mapping each column by index or name.

For each CurrentRow in CurrentPageData
    Insert row into data table AllOrdersData
    Values: [CurrentRow['PO Number'], CurrentRow['Vendor Name'], 
             CurrentRow['Amount'], CurrentRow['Date'], CurrentRow['Status']]
End

This approach — extract per page, loop and insert, then paginate — is more reliable than trying to concatenate DataTables directly, because it gives you a clear place to add per-row validation logic later.

---

Building the Pagination Loop

This is where most automations fall apart. Pagination looks simple but has more edge cases than almost any other part of web scraping. Let's handle it properly.

Identifying Your Pagination Pattern

Before writing the loop, you need to know which pattern your target site uses:

Pattern A — Next button with disabled state: A "Next >" button that becomes disabled or disappears on the last page. This is the most common and easiest to handle.

Pattern B — Numbered page links: Clickable page numbers (1, 2, 3...) with the current page highlighted. You need to calculate how many pages exist or detect the "last" state.

Pattern C — Load More button: A single button that appends more rows to the existing table without navigating. Requires a different loop structure.

Pattern D — Infinite scroll: New rows appear as you scroll down. Requires simulated scroll actions and is outside the scope of this lesson (see Automating Multi-Step Web Scraping and Pagination Handling in Power Automate Desktop for that pattern).

For this lesson we'll build the complete Pattern A flow — it covers the vast majority of enterprise portals — and then show how to adapt for Pattern B.

Pattern A: Next Button Pagination Loop

The outer structure of your flow should look like this:

Set variable AllOrdersData = empty DataTable (with headers)
Set variable MorePages = True
Set variable CurrentPage = 1

Loop while MorePages = True
    
    [Wait for table to be ready - see below]
    Extract data from web page → CurrentPageData
    
    For each CurrentRow in CurrentPageData
        Insert row into AllOrdersData [all column values]
    End
    
    If web element exists [Next button, not disabled]
        Click web link/button [Next button]
        Set variable CurrentPage = CurrentPage + 1
    Else
        Set variable MorePages = False
    End
    
End Loop

The critical detail is the "If web element exists" condition. You need two separate UI elements defined:

• NextButton_Active: The Next button in its clickable state. Your selector should target something like [data-testid='pagination-next']:not([disabled]) or an element with a class like btn-next that doesn't have a disabled class.
• You don't need a separate element for the disabled state — you just check if the active one exists.

Building Robust Selectors for Pagination Controls

In PAD's UI element editor, build your Next button selector using CSS selectors. A robust selector for a typical pagination control looks like:

css: button[aria-label='Next page']:not([disabled])

Or for sites using class-based disabled states:

css: a.pagination-next:not(.disabled):not(.inactive)

The :not() pseudo-class is your best friend here. It makes the selector fail (return "element not found") exactly when you want the loop to stop.

For more on building selectors that survive site updates, see UI Elements and Selectors in Power Automate Desktop: Building Automations That Don't Break.

Pattern B: Numbered Page Navigation

When you can see the total number of pages (common in portals that show "Page 1 of 47" text), you can use a simpler For loop:

Extract text from web element [page-count-element] → TotalPagesText
# TotalPagesText = "Page 1 of 47"
Set variable TotalPages = TotalPagesText after last ' of '  # → "47"
Convert text to number: TotalPages → TotalPagesNum

For LoopIndex from 1 to TotalPagesNum
    [Extract current page data and merge]
    
    If LoopIndex < TotalPagesNum
        Click web element [Next button]
        [Wait for page load]
    End
End For

This is cleaner and more predictable than the while loop because you know exactly how many iterations to expect. If the run takes significantly longer than TotalPagesNum * avg_page_time, you know something went wrong.

---

Handling Dynamic Grids and JavaScript-Rendered Tables

Now we get to the harder problem. Many modern enterprise portals — Salesforce reports, ServiceNow tables, custom React dashboards — render their data grids as <div> structures that look like tables visually but aren't <table> elements in HTML. PAD's auto-detection won't recognize these.

Detecting Whether You Have a True HTML Table

Before you spend time troubleshooting, confirm the structure. In Chrome or Edge DevTools (F12), inspect the data area. If you see <table> → <tbody> → <tr> → <td>, you have a standard HTML table. If you see something like:

<div class="grid-container">
  <div class="grid-row" role="row">
    <div class="grid-cell" role="gridcell">PO-2024-00891</div>
    <div class="grid-cell" role="gridcell">Acme Supplies Inc.</div>
    ...
  </div>
</div>

You have a div-based grid, and you'll need a different approach.

Wait Strategy for JavaScript-Rendered Content

For both table types, you need to wait for the content to be present before extracting. The naive approach — "Wait 3 seconds" — is fragile and slow. Use the "Wait for web page element" action instead.

Wait for web element [FirstDataRow] to exist
Timeout: 30 seconds
On timeout: Go to error handling block

Define FirstDataRow as the first row of your data (not the header). When this element exists, you know the table has rendered. This is faster than a fixed wait and more reliable.

Extracting Div-Based Grids with JavaScript

For div grids, the most reliable approach is to write a small JavaScript extraction function using PAD's "Run JavaScript on web page" action. The script reads the DOM directly and returns JSON-formatted data.

var rows = document.querySelectorAll('.grid-row[role="row"]');
var result = [];

rows.forEach(function(row) {
    var cells = row.querySelectorAll('.grid-cell[role="gridcell"]');
    if (cells.length >= 5) {
        result.push({
            'PO Number': cells[0].innerText.trim(),
            'Vendor Name': cells[1].innerText.trim(),
            'Amount': cells[2].innerText.trim(),
            'Date': cells[3].innerText.trim(),
            'Status': cells[4].innerText.trim()
        });
    }
});

return JSON.stringify(result);

In PAD, configure the action:

Action: Run JavaScript on web page
Browser instance: ProcurementBrowser
JavaScript: [the script above]
Result stored in: JSResult

JSResult comes back as a text string containing JSON. You then need to parse it into usable data. Use a "Run JavaScript on web page" pass (or a PowerShell script via the scripting action) to convert the JSON to delimited text, or use PAD's JSON parsing via the "Convert JSON to custom object" action, then loop through the resulting list.

Here's the PAD action chain for parsing:

Convert JSON to custom object: JSResult → ParsedData
# ParsedData is now a List of custom objects

For each RecordObj in ParsedData
    Insert row into AllOrdersData
    Values: [RecordObj['PO Number'], RecordObj['Vendor Name'],
             RecordObj['Amount'], RecordObj['Date'], RecordObj['Status']]
End

Handling Tables Where Columns Shift Between Pages

Some poorly-designed portals dynamically change which columns are visible based on filters, user preferences, or screen width. If you hardcode column indices (cells[0], cells[1], etc.), your extraction breaks silently — you get data in the wrong columns with no error.

The safer approach: extract column headers first, then build a mapping.

var headers = document.querySelectorAll('.grid-header-cell');
var headerMap = {};
headers.forEach(function(h, index) {
    headerMap[h.innerText.trim()] = index;
});

var rows = document.querySelectorAll('.grid-row[role="row"]');
var result = [];

rows.forEach(function(row) {
    var cells = row.querySelectorAll('.grid-cell[role="gridcell"]');
    result.push({
        'PO Number': cells[headerMap['PO #']] ? cells[headerMap['PO #']].innerText.trim() : '',
        'Vendor Name': cells[headerMap['Vendor']] ? cells[headerMap['Vendor']].innerText.trim() : '',
        'Amount': cells[headerMap['Total Amount']] ? cells[headerMap['Total Amount']].innerText.trim() : '',
        'Date': cells[headerMap['Order Date']] ? cells[headerMap['Order Date']].innerText.trim() : '',
        'Status': cells[headerMap['Status']] ? cells[headerMap['Status']].innerText.trim() : ''
    });
});

return JSON.stringify(result);

This script is defensive by design: it maps column position by header label, not by index, so rearranged columns don't corrupt your data.

---

Writing the Final Data to Excel

Once your loop finishes and AllOrdersData contains all rows from all pages, writing to Excel is straightforward — but a few details make the difference between a clean output and a messy one.

Setting Up the Excel Workbook

Use "Launch Excel" to open either an existing template file or a new workbook. For a recurring report, using a template (an .xlsx file with headers and formatting pre-defined) is cleaner than recreating the header row every time.

Action: Launch Excel
Launch: With a blank document
[Or: Open the following document → C:\Reports\PO_Template.xlsx]
Make visible: True
Store in: ExcelInstance

If using a blank document, write your headers first:

Action: Write to Excel worksheet
Excel instance: ExcelInstance
Value to write: 'PO Number'
Column: 1, Row: 1

[Repeat for each header column]

Or more efficiently, write the header row as a single operation using a list:

Set variable Headers = ['PO Number', 'Vendor Name', 'Amount', 'Date', 'Status']

For ColumnIndex from 0 to Headers.Count - 1
    Write to Excel worksheet
    Value: Headers[ColumnIndex]
    Column: ColumnIndex + 1
    Row: 1
End

Writing the DataTable to Excel

PAD has a dedicated "Write to Excel worksheet" mode that accepts a DataTable directly and writes all rows starting at a specified cell. This is dramatically faster than row-by-row writing:

Action: Write to Excel worksheet
Excel instance: ExcelInstance
Value to write: %AllOrdersData%
Write mode: On specified cell
Starting column: A
Starting row: 2  (row 1 is headers)

When you pass a DataTable to this action, PAD writes all rows and columns automatically. The column order in Excel matches the column order in your DataTable.

For more advanced scenarios — writing to named ranges, applying formatting, or working across multiple sheets — see Working with Excel Ranges, Sheets, and Named Cells in Power Automate Desktop.

Saving and Closing

Always save explicitly before closing — PAD won't prompt you:

Action: Save Excel
Excel instance: ExcelInstance
Save mode: Save document as
Document path: C:\Reports\PO_Extract_%CurrentDateTime%dd-MM-yyyy%%.xlsx

Action: Close Excel
Excel instance: ExcelInstance
Before closing: Do not save document  (already saved above)

Using a timestamp in the filename prevents overwrites and creates a natural audit trail. If this flow runs daily, you'll automatically have a dated archive.

---

Adding Error Handling and Stability

A flow that works once in testing isn't production-ready. Here's what to add to make it resilient.

Wrap the Pagination Loop in Error Handling

Wrap your entire pagination loop in an "On block error" structure. If a network timeout, session expiration, or unexpected popup interrupts extraction mid-way, you want to save whatever data you've already collected rather than losing everything.

For a full treatment of PAD error handling patterns, see Error Handling in Desktop Flows: On Block Error, Retry Policies, and Recovery Screenshots.

On block error:
    If AllOrdersData.RowCount > 0
        Write AllOrdersData to Excel (partial results)
        Save Excel with filename including "_PARTIAL"
        Send notification email: "Extraction interrupted at page %CurrentPage%"
    End
    Take screenshot
    Stop flow with error message
End on block error

Handle Session Timeouts

Procurement portals often have aggressive session timeouts. If your extraction takes 20+ minutes (large datasets, slow pages), you may get logged out mid-run. Add a session check action between pages:

If web element exists [LoginButton]
    # We've been logged out — re-authenticate
    Click [LoginButton]
    Set text [UsernameField] to %PortalUsername%
    Set text [PasswordField] to %PortalPassword%
    Click [SubmitButton]
    Wait for web element [FirstDataRow] to exist
    # Navigate back to the correct page number
    Navigate to [PageURL with CurrentPage parameter]
End

For credential management, avoid hardcoding usernames and passwords in your flow. Use sensitive variables or Azure Key Vault — see Handling Credentials Securely in Desktop Flows: Sensitive Variables and Azure Key Vault.

Add a Row Count Sanity Check

After extraction on each page, verify you got a reasonable number of rows:

If CurrentPageData.RowCount = 0 AND CurrentPage < TotalPagesNum
    # This shouldn't happen — page loaded but extracted nothing
    Wait 5 seconds
    Extract data from web page → CurrentPageData  (retry once)
    If CurrentPageData.RowCount = 0
        Log warning: "Zero rows on page %CurrentPage% — possible render failure"
    End
End

This catches the case where extraction fires before rendering completes, retries once, and logs the anomaly without crashing the entire flow.

---

Hands-On Exercise: Build the Complete PO Extraction Flow

Let's put it all together. Here's the complete flow structure to build as your hands-on exercise. If you don't have a real procurement portal to test against, you can use any paginated public dataset — try a government data portal, a public financial database, or even a Wikipedia article with a sortable table.

Exercise objective: Extract a multi-page HTML table from a public source, accumulate all pages into a DataTable, and write the result to a timestamped Excel file.

Suggested test site: The OSHA enforcement data browser at https://www.osha.gov/ords/imis/establishment.html has paginated inspection tables — it's a real government dataset with real pagination. Don't hammer it with automated requests; a single educational run is fine.

Flow outline to build:

1. Initialize variables: AllData (empty DataTable), MorePages = True, CurrentPage = 1
2. Launch Edge, navigate to the target URL
3. Wait for first data row to exist
4. Begin While loop (MorePages = True)
5. Extract current page data → CurrentPageData
6. Validate row count (log warning if 0)
7. Loop through CurrentPageData, insert each row into AllData
8. Check if Next button exists (not disabled)
9. If yes: click Next, increment CurrentPage, wait for new data to load
10. If no: set MorePages = False
11. End While loop
12. Launch Excel (blank document)
13. Write header row to row 1
14. Write AllData DataTable starting at row 2
15. Save with timestamped filename
16. Close Excel
17. Display message: "Extracted %AllData.RowCount% rows across %CurrentPage% pages"

Checkpoint questions to validate your build:

• Does the flow stop gracefully on the last page without throwing an error?
• If you interrupt it on page 3 and re-run, does it start fresh from page 1 (as expected)?
• Does the Excel file have correct headers in row 1 and data starting in row 2?
• Is the row count in the final message accurate?

---

Common Mistakes & Troubleshooting

"My extraction returns an empty DataTable even though I can see data on the page"

Most likely cause: JavaScript rendering delay. The action fired before the table populated.

Fix: Add a "Wait for web element" action before the extraction, targeting a cell that you know will contain data. Use a 30-second timeout. If the wait itself times out consistently, your selector for that cell is wrong — verify it in DevTools.

"The loop keeps clicking Next forever and never stops"

Most likely cause: Your "Next button active" selector matches the button even in its disabled state on the last page.

Fix: Open DevTools on the last page and inspect the Next button's HTML. Find what changes between enabled and disabled states (usually a CSS class or the disabled attribute). Update your selector to include a :not([disabled]) or :not(.disabled) exclusion.

"My DataTable has data from page 1 in all columns, but pages 2+ have shifted columns"

Most likely cause: The target site renders different column sets on different pages (this happens with filtered portals), or your column-index-based extraction assumes a fixed structure.

Fix: Switch to the header-mapping JavaScript approach described in the "Columns That Shift" section above.

"The flow crashes when clicking the Next button because the page shows a confirmation dialog"

Most likely cause: Some portals inject cookie consent dialogs, session renewal prompts, or advertisement overlays after a few pages.

Fix: Add dialog detection before each pagination click. For handling browser-level dialogs and popup windows in desktop flows, the pattern is covered in Automating Windows System Dialogs and Pop-Up Handling in Power Automate Desktop.

"The Excel file opens but has no data — it's just headers"

Most likely cause: AllData.RowCount is 0, meaning the merge loop never ran. Usually caused by the DataTable column names not matching the extraction result column names, so the insert-row action silently failed.

Fix: Add a "Display message" after the first page extraction that shows %CurrentPageData.RowCount% and %CurrentPageData.Columns%. Verify the column names exactly match what you're referencing in the insert-row action.

"Extraction is very slow — 47 pages takes over an hour"

Most likely cause: Fixed wait times are too conservative, or page navigation is slow due to network latency.

Fix: Replace "Wait 3 seconds" with dynamic waits ("Wait for web element to exist"). Also check whether the portal supports URL-based pagination — if you can navigate directly to ?page=2, ?page=3, etc., you eliminate some of the UI clicking overhead. URL-based pagination is also more resilient than clicking UI elements.

---

Summary & Next Steps

You now have a complete, production-viable approach to structured web table extraction in Power Automate Desktop. The key principles to carry forward:

• Wait for rendering, not for time. Element-based waits are faster and more reliable than fixed delays.
• Accumulate first, write once. Build your complete DataTable across all pages before touching Excel. It's faster and makes error recovery cleaner.
• Validate at each page. A zero-row page mid-extraction is a symptom, not a normal state. Catch it explicitly.
• Let selectors do the pagination logic. A selector that only matches the enabled Next button is simpler and more reliable than checking button state separately.
• Use JavaScript extraction for div grids. PAD's built-in wizard works beautifully for real HTML tables; for everything else, a small JS function gives you complete control.

Where to go next:

If your target data lives across multiple applications — for example, you scrape from a portal but also need to reconcile against data in a Windows desktop app — the multi-application workflow patterns in Automating Multi-Application Workflows in Power Automate Desktop: Transferring Data Between Windows Apps, Web Browsers, and Excel in a Single Desktop Flow extend directly from what you've learned here.

If you want this flow to run on a schedule without someone sitting at the machine, the path forward is unattended automation — your flow is already structured correctly for it. The next configuration steps are covered in Attended vs Unattended RPA: Choosing a Run Mode and Configuring Machines in Power Automate.

And if you want to wrap this extraction flow into a larger orchestration — triggered by a cloud flow, passing the output file path back as a return value — see Triggering Desktop Flows from Cloud Flows: Passing Inputs and Returning Outputs.

Web table extraction is one of those automation skills that compounds: once you've built one paginated extraction flow, the next one takes a fraction of the time. The structure is always the same — wait, extract, merge, paginate, write. What changes is the HTML and the edge cases. You now know how to handle both.