Wicked Smart Data
LearnInsightsAboutContact
Sign InLet's Build
LearnInsightsAboutContact
Sign InLet's Build
Wicked Smart Data

Intelligence, automation, and expert execution — plus an elite library of free knowledge. We turn complexity into competitive advantage.

Start a conversation

Platform

  • Learning Paths
  • Insights
  • RSS Feed

Company

  • About
  • Contact
  • Work With Us

Legal

  • Privacy Policy
  • Terms of Service

© 2026 Wicked Smart Data. All rights reserved.

Intelligence · Automation · Advantage

All Insights
Power Automate

Monitoring and Troubleshooting Desktop Flow Runs at Scale

When you're running dozens of unattended bots across a machine fleet, "check the portal" stops being a monitoring strategy. Learn how to build production-grade observability for Power Automate desktop flows — from structured telemetry and automated alerting to systematic diagnostic frameworks for the five most common failure categories at scale.

🔥 Expert35 min readSep 22, 2026Updated Sep 22, 2026
Monitoring and Troubleshooting Desktop Flow Runs at Scale
On this page
  • Introduction
  • Prerequisites
  • Understanding the Monitoring Ecosystem
  • Layer 1: The Power Automate Portal Run History
  • Layer 2: Desktop Flow Run Details in the Monitor Section
  • Layer 3: Dataverse Tables
  • Layer 4: Custom Logging Within Flows
  • Designing Flows for Observability
  • The Minimum Viable Log Record
  • Structured Error Payloads
  • Correlation IDs
  • Building an Automated Monitoring Cloud Flow
  • The Architecture
  • Querying the flowsession Table
  • Building the Alert Message
  • Systematic Diagnostic Framework
  • Category 1: Machine and Connectivity Failures
  • Category 2: Application State Failures
  • Category 3: Data Quality Failures
  • Category 4: Performance Degradation and Timeouts
  • Category 5: Concurrency and State Conflicts
  • Building a Production Operations Dashboard
  • Power BI Dashboard Architecture
  • Operational Runbook Integration
  • Advanced Instrumentation Patterns
  • Instrumentation Subflow Pattern
  • Performance Timing Pattern
  • Canary Run Pattern
  • Scaling Machine Fleet Management
  • Machine Health Monitoring
  • Queue Depth and Throughput Monitoring
  • Scaling Out vs. Scaling Up
  • Hands-On Exercise
  • Common Mistakes & Troubleshooting
  • "I see runs failing but there's no error detail"
  • "Runs show as 'Running' in the portal but the machines are idle"
  • "My monitoring cloud flow fires too many alerts and people start ignoring them"
  • "The flow fails differently on different machines"
  • "Everything worked for weeks and then suddenly everything started failing simultaneously"
  • Summary & Next Steps
  • Monitoring and Troubleshooting Desktop Flow Runs at Scale

    Introduction

    It's 7:47 AM on a Tuesday. Your overnight batch of unattended desktop flows was supposed to process 340 invoice records from a legacy ERP system, update a SharePoint list, and drop a summary Excel file on a shared drive before the finance team arrived at 8:00. You open Power Automate and see a wall of red. Forty-three runs failed. Twelve are still marked "running" but the machines show idle. Three completed successfully but produced no output files. The finance team is already pinging you on Teams.

    This scenario isn't hypothetical — it's the reality of operating RPA at scale. A single desktop flow working on your laptop during development is one thing. A fleet of bots running nightly across a dozen machines, triggered by cloud flows, processing thousands of records, and integrating with three different enterprise systems is something else entirely. The debugging skills that work in a studio session — step through the flow, look at the variable values, rerun it — simply don't translate. You need a different mental model, different tooling, and a different operational discipline.

    By the end of this lesson, you'll have exactly that. We'll build a complete picture of how to monitor desktop flow operations at production scale, diagnose failures systematically rather than by guesswork, instrument your flows so they tell you what went wrong before you have to ask, and design operational processes that turn reactive firefighting into proactive management.

    What you'll learn:

    • How the Power Automate monitoring ecosystem works, from the portal's run history to Dataverse event data, and what each layer can and can't tell you
    • How to instrument desktop flows with structured logging so failures are self-describing
    • How to build a custom monitoring dashboard using cloud flows that surfaces actionable alerts before problems compound
    • Systematic diagnostic techniques for the five most common categories of production failure
    • How to design machine group configurations, retry architectures, and flow telemetry that make scale-up sustainable rather than fragile

    Prerequisites

    This is an expert-level lesson. You should already be comfortable with:

    • Building and running desktop flows in Power Automate Desktop, including subflows and error handling (if error handling is new territory, work through Error Handling in Desktop Flows: On Block Error, Retry Policies, and Recovery Screenshots first)
    • Triggering desktop flows from cloud flows and understanding the connection between the two (Triggering Desktop Flows from Cloud Flows: Passing Inputs and Returning Outputs)
    • Machine and machine group configuration for unattended RPA (Managing Machines and Machine Groups for Scalable Unattended Automation in Power Automate)
    • Basic Dataverse concepts and Power Automate cloud flow authoring

    Understanding the Monitoring Ecosystem

    Before you can troubleshoot at scale, you need to understand what data is available to you and where it lives. Power Automate's monitoring surface is more layered than it first appears, and most practitioners only use the top layer — which is like trying to fix a car engine by looking at the paint.

    Layer 1: The Power Automate Portal Run History

    The most visible layer is the run history panel you see when you open any cloud flow in the portal. For flows that trigger desktop flows, this shows you whether the parent cloud flow succeeded or failed, and you can drill into each action to see the desktop flow invocation result.

    This is useful for quick triage on small volumes. If you had five runs today, you can click through them manually. At 340 runs? You need automation to aggregate this.

    The key limitation here is latency and granularity. The portal shows you the final status of each run — succeeded, failed, or timed out — but the error message is whatever the desktop flow surface back to the cloud flow. If your desktop flow has solid error handling that catches exceptions and returns structured error data, the portal message is useful. If it just propagates the raw exception from a failed UI interaction, you get something like Element not found: ComboBox1 with no context about which record was being processed, what state the application was in, or how many retries were attempted.

    Layer 2: Desktop Flow Run Details in the Monitor Section

    Navigate in the Power Automate portal to Monitor, then Desktop Flow Runs. This view is specifically for desktop flow telemetry and is distinct from the cloud flow run history. Here you can filter by:

    • Date range
    • Machine or machine group
    • Flow name
    • Status (Succeeded, Failed, Queued, Timed Out)

    Each run entry shows you start time, end time, the machine it ran on, and the status. Clicking into a run shows you the action-by-action trace — every action that executed, how long it took, and whether it succeeded. This is the closest thing to a step debugger for production runs.

    Key insight

    The action-level trace in Desktop Flow Run Details is only available for runs within the past 28 days. If you need historical data for trend analysis — say, to understand whether failures are increasing over time or correlate with a specific machine — you must extract this data before the 28-day window closes. Build that extraction into your operational process from day one.

    Layer 3: Dataverse Tables

    Everything in the monitoring ecosystem ultimately writes to Dataverse. The tables that matter most for desktop flow monitoring are:

    • flowsession — One row per desktop flow run. Contains status, start/end timestamps, the machine it ran on, error codes, and a reference to the flow definition.
    • flowmachine — One row per registered machine. Contains machine status, connectivity state, and last heartbeat time.
    • flowmachinegroup — Machine group definitions and membership.
    • workflowbinary — The flow definition itself (less useful for monitoring, more for ALM).

    You can query these tables directly with Power Automate cloud flows using the Dataverse connector, with Power Apps, with Power BI, or even from external systems via the Dataverse Web API. This is how you build real-time dashboards and automated alerting that doesn't require a human to stare at the portal.

    Tip

    The flowsession table is your single source of truth for run telemetry. Every monitoring tool you build should ultimately be reading from or writing to this table. Spend time understanding its schema — open it in the maker portal via Data > Tables > find flowsession — before you try to build automated monitoring.

    Layer 4: Custom Logging Within Flows

    The three layers above tell you what happened at the run level. Custom logging tells you what happened inside the run — which record caused a failure, what value triggered an unexpected condition, which branch of a decision tree was taken 400 rows into a dataset.

    This layer is entirely your responsibility to build. Power Automate Desktop doesn't automatically log variable values or application state. Every meaningful piece of context you'll want during an incident investigation needs to be explicitly written somewhere during execution.

    We'll build this instrumentation out in detail shortly.


    Designing Flows for Observability

    The single biggest quality-of-life difference between a painful production incident and a manageable one is whether your flows were designed to be observable. Observability means the internal state of your automation is visible from the outside without having to reproduce the failure.

    The Minimum Viable Log Record

    Every desktop flow run should emit at minimum one structured log record per significant operation. For an invoice processing flow, "significant operation" means one record per invoice — not one record per run.

    Here's what a minimum viable log record looks like. Design a SharePoint list (or Dataverse table, or even a CSV on a shared drive) with these columns:

    Column Type Purpose
    RunId Text Unique identifier for the desktop flow run
    FlowName Text Which desktop flow
    MachineName Text Which machine executed it
    RecordId Text The business identifier (invoice number, employee ID, etc.)
    Step Text Named checkpoint within the flow
    Status Choice Started / Completed / Failed / Skipped
    ErrorCode Text Application-level error code if applicable
    ErrorMessage Text Human-readable error detail
    Timestamp DateTime When this log entry was created
    DurationSeconds Number How long this step took

    Populating this during a flow run is straightforward. Use a SharePoint "Create item" action inside your desktop flow (via the Web or a direct API call using the HTTP action), or write to a Dataverse table using the Dataverse connector. If your flows process high volumes and latency matters, write to a flat file during execution and flush it to your log destination at the end of the run.

    Warning

    Don't try to use the Power Automate Desktop built-in logging display (the bottom panel in the designer) as a substitute for a proper log store. That panel is a development tool. In production unattended runs, nobody is watching it, and the data doesn't persist anywhere accessible to your monitoring infrastructure.

    Structured Error Payloads

    When a flow fails, the error information passed back to the calling cloud flow is whatever you put in your On Block Error handler. Most flows in the wild do one of two things: nothing (the raw exception propagates), or a generic message like "Invoice processing failed." Neither is actionable at 7:47 AM.

    Instead, build error payloads as structured text. In your On Block Error handler, compose a JSON string that captures all the context available at the point of failure:

    SET ErrorPayload TO $'''{
      "flow": "InvoiceProcessor_v3",
      "machine": "%MachineName%",
      "recordId": "%CurrentInvoiceId%",
      "step": "%CurrentStep%",
      "errorMessage": "%LastError%",
      "loopIteration": %LoopIndex%,
      "applicationState": "%AppWindowTitle%",
      "timestamp": "%CurrentDateTime%"
    }'''
    

    Return this payload as an output variable from your desktop flow. The calling cloud flow can then parse this JSON, route the error to the appropriate alert channel, and store it in your log destination with full context.

    Tip

    Use a CurrentStep variable that you update at each named checkpoint in your flow — SET CurrentStep TO 'OpeningERPApplication', then SET CurrentStep TO 'NavigatingToInvoiceEntry', etc. When an error occurs anywhere, CurrentStep tells you exactly how far the flow got. This one pattern eliminates a huge percentage of "I have no idea what it was doing when it crashed" incidents.

    Correlation IDs

    In a scaled operation, you have cloud flows triggering desktop flows, and the same run might generate entries in multiple log stores. Without a correlation mechanism, reconstructing what happened during a specific run requires cross-referencing timestamps and hoping they line up.

    Use a correlation ID: a unique identifier generated by the cloud flow before it calls the desktop flow, passed as an input to the desktop flow, and stamped on every log record that run produces. UUID generation in a cloud flow is as simple as using the guid() expression. Pass it as an input variable named CorrelationId.

    Now when you're investigating a failure, you filter your log store on CorrelationId = '3f7a2b...' and you see the complete picture: every checkpoint the flow hit, exactly where it stopped, what the application state was, and how long each step took. This is the difference between a 10-minute diagnosis and a two-hour archeological excavation.


    Building an Automated Monitoring Cloud Flow

    Reading the portal manually is not a monitoring strategy — it's a ritual. At scale, you need monitoring that comes to you, not the other way around. Let's build a cloud flow that watches your desktop flow runs and surfaces problems proactively.

    The Architecture

    The monitoring cloud flow runs on a schedule — every 15 minutes is a reasonable starting point for batch operations, every 5 minutes for time-sensitive processes. It queries the flowsession Dataverse table, identifies runs that have failed or been running longer than expected, and sends alerts to Teams or email.

    Here's the logical flow:

    1. Trigger: Recurrence, every 15 minutes
    2. Calculate lookback window: Current time minus 20 minutes (slightly longer than the trigger interval to avoid gaps)
    3. Query flowsession: Get all sessions where createdon is in the lookback window and statuscode indicates failure, or where statuscode indicates running and start time is older than your expected maximum runtime
    4. Deduplicate: Check a "already alerted" SharePoint list or Dataverse table to avoid sending duplicate alerts for the same run
    5. Enrich: Join with flowmachine data to get the machine name
    6. Alert: Send a structured Teams message with the flow name, machine, error summary, and a direct link to the run in the portal
    7. Record: Write the alerted run IDs to the dedup store

    Querying the flowsession Table

    In your cloud flow, add a Dataverse "List rows" action targeting the flowsession table. The critical filter uses OData syntax:

    createdon ge @{addMinutes(utcNow(), -20)} and statecode eq 1 and statuscode eq 8
    

    Status codes in flowsession:

    • statuscode = 2 — Running
    • statuscode = 4 — Succeeded
    • statuscode = 8 — Failed
    • statuscode = 9 — Timed Out
    • statuscode = 10 — Cancelled

    To catch stuck runs — flows that show "Running" but have been executing longer than expected — add a second query:

    createdon le @{addMinutes(utcNow(), -90)} and statuscode eq 2
    

    This catches runs that started more than 90 minutes ago and are still marked as running. Adjust the threshold to match your expected maximum runtime per flow type. An invoice processor that normally takes 8 minutes per batch shouldn't ever be running for 90 minutes — if it is, either the machine is hung, the application locked up, or the flow is in an infinite loop.

    Key insight

    The "stuck run" query is often more valuable than the "failed run" query for catching serious problems. A failed run at least completed — its error is recorded and you can examine it. A stuck run might be holding a machine hostage, blocking your queue, and locking an application that other processes need. Detecting and killing stuck runs quickly is critical for maintaining throughput.

    Building the Alert Message

    A Teams alert that just says "Desktop flow failed" is almost as useless as no alert. Your alert messages should answer: what failed, where, when, why (as much as is known), and what to do next.

    Here's a Teams message template that works in practice:

    🔴 Desktop Flow Alert — Action Required
    
    Flow: [flowname]
    Machine: [machine name]  
    Run ID: [flowsessionid]
    Started: [createdon]
    Status: FAILED
    
    Error Summary:
    [errordetail — your structured error payload if available]
    
    Links:
    • View run details: https://make.powerautomate.com/...
    • Flow definition: https://make.powerautomate.com/...
    
    Runbook: [link to your incident runbook for this flow]
    

    The runbook link is often overlooked but enormously valuable. When the alert fires at 7:47 AM and the person who built the flow is on vacation, whoever picks it up needs a runbook that explains what the flow does, what systems it touches, what common failures look like, and how to triage them. Bake the runbook link directly into the alert.


    Systematic Diagnostic Framework

    When a production failure lands in your lap, the worst thing you can do is start randomly poking at things. That approach sometimes gets lucky, but it creates no durable understanding and will leave you in the same position next time. Instead, work through a systematic diagnostic framework.

    Category 1: Machine and Connectivity Failures

    These failures happen before your flow logic even starts executing. Symptoms: runs go to "Failed" almost instantly, or sit in "Queued" indefinitely without a machine picking them up.

    Diagnosis steps:

    1. Open Monitor > Machines and check the machine status. A healthy machine shows as "Online." If it shows "Offline," the machine agent isn't connecting to the Power Automate service.

    2. On the machine itself, check the Power Automate Machine Runtime agent in the system tray. If it's not running, start it. Check Windows Services for UIFlowService — it should be running.

    3. Check the machine's network connectivity. The agent needs to reach Power Automate service endpoints. If your organization has network restrictions or a proxy, verify the agent's proxy configuration.

    4. If runs are queuing but not executing, check whether the machine is available for the correct connection. In unattended mode, the machine needs to be configured with a valid Windows account, and that account must be able to log in. Check that the account password hasn't expired — this is one of the most common "suddenly everything stopped" causes in enterprise environments.

    Warning

    Machine account password expiration is a silent killer. Windows accounts used for unattended desktop flow execution will stop working when their passwords expire, and the failure mode is not always obvious — you might see generic "cannot connect to machine" errors rather than an explicit authentication failure. Implement a monitoring check specifically for account password expiration, or use a service account with "password never expires" policy (in compliance with your security team's guidance).

    1. For machine groups, check whether the group has healthy members. If all machines in a group are offline simultaneously, check whether there was a Windows Update or patch deployment overnight.

    Category 2: Application State Failures

    These failures happen during execution, typically when the desktop flow can't find a UI element, the target application is in an unexpected state, or a previous failed run left the application open and mid-operation.

    Symptoms: errors like "Element not found," "Cannot perform action on element," "Window not found," or more specific application error messages.

    Diagnosis steps:

    1. Check whether the failure is consistent (fails on every run) or intermittent (fails on some runs). Consistent failures usually indicate a change in the application — an update changed the UI, a form was redesigned, or a configuration changed. Intermittent failures usually indicate timing issues or application state problems.

    2. For consistent failures, compare the error location with recent application changes. Was there a deployment last night? Did the application version change? Look at the selector that failed and verify it still describes the correct element. This is deeply connected to the challenge of building selectors that don't break over time.

    3. For intermittent failures, look at the timing. Did they happen at specific times, on specific machines, or against specific records? A failure that only happens on machine 3 suggests a machine-specific configuration issue. A failure that only happens after 2 hours of running suggests memory pressure or application degradation over long sessions.

    4. Check whether the application was left open from a previous failed run. If your error handling doesn't close the application cleanly, the next run might find the application in a locked state, a "do you want to save" dialog, or mid-way through a previous operation. Your On Block Error handler must include application cleanup logic.

    5. Recovery screenshots (if enabled) are invaluable here — they capture what was on screen at the moment of failure. If you haven't enabled recovery screenshots, do it now. The configuration is in the flow's run settings in the portal. Yes, it uses storage, but the diagnostic value in production incidents is enormous.

    Category 3: Data Quality Failures

    These failures occur when the flow encounters data it wasn't designed to handle — null values where a value was expected, text in a numeric field, a date in an unexpected format, or a record that exists in one system but not in another.

    Symptoms: type conversion errors, null reference errors, "item not found" errors in lookups, or silent completion (the flow succeeded but produced wrong output).

    Key insight

    Silent failures are more dangerous than explicit failures. A flow that completes successfully but writes wrong data is worse than a flow that fails loudly, because the silent failure might not be detected for days. This is why validating output — not just tracking flow status — is part of a complete monitoring strategy.

    Diagnosis steps:

    1. Identify the record that caused the failure. Your RecordId in the log is your starting point. Go look at that record in the source system. What's unusual about it?

    2. Common culprits: special characters in text fields (particularly in fields used as file names or SQL inputs), very long field values, records in an unusual status or lifecycle state, records created by a different process with a different data format, or records that reference foreign keys that no longer exist.

    3. For flows that process data tables, add validation logic at the beginning of each record's processing: check for null, verify expected formats, confirm required fields are populated. Fail fast on invalid data with a clear error message rather than propagating the bad data through the flow and getting a cryptic error 20 steps later.

    4. If your flow processes records from an Excel source, pay particular attention to type coercion behavior — Excel and Power Automate Desktop interact in ways that aren't always obvious, especially around date and number formats.

    Category 4: Performance Degradation and Timeouts

    These failures are particularly tricky because the flow logic is correct, the data is valid, but operations just take too long.

    Symptoms: "Timed out" status, runs that succeed but take much longer than baseline, intermittent element-not-found errors that resolve on retry (usually caused by the application taking longer to respond than the flow's wait timeout).

    Diagnosis steps:

    1. Compare current run duration against baseline. This is where historical monitoring data earns its keep — if you've been logging DurationSeconds per step, you can identify exactly which step started taking longer, and when.

    2. Common causes of performance degradation:

      • The target application is slow (database performance issue, high load on an ERP system, network latency to a remote application)
      • The machine is under resource pressure (high CPU, low memory, disk I/O saturation)
      • The dataset being processed grew larger (more records to process per batch)
      • The application's internal data grew larger (a slow application search that was fast when the database had 10,000 records is slow when it has 500,000)
    3. Check machine resource utilization during the run window. If you don't have machine monitoring (Windows Performance Monitor, Azure Monitor for Arc-enabled machines, or a third-party agent), add it. You can't diagnose performance problems without performance data.

    4. Review your flow's wait strategies. Flows that use fixed Wait actions (wait 3 seconds, then proceed) become fragile as applications slow down. Replace fixed waits with condition-based waits: wait until the element is present, wait until the application window title changes, wait until the status label shows a specific value. These are more resilient to variable application performance.

    Category 5: Concurrency and State Conflicts

    These failures are the most complex and most common as you scale up, because they emerge from interactions between runs rather than from a single run's logic.

    Symptoms: inconsistent failures that seem random, data corruption or duplicate processing, errors that suggest two flows were interacting with the same resource simultaneously.

    Diagnosis steps:

    1. Look at the timing overlap between runs. If machine A and machine B were both processing records from the same source simultaneously, and they both read the same "next available record" before either marked it as "in progress," they'll both process the same record. This is a classic queue-consumer race condition.

    2. Common concurrency patterns that cause problems at scale:

      • Two machines reading from the same Excel file simultaneously (Excel file locking means one will fail or get a stale read)
      • Multiple flows appending to the same log file
      • Multiple flows processing the same work queue without proper lease/lock semantics
      • Session conflicts when two unattended flows try to use the same Windows session
    3. For work queue management, use Power Automate's built-in Desktop Flow Work Queue feature (available in premium licenses), or implement a Dataverse-based queue with optimistic concurrency. The work queue provides built-in lease semantics — a record checked out by one run isn't visible to other runs until the lease expires or is released.

    4. For machine group configuration, verify that your session isolation is correct. Each unattended run should get a clean Windows session. If session reuse is happening unexpectedly, investigate the machine configuration. See attended vs. unattended run mode configuration for the specifics.


    Building a Production Operations Dashboard

    Manual triage is necessary during incidents. But the goal of good monitoring is to answer operational questions before they become incidents. A production operations dashboard should answer:

    • What is the current health of my machine fleet?
    • How many runs are queued, running, succeeded, and failed in the last 24 hours?
    • Which flows have the highest failure rates?
    • Which machines have the highest failure rates?
    • Are there any stuck runs right now?
    • What is the trend in processing volume and duration over the past 30 days?

    Power BI Dashboard Architecture

    Build your dashboard in Power BI against the Dataverse tables directly. The Power BI connector for Dataverse handles the authentication, and you get a live semantic layer over your operational data.

    Key measures to build:

    Run Success Rate (rolling 24h):

    Success Rate = 
    DIVIDE(
      COUNTROWS(FILTER(flowsession, flowsession[statuscode] = 4)),
      COUNTROWS(flowsession),
      0
    )
    

    Average Run Duration by Flow (last 30 days): Query the difference between modifiedon and createdon for completed sessions, group by flow name.

    Machine Utilization: Count runs per machine per hour over the last 7 days to understand load distribution and identify machines that are consistently overloaded or underutilized.

    Failure Rate by Hour of Day: Identify whether failures cluster at specific times — early morning when batch jobs compete for resources, business hours when attended users are on machines, late night when application maintenance windows occur.

    Tip

    Set up Power BI data alerts on your key measures. When the 24-hour success rate drops below 95%, you want a push notification to Teams before anyone files a support ticket. Power BI alerts can trigger Power Automate flows, which can page your on-call engineer, create a ServiceNow incident, and spin up a Teams channel — all automatically.

    Operational Runbook Integration

    Every production desktop flow operation should have a runbook. Not a lengthy Word document — a living, linked document that an engineer who didn't build the flow can use to triage an incident at 7:47 AM.

    A minimal runbook for a desktop flow covers:

    1. What the flow does: Business purpose, systems it touches, data it processes
    2. Normal operating parameters: Expected duration, expected volume, expected success rate, machine assignment
    3. Common failure modes: The specific errors that have occurred before and their resolutions
    4. Triage steps: In order — what to check first, what to check second, who to escalate to
    5. Manual recovery procedure: If the flow can't complete automatically, how does a human do the work? (Critical for flows that support time-sensitive business processes)
    6. Contacts: Who owns the flow, who owns the target application, who owns the machine infrastructure

    Store runbooks in a SharePoint wiki linked directly from your dashboard. The operational maturity jump from "we have monitoring" to "we have monitoring and runbooks" is enormous.


    Advanced Instrumentation Patterns

    For teams operating dozens or hundreds of desktop flows, manual instrumentation becomes its own maintenance burden. Here are advanced patterns that scale better.

    Instrumentation Subflow Pattern

    Rather than adding logging code directly to each flow, build a standard logging subflow that all flows call. The subflow accepts parameters — LogLevel, Step, Message, RecordId, CorrelationId — and handles writing to your log destination.

    This means the log destination, the log format, and the retry logic for failed log writes are all in one place. When you want to add a new field to your log records, you change one subflow. This is the subflow and reusable logic pattern applied to observability.

    # LogEvent subflow signature
    Parameters:
      - LogLevel (Text)         # "INFO", "WARN", "ERROR"
      - Step (Text)             # Named checkpoint
      - Message (Text)          # Human-readable description
      - RecordId (Text)         # Business identifier
      - CorrelationId (Text)    # Run correlation ID
      - DurationSeconds (Number) # Optional step duration
    
    # Body
    SET LogRecord TO $'''{"level": "%LogLevel%", "step": "%Step%", "message": "%Message%", "recordId": "%RecordId%", "correlationId": "%CorrelationId%", "machine": "%MachineName%", "duration": %DurationSeconds%, "timestamp": "%CurrentDateTime%"}'''
    
    # Write to log destination
    # Use HTTP action to POST to your log endpoint, or
    # write to Dataverse using the Dataverse REST API
    

    Performance Timing Pattern

    Many production issues are performance regressions — the flow is doing the right thing, just slowly. Catch these before they become failures by timing individual steps.

    SET StepStartTime TO %CurrentDateTime%
    
    # ... the step logic here ...
    
    SET StepEndTime TO %CurrentDateTime%
    SET StepDurationSeconds TO [compute difference between StepStartTime and StepEndTime]
    
    CALL LogEvent with LogLevel: "INFO", Step: "ProcessInvoiceInERP", DurationSeconds: %StepDurationSeconds%
    

    When you chart DurationSeconds per step over time in Power BI, you'll see performance regressions as a rising trend before they cross the threshold into timeout failures. This moves you from reactive (it timed out, now what?) to proactive (it's been getting slower for 3 weeks, let's investigate the ERP performance now).

    Canary Run Pattern

    For high-stakes automated operations, implement a canary run: a synthetic test execution that runs a known-good test case before the real batch starts. The canary run processes a test record — one that won't affect production data — and validates that all systems are responding correctly. If the canary fails, abort the main batch and alert. Don't wait to discover that all 340 invoices failed after spending 6 hours trying to process them.

    Implement the canary as a separate desktop flow triggered at the beginning of your cloud flow orchestrator, before the main processing loop starts. If it fails, the cloud flow branches to an abort-and-alert path rather than continuing.

    Warning

    The canary test case must be truly representative. A canary that only validates that the application opens and you can log in doesn't catch failures in the actual business logic. Design your canary to execute a complete end-to-end test case, including all systems the main flow touches.


    Scaling Machine Fleet Management

    Monitoring isn't just about individual runs — it's about the health of your entire execution infrastructure. At scale, machine fleet management becomes a critical operational discipline.

    Machine Health Monitoring

    Query the flowmachine Dataverse table in your monitoring cloud flow to check machine health independently of run status. A machine that's online and not processing runs is idle — that might be expected (off-peak hours) or it might indicate a queue routing problem. A machine that shows offline but should be processing overnight batch jobs needs immediate attention.

    Build a machine health check into your monitoring flow alongside the run status check. Alert separately on:

    • Machines offline when they should be online
    • Machines that have been continuously running for longer than your maximum expected session duration
    • Machines with failure rates significantly higher than the group average (outlier detection — this usually means a machine-specific configuration problem)

    Queue Depth and Throughput Monitoring

    As your operation scales, the ability to answer "will this batch complete before the business opens?" becomes critical for planning. Build throughput metrics:

    • Current queue depth: How many runs are queued right now?
    • Average throughput: Runs completed per hour over the past 4 hours
    • Estimated completion time: Current queue depth ÷ current throughput rate

    When your overnight batch typically processes 300 records at 15 records/hour/machine, and you have 4 machines, you have 5 hours of processing time. If something reduced throughput to 8 records/hour/machine at 2 AM, you know at 2 AM — not at 8 AM when the business calls.

    Key insight

    Throughput monitoring reveals problems that success-rate monitoring misses. If all your runs are succeeding but each one is taking twice as long as usual, your success rate stays at 100% while your batch is headed for a catastrophic overrun. Always monitor both.

    Scaling Out vs. Scaling Up

    When your fleet consistently can't process the required volume within the available window, you have three options: process faster (optimize the flows), process in parallel (add machines), or process less (reduce the scope of each run).

    Adding machines is the most tempting solution but not always the right one. Before adding machines, profile where the time goes. If 70% of the run time is the target application thinking — waiting for an ERP query to return — adding machines means 10 machines simultaneously hammering the ERP, which might make it slower for everyone. In that case, optimizing the flow (batching queries, using direct API calls where available instead of UI automation) is the right answer.

    If the bottleneck is genuinely in the desktop flow execution itself — navigation, form filling, waiting for UI elements — adding machines scales linearly and is the right choice. Use Power Automate's machine group load balancing to distribute work automatically across the group.


    Hands-On Exercise

    Let's build a concrete monitoring solution for a realistic scenario. Imagine you have an unattended desktop flow called AccountsPayable_InvoiceProcessor that runs nightly, processing 200-400 invoices from a legacy ERP system. It's been in production for 3 months, runs on a group of 4 machines, and is triggered by a cloud flow that reads a Dataverse work queue.

    Your task: Build the monitoring layer for this operation.

    Step 1: Instrument the desktop flow

    Open AccountsPayable_InvoiceProcessor in Power Automate Desktop. Add a CorrelationId input variable. Add a CurrentStep text variable at the top of the main flow. Add a LogEvent subflow using the pattern described in the Advanced Instrumentation section above.

    Add step logging at these specific points:

    • Flow started (log the batch size, the correlation ID, the machine name)
    • Each invoice processing started (log the invoice number)
    • Each invoice processing completed successfully (log duration)
    • Each invoice processing failed (log the error, the invoice number, the current application state)
    • Flow completed (log total processed, total succeeded, total failed)

    Step 2: Build the monitoring cloud flow

    Create a new cloud flow with a recurrence trigger: every 15 minutes.

    Add actions to:

    1. Calculate a lookback window: addMinutes(utcNow(), -20)
    2. Query flowsession for failed and stuck runs in the window
    3. For each result, compose a Teams alert card with flow name, machine name, error detail, run ID, and a direct link to the portal run details
    4. Post the card to your RPA operations Teams channel
    5. Record the run IDs to a SharePoint list to prevent duplicate alerts on the next cycle

    Step 3: Build a simple Power BI report

    Connect Power BI Desktop to your Dataverse environment. Import the flowsession and flowmachine tables. Build these four visuals:

    1. A card showing today's success rate as a percentage
    2. A bar chart of runs by status (succeeded/failed/timed out) for the past 7 days
    3. A line chart of average run duration per day for the past 30 days
    4. A table of currently queued or running flows with their machine assignment and queue time

    Publish to Power BI Service and set up a data refresh every hour.

    Step 4: Write the runbook

    In your SharePoint wiki, create a page titled "AccountsPayable_InvoiceProcessor Runbook." Populate the sections described in the Operational Runbook Integration section. Be specific — document the actual error messages you've seen in 3 months of production, what they mean, and what resolves them. Link this runbook from your Teams alert message template.


    Common Mistakes & Troubleshooting

    "I see runs failing but there's no error detail"

    The most common cause: the desktop flow isn't returning any output variables, so the cloud flow's desktop flow action has nothing to report. Add structured output variables to your desktop flow — at minimum an IsSuccess boolean and an ErrorDetails JSON string — and map them as outputs. Then in your cloud flow, log these values regardless of whether the run succeeded or failed.

    The second most common cause: the flow crashed before your error handling had a chance to capture context. If your On Block Error handler is only on certain subflows but not the entire flow, a crash in the main flow body might not be caught. Move your outermost error handler to wrap the entire Main flow.

    "Runs show as 'Running' in the portal but the machines are idle"

    This is almost always a communication failure between the machine agent and the Power Automate service. The machine completed the run (or crashed) but didn't successfully report the final status back to the service.

    Resolution: The runs will eventually time out and transition to "Timed Out" status, typically within 4-8 hours. To prevent this, implement a health-check mechanism: the calling cloud flow should have a timeout configured on the "Run a desktop flow" action. When the action exceeds the timeout, the cloud flow treats it as failed and can alert appropriately. Don't rely on the portal to eventually clean up stuck runs — implement explicit timeouts.

    "My monitoring cloud flow fires too many alerts and people start ignoring them"

    Alert fatigue is a real operational problem. Fix it by:

    1. Adding proper deduplication — don't alert on the same run ID more than once
    2. Grouping alerts — instead of one Teams message per failed run, send one message per monitoring cycle that lists all failures in that window
    3. Implementing severity tiers — a single isolated failure might generate a low-priority notification, while a failure rate above 20% generates a high-priority page
    4. Routing alerts by flow criticality — failures in your payment processing flow wake someone up; failures in your report generation flow generate a next-business-day ticket

    "The flow fails differently on different machines"

    Machine-specific failures usually mean one of: different software versions installed, different application configurations, different Windows settings, or different network access. Start by running the flow on the failing machine in attended mode while screen sharing, and compare the behavior to a healthy machine. Pay particular attention to application window sizes (if your selectors depend on element position), screen resolution differences, and regional settings (date/number formats).

    For legacy Windows application automation, machine-specific differences in visual themes and accessibility settings can cause UI element recognition to behave differently across machines.

    "Everything worked for weeks and then suddenly everything started failing simultaneously"

    Simultaneous failure across all machines almost always points to a shared dependency that changed. The candidates in roughly decreasing frequency:

    • The target application was updated (UI changed, authentication changed, API endpoint changed)
    • Network or security policy changed (certificate expired, firewall rule changed, proxy configuration changed)
    • Credentials expired (service account password, shared API key, OAuth token)
    • The data source changed (file moved, SharePoint list permissions changed, database schema changed)

    Work backwards from when the failures started. What changed at that time? Check deployment logs, change management records, and system event logs on the machines. Correlate the failure start time with the change log — you'll almost always find the cause quickly once you know when it started.


    Summary & Next Steps

    Monitoring and troubleshooting desktop flows at scale is fundamentally a systems engineering discipline, not just a Power Automate skill. The principles — observability, structured telemetry, automated alerting, systematic diagnosis, runbooks — come from decades of site reliability engineering practice applied to the specific domain of RPA.

    The key principles to carry forward:

    Design for observability from the start. Retrofitting monitoring into a production flow is painful and incomplete. The CurrentStep variable, correlation IDs, structured error payloads, and step-level logging should be standard practice in every flow you build.

    Monitor at multiple layers. Portal run history answers "did it run?" Dataverse queries answer "what's the pattern?" Custom instrumentation answers "what happened inside?" You need all three to be effective at scale.

    Automate your monitoring. Humans reading the portal is not a monitoring strategy. Build the cloud flows that watch your desktop flows, alert proactively, and give you the information you need to act — not just the information that something happened.

    Build systematic diagnostic processes. When something fails, work through the five failure categories methodically rather than randomly. The right diagnosis the first time is always faster than the wrong diagnosis three times.

    Invest in operational infrastructure. Runbooks, dashboards, alert routing, and on-call procedures aren't overhead — they're what makes a scaled RPA operation sustainable for the team operating it.

    From here, consider deepening your understanding of how governance and auditing intersect with RPA operations — the CoE Toolkit for governing Power Automate at scale is a natural next step. If your desktop flows are part of larger enterprise automation solutions, understanding how to deploy and manage solutions across environments will help you manage the full lifecycle. And if you're handling sensitive data or credentials in your flows — almost inevitable at enterprise scale — review the practices in handling credentials securely in desktop flows.

    The goal of all of this is the same: automation that your organization can rely on, that gives you confidence rather than anxiety, and that frees you to build more instead of constantly fixing what's already running.

    Work With Us

    From insight to implementation

    Reading is the start. When you're ready to build the data, automation, or AI systems behind it, our team turns strategy into shipped results.

    Let's Build

    Power Automate Desktop & RPA

    Previous

    Scripting Inside Desktop Flows: Running PowerShell, Python, and VBScript Actions

    Next

    Capturing and Replaying Mouse Clicks and Keystrokes with the Power Automate Desktop Recorder

    Related Insights

    Power AutomateFoundation

    Automating Windows Clipboard Operations in Power Automate Desktop: Capturing, Transforming, and Pasting Data Across Applications

    17 min
    Power AutomateFoundation

    Automating Windows Clipboard Operations in Power Automate Desktop: Copying, Pasting, and Transferring Data Between Applications Without UI Interaction

    18 min
    Power AutomateExpert

    Building a Resilient Unattended RPA Orchestration Framework in Power Automate Desktop: Queue-Driven Job Dispatch, Machine Load Balancing, and Automated Recovery for 24/7 Production Bot Fleets

    30 min

    On this page

    • Introduction
    • Prerequisites
    • Understanding the Monitoring Ecosystem
    • Layer 1: The Power Automate Portal Run History
    • Layer 2: Desktop Flow Run Details in the Monitor Section
    • Layer 3: Dataverse Tables
    • Layer 4: Custom Logging Within Flows
    • Designing Flows for Observability
    • The Minimum Viable Log Record
    • Structured Error Payloads
    • Correlation IDs
    • Building an Automated Monitoring Cloud Flow
    • The Architecture
    • Querying the flowsession Table
    • Building the Alert Message
    • Systematic Diagnostic Framework
    • Category 1: Machine and Connectivity Failures
    • Category 2: Application State Failures
    • Category 3: Data Quality Failures
    • Category 4: Performance Degradation and Timeouts
    • Category 5: Concurrency and State Conflicts
    • Building a Production Operations Dashboard
    • Power BI Dashboard Architecture
    • Operational Runbook Integration
    • Advanced Instrumentation Patterns
    • Instrumentation Subflow Pattern
    • Performance Timing Pattern
    • Canary Run Pattern
    • Scaling Machine Fleet Management
    • Machine Health Monitoring
    • Queue Depth and Throughput Monitoring
    • Scaling Out vs. Scaling Up
    • Hands-On Exercise
    • Common Mistakes & Troubleshooting
    • "I see runs failing but there's no error detail"
    • "Runs show as 'Running' in the portal but the machines are idle"
    • "My monitoring cloud flow fires too many alerts and people start ignoring them"
    • "The flow fails differently on different machines"
    • "Everything worked for weeks and then suddenly everything started failing simultaneously"
    • Summary & Next Steps