The Library · Insights
Deep dives across data, automation & AI
29 in-depth articles — from Excel foundations to production machine learning. Filter by topic or difficulty to find exactly what you need.
WSD
Microsoft FabricFoundation
Calling the Fabric Lakehouse SQL Analytics Endpoint from Power BI Desktop: Connecting, Writing Measures, and Publishing a Report Without Direct Lake
Learn how to connect Power BI Desktop to a Fabric lakehouse SQL Analytics Endpoint, build a star schema with DAX measures, and publish a production-ready report — no Direct Lake required. This lesson explains every step from finding the connection string to scheduling refreshes, with clear guidance on when to use this approach over Direct Lake mode.
21 min read
Microsoft FabricFoundation
Writing Data from a Spark Notebook to a Fabric Lakehouse Delta Table: Append, Overwrite, and Merge Patterns with PySpark
Learn the three essential write patterns for Delta tables in Microsoft Fabric: append, overwrite, and merge. This hands-on lesson teaches you exactly when to use each pattern — and how to avoid the silent data quality problems that come from choosing the wrong one.
16 min read
Microsoft FabricExpert
Implementing Row-Level Security in a Fabric Warehouse and Lakehouse SQL Analytics Endpoint: Dynamic Policies, Workspace Roles, and Testing Access as a Business User
Row-Level Security in Microsoft Fabric isn't just a T-SQL exercise — it's an architecture decision with serious implications for workspace roles, Power BI Direct Lake, and Spark access paths. This lesson walks through building dynamic RLS policies from scratch, understanding where the security envelope stops, and verifying your policies work as intended before real users hit production.
31 min read
Microsoft FabricPractitioner
Parameterizing Dataflow Gen2 Queries with Pipeline Integration: Passing Dynamic Values to Power Query for Reusable Ingestion Flows
Learn how to define Power Query parameters in Dataflow Gen2 and pass runtime values from Fabric pipelines — building a single, reusable ingestion flow that adapts to different regions, time periods, or environments without duplicating logic. Includes a full hands-on exercise and troubleshooting guide.
21 min read
Microsoft FabricFoundation
Loading Data into a Fabric Warehouse with COPY INTO and the Pipeline Copy Activity: Bulk Ingestion from Parquet and CSV Files in OneLake
Learn how to bulk-load Parquet and CSV files from OneLake into a Microsoft Fabric Warehouse using COPY INTO and the Pipeline Copy Activity. This hands-on lesson covers file path patterns, CSV configuration options, column mapping, and post-load verification — everything you need to run production-grade ingestion jobs.
19 min read
Microsoft FabricExpert
Optimizing Delta Table Performance in a Fabric Lakehouse: V-Order, OPTIMIZE, VACUUM, and Z-Order for Faster Queries and Smaller Storage
Delta tables don't optimize themselves. Learn how V-Order encoding, file compaction with OPTIMIZE, multi-dimensional data skipping with Z-Order, and storage reclamation with VACUUM work together to make your Fabric Lakehouse queries dramatically faster and cheaper to store. This lesson goes deep into the internals so you know exactly when and why to apply each technique.
32 min read
Microsoft FabricPractitioner
Incrementally Loading Data into a Fabric Lakehouse with Watermarks and Pipeline Lookup Activities
Full table loads don't scale — and sooner or later every production pipeline needs an incremental strategy. Learn how to build a complete watermark-based incremental load pattern in Microsoft Fabric using Lookup activities, parameterized Copy activities, and PySpark notebook watermark updates.
20 min read
Microsoft FabricFoundation
Querying Lakehouse Data with the SQL Analytics Endpoint: Writing T-SQL Against Delta Tables Without a Warehouse
Every Fabric lakehouse comes with a built-in SQL interface that lets you run T-SQL against Delta tables — no warehouse required. Learn how to write queries, create views, join across tables, and connect from SSMS or Power BI, all against the same data your Spark notebooks write.
17 min read
Microsoft FabricPractitioner
Connecting a Power BI Semantic Model to a Fabric Lakehouse in Direct Lake Mode: Creating, Refreshing, and Optimizing Delta Tables for Reporting
Direct Lake mode is Power BI's fastest connection strategy for Fabric lakehouses — but only if your Delta tables are structured correctly. This lesson teaches you how to create V-Ordered gold tables, manage framing, diagnose DirectQuery fallback, and keep your reports fresh without expensive import cycles.
22 min read
Microsoft FabricFoundation
Writing and Running Your First PySpark Notebook in Microsoft Fabric: Reading Files, Creating Delta Tables, and Saving to the Lakehouse
Learn how to build a real PySpark notebook in Microsoft Fabric from scratch — reading CSV files from Lakehouse storage, cleaning and transforming data, and writing Delta tables that are immediately queryable via SQL. This hands-on lesson teaches Spark fundamentals in the context you'll actually use them.
17 min read
Microsoft FabricFoundation
Scheduling and Automating Fabric Data Pipeline Runs with Activity-Level Retries, Alerts, and Email Notifications
Learn how to turn a basic Fabric pipeline into a production-grade, self-healing automation. This lesson covers schedule triggers, activity-level retry policies, workspace alerts, and custom email notifications with dynamic content — everything you need to stop monitoring pipelines manually.
18 min read
Microsoft FabricFoundation
Monitoring Fabric Capacity Usage and Pipeline Activity with the Monitoring Hub
When pipelines run slow or fail silently, you need more than hope — you need visibility. This lesson teaches you how to use the Fabric Monitoring Hub and Capacity Metrics app to track activity, diagnose failures, and understand what your workloads are actually costing your capacity.
17 min read