Microsoft Fabric promises to end the fragmentation of modern data stacks — but what actually is it, and how does it work? This foundational lesson breaks down every Fabric workload, explains how OneLake eliminates data duplication, and shows exactly where Power BI fits into the picture.
OneLake is the storage backbone of Microsoft Fabric — but understanding it deeply means going beyond the marketing pitch. This lesson covers how OneLake is physically structured, how Delta Lake transaction logs actually work, how Shortcuts virtualize data across clouds, and the operational pitfalls that will slow you down if you skip them.
Learn how to create a Microsoft Fabric Lakehouse from scratch, load CSV files into both the Files and Tables zones, and query your data using the SQL Analytics Endpoint. This hands-on lesson builds the foundational skills you need for every Fabric data engineering project.
Both the Fabric Lakehouse and Warehouse store Delta tables in OneLake — so why does picking the wrong one cause so much pain? This lesson explains the architectural differences that actually matter and shows you how to make a confident, defensible choice for any workload.
Dataflow Gen2 is more than a drag-and-drop ETL tool — it's a Power Query runtime with query folding, a staging layer, and Delta table output built into Microsoft Fabric. This deep-dive lesson teaches you how to connect to diverse sources, write effective M transformations, maximize query folding, configure incremental refresh, and troubleshoot the failures that trip up even experienced practitioners.
Learn how to build reliable, automated data pipelines in Microsoft Fabric from the ground up. This hands-on lesson covers Copy Activities, dynamic parameters, activity chaining with dependencies, and scheduled triggers — everything you need to load data without manual intervention.
Learn how to use PySpark in Fabric Spark notebooks to clean, enrich, and write production-quality Delta tables to your lakehouse. Covers deduplication, Delta merge, partitioning, and parameterized notebooks that plug into Data Pipelines.
Learn how to design and build a production-ready medallion architecture in Microsoft Fabric using three separate lakehouses for Bronze, Silver, and Gold layers. This lesson goes beyond theory — you'll write real PySpark notebooks with MERGE patterns, quarantine logic, and Delta optimization for Power BI Direct Lake reporting.
Direct Lake mode reads Delta Parquet files directly from OneLake into Power BI's in-memory engine — delivering Import-speed performance without scheduled refreshes. Learn how framing, transcoding, and fallback work, and build a production-ready Direct Lake semantic model from a Fabric lakehouse.
Learn how to design and build a production-ready dimensional model in Microsoft Fabric Warehouse using T-SQL. This lesson covers DDL for fact and dimension tables, view design patterns, and cross-database queries that join warehouse and lakehouse data without moving it.
OneLake Shortcuts let you query data in ADLS Gen2 and Amazon S3 directly from Microsoft Fabric — no copying, no pipelines, no duplication. This deep-dive lesson covers the internal architecture, credential configuration, Spark and T-SQL querying patterns, performance optimization, and the security model you must understand before deploying shortcuts in production.
Learn how to build a complete real-time analytics pipeline in Microsoft Fabric using Eventstreams for ingestion, Eventhouses for ultra-fast time-series storage, and KQL Querysets for millisecond-latency queries. This expert-level lesson covers architecture internals, KQL query patterns, materialized views, and Power BI integration.
Database Mirroring in Microsoft Fabric creates continuous, near-real-time replication from Azure SQL Database and Snowflake directly into OneLake as Delta tables — eliminating scheduled ETL and enabling Direct Lake Power BI reporting on always-fresh operational data. This deep-dive lesson covers CDC configuration, Snowflake change tracking, schema change handling, and production operations.
Learn how to combine Microsoft Fabric's Git integration and deployment pipelines into a production-grade SDLC. This deep-dive lesson covers repository serialization, branching strategies, deployment rules, CI/CD automation, and the edge cases that trip up even experienced Fabric engineers.
Fabric's security model has four distinct permission layers — and most teams only know about one of them. This deep-dive lesson teaches you how workspace roles, item permissions, and OneLake data access policies interact so you can design and audit a governance model that actually holds up in production.
Learn how to use Microsoft Fabric's Pipeline Copy Activity to ingest data from REST APIs, Azure Blob Storage, and Azure SQL databases into a Lakehouse. This hands-on lesson walks through real configuration steps, explains every key decision, and covers common pitfalls so your first ingestion pipeline actually works.
When pipelines run slow or fail silently, you need more than hope — you need visibility. This lesson teaches you how to use the Fabric Monitoring Hub and Capacity Metrics app to track activity, diagnose failures, and understand what your workloads are actually costing your capacity.
Learn how to turn a basic Fabric pipeline into a production-grade, self-healing automation. This lesson covers schedule triggers, activity-level retry policies, workspace alerts, and custom email notifications with dynamic content — everything you need to stop monitoring pipelines manually.
Learn how to build a real PySpark notebook in Microsoft Fabric from scratch — reading CSV files from Lakehouse storage, cleaning and transforming data, and writing Delta tables that are immediately queryable via SQL. This hands-on lesson teaches Spark fundamentals in the context you'll actually use them.
Every Fabric lakehouse comes with a built-in SQL interface that lets you run T-SQL against Delta tables — no warehouse required. Learn how to write queries, create views, join across tables, and connect from SSMS or Power BI, all against the same data your Spark notebooks write.
Full table loads don't scale — and sooner or later every production pipeline needs an incremental strategy. Learn how to build a complete watermark-based incremental load pattern in Microsoft Fabric using Lookup activities, parameterized Copy activities, and PySpark notebook watermark updates.
Delta tables don't optimize themselves. Learn how V-Order encoding, file compaction with OPTIMIZE, multi-dimensional data skipping with Z-Order, and storage reclamation with VACUUM work together to make your Fabric Lakehouse queries dramatically faster and cheaper to store. This lesson goes deep into the internals so you know exactly when and why to apply each technique.
Learn how to bulk-load Parquet and CSV files from OneLake into a Microsoft Fabric Warehouse using COPY INTO and the Pipeline Copy Activity. This hands-on lesson covers file path patterns, CSV configuration options, column mapping, and post-load verification — everything you need to run production-grade ingestion jobs.
Learn how to define Power Query parameters in Dataflow Gen2 and pass runtime values from Fabric pipelines — building a single, reusable ingestion flow that adapts to different regions, time periods, or environments without duplicating logic. Includes a full hands-on exercise and troubleshooting guide.
Row-Level Security in Microsoft Fabric isn't just a T-SQL exercise — it's an architecture decision with serious implications for workspace roles, Power BI Direct Lake, and Spark access paths. This lesson walks through building dynamic RLS policies from scratch, understanding where the security envelope stops, and verifying your policies work as intended before real users hit production.
Learn the three essential write patterns for Delta tables in Microsoft Fabric: append, overwrite, and merge. This hands-on lesson teaches you exactly when to use each pattern — and how to avoid the silent data quality problems that come from choosing the wrong one.
Learn how to connect Power BI Desktop to a Fabric lakehouse SQL Analytics Endpoint, build a star schema with DAX measures, and publish a production-ready report — no Direct Lake required. This lesson explains every step from finding the connection string to scheduling refreshes, with clear guidance on when to use this approach over Direct Lake mode.
Learn how to use Spark notebooks in Microsoft Fabric to read CSV and Parquet files from OneLake, write clean Delta tables to the lakehouse Tables zone, and validate your results in the Lakehouse Explorer — the complete data engineering loop from raw files to queryable data. This foundation-level lesson explains every step from first principles with realistic examples and working PySpark code.
Dataflow Gen2 lets you attach multiple output destinations to a single query — run your transformation once and write results to both a Fabric Lakehouse table and a Warehouse table simultaneously. This lesson teaches you the complete pattern: from Power Query transformation to dual-destination configuration, data type gotchas, and production-ready pipeline orchestration.
Schema changes in production Delta lakehouses cascade across medallion layers in ways that break pipelines, corrupt reports, and silently produce wrong results. This deep-dive lesson teaches you exactly how Delta Lake enforces and evolves schemas, how to build drift detection into your pipelines, and how to use CHECK and NOT NULL constraints to enforce data quality where it matters.
Hardcoded file paths and table names don't belong in production notebooks. Learn how to use Fabric's parameter cell pattern to pass dynamic values from a data pipeline into PySpark, turning a one-time script into a reusable ingestion engine.
Manual refreshes don't scale. Learn how to schedule Dataflow Gen2 refreshes automatically and chain them into a Fabric Data Pipeline alongside Copy Activities and Spark Notebooks — so your data is ready before your team arrives each morning.
Learn how to build production-quality SCD Type 2 dimensions in a Microsoft Fabric Lakehouse using PySpark and Delta Lake's MERGE statement. This lesson covers the full pattern: change detection with attribute hashing, two-phase expire-and-insert MERGE logic, surrogate key generation, and Gold layer point-in-time joins — with a hands-on exercise you can run in your own environment.
Delta Lake time travel turns catastrophic data overwrites into two-minute recoveries. Learn how to query historical snapshots by version and timestamp, execute safe rollbacks with RESTORE TABLE, build a persistent audit trail, and manage retention windows so time travel is available when you actually need it.
Learn how to design and build a production-quality star schema in your Fabric Lakehouse Gold layer using PySpark — covering surrogate key strategy, dimension and fact table construction, orphan validation, and post-write optimization for Direct Lake reporting in Power BI.
Learn how to wire Fabric PySpark notebooks into a production pipeline using Notebook activities, On Success/On Failure dependencies, and mssparkutils.notebook.exit() output variables. Build a complete Bronze→Silver→Gold medallion pipeline that passes context between stages and handles failures gracefully.
Direct Lake incremental refresh isn't a checkbox — it requires aligning Delta table partitioning, TMDL framing policies, and XMLA-triggered refresh operations into a precise orchestration chain. This lesson gives you the full architecture, the code, and the troubleshooting knowledge to implement it correctly in production.
Dirty data in bronze silently corrupts silver and gold. Learn how to build a production-grade PySpark cleansing pipeline that removes exact and soft duplicates, handles nulls with intentional strategies, quarantines bad rows, and enforces data quality rules with exceptions — all structured around the medallion architecture in Microsoft Fabric.
Build production-grade error handling in Microsoft Fabric pipelines that actually catches failures, routes them intelligently, and sends actionable email alerts — across Copy, Dataflow Gen2, and Notebook activities. Learn the dependency condition model, activity output expressions, and centralized error handler patterns that keep your medallion pipelines resilient.
Unpartitioned Delta tables make Spark read everything before filtering anything. This lesson teaches you how to choose partition keys that actually improve performance, write partitioned gold-layer tables with PySpark, and verify that partition pruning is firing in both Spark and the SQL analytics endpoint.
Learn how to read Delta tables from multiple Fabric workspaces — across lakehouses and warehouses — in a single PySpark notebook. This expert lesson covers OneLake ABFS path construction, programmatic GUID resolution, multi-table joins with broadcast optimization, and writing results to a gold-layer Delta table using overwrite and MERGE patterns.
Learn how to use advanced Power Query M techniques in Dataflow Gen2 to unpivot wide source data, aggregate with Table.Group, and reshape tables into clean, typed Delta table schemas in your Microsoft Fabric lakehouse. This lesson covers the full M query structure, type-fixing patterns, and the honest trade-offs between Dataflow Gen2 and Spark for complex transformations.
Bad data that passes silently through your medallion pipeline is worse than a pipeline that fails loudly. This lesson walks you through building a reusable PySpark quality framework in Microsoft Fabric that validates row counts, null rates, and referential integrity at each layer — and wires the results into your pipeline so failures actually stop promotion.
Delta tables evolve constantly, but Direct Lake semantic models don't automatically follow. Learn how to detect schema drift between your gold-layer tables and your Power BI model, then trigger a targeted metadata refresh using Python, sempy, and the Fabric REST API — all from inside a Fabric notebook you can drop into any pipeline.
Learn how to create OneLake shortcuts in a Fabric Warehouse that point to lakehouse Delta tables — enabling T-SQL queries across both engines with zero data duplication. This lesson covers the full pattern from shortcut creation to cross-engine joins, security configuration, metadata sync, and performance tuning.
Learn how to build a production-grade test harness that runs pytest inside Microsoft Fabric Spark notebooks, validates Delta table schemas and row counts against versioned contracts, and gates medallion layer promotion via the Fabric REST API. This lesson goes beyond basic assertions to cover referential integrity, Delta version auditing, and pipeline-integrated pass/fail routing — the complete picture for data quality enforcement at scale.
Your gold layer serves Spark consumers perfectly — but your SQL analysts need pre-computed summary tables, T-SQL views, and sub-second query latency. This lesson shows you how to aggregate gold-layer Delta tables with PySpark, write results into a Fabric Warehouse via JDBC, and expose lakehouse tables as cross-engine T-SQL views — all without duplicating data or maintaining two separate pipelines.
When pipelines and Spark jobs collide under shared Fabric capacity, the result isn't graceful degradation — it's cascading failures. This deep-dive lesson teaches you exactly how Fabric's 10-minute smoothing window, throttling stages, and Capacity Admin settings work, and how to tune them to prevent job failures under real production load.
Learn how to build a complete real-time telemetry pipeline in Microsoft Fabric — from configuring an Eventstream source to defining KQL tables with update policies and materialized views, to writing operational queries against live streaming data. Uses a realistic IoT fleet monitoring scenario throughout.
Stop copying data between Fabric workspaces. This expert-level lesson shows you how to use OneLake shortcuts to expose lakehouse and warehouse tables to downstream workspaces, covering permissions, performance, warehouse security caveats, and a hands-on multi-workspace exercise. Learn the hub-and-spoke shortcut pattern used by mature Fabric platform teams.
Data Activator turns your Fabric environment into an active, event-driven platform — not just a reporting system. Learn how to define objects, properties, and trigger conditions against Power BI reports and Eventstreams, then wire up automated responses including Teams notifications and live pipeline runs.
Stop building one pipeline per data source. This deep-dive lesson teaches you to build a single, parameterized Fabric pipeline template that reads from a control table and dynamically dispatches Copy Activities, Notebook runs, and Dataflow Gen2 refreshes — with per-source retry logic, execution logging, and room to grow. If you're managing more than five data sources in Fabric, this framework will change how you work.