Fabric Data Factory Interview Questions
40 questions across architecture, Dataflow Gen2, pipeline orchestration, connectivity, and migration. Verified June 2026 — includes Copy Job, Mapping Data Flows (native, June 2026), the Modern Query Evaluator, and GA Copilot features.
Senior ETL interviews test whether you know the tool landscape expanded from two options to five in the last 18 months. The highest-signal questions: Copy Job vs Copy Activity (most candidates still describe only Copy Activity), whether SSIS/ADF Mapping Data Flows are supported (yes, natively, as of June 2026 — reverses the old “rewrite everything” answer), and the Modern Query Evaluator’s ~30% CU savings that overturned Dataflow Gen2’s reputation as slow.
Fabric Data Factory Architecture
The tool landscape expanded significantly — Copy Job joined Copy Activity, Mapping Data Flows arrived natively, and dbt now runs inside Fabric. Know when each applies.
Q1
Fabric Data Factory combines Data Pipelines (Azure Data Factory orchestration), Dataflow Gen2 (Power Query transformation, now including native Mapping Data Flows), Copy Job (standalone bulk/incremental/CDC movement), Mirroring (zero-ETL database replication), and native dbt — all built on OneLake so results land directly in shared storage without inter-service copying.
The unifying architectural principle: every tool writes to OneLake, so data produced by one tool is immediately available to Spark, T-SQL, Power BI, or any other Fabric workload without a hand-off step. This is different from Azure Data Factory, where the output typically lands in a separate storage account that other Azure services then had to be granted access to.
Whether you can name all five tools, not just Pipelines and Dataflow Gen2. Candidates who describe only two options are giving a 2023 answer to a 2026 question.
Q2
Copy Job is a standalone Fabric item, not a pipeline step. It supports bulk copy, incremental copy, and CDC replication, and auto-creates destination tables — no manual schema setup. Copy Activity is a step inside a pipeline, giving fine-grained control over parallelism and letting the copy participate in broader conditional logic.
| Dimension | Copy Job | Copy Activity |
|---|---|---|
| Item type | Standalone Fabric item | Pipeline activity |
| Schema handling | Auto-creates destination tables | Requires pre-defined or mapped schema |
| CDC support | Native, built-in | Requires custom watermark logic |
| Orchestration | Runs independently on its own schedule | Chains with other activities, conditional branches |
| Best for | Simple, high-volume, standalone movement | Copy as one step in a larger orchestrated flow |
Whether you know Copy Job exists as a separate item type, not just a faster Copy Activity. This is the single highest-signal question in this module — most candidates only know Copy Activity.
Q3
Pipelines orchestrate — control flow (If/Else, ForEach, retries), scheduling, and high-speed data movement. Dataflow Gen2 transforms — cleaning, reshaping, and merging data via a visual Power Query interface, now also supporting native Mapping Data Flows for Spark-scale logic. Use Pipelines to move and coordinate; use Dataflow Gen2 to change.
The boundary has blurred somewhat with 2026 additions: Pipelines now include a Lakehouse Utility Suite for maintenance tasks, and Dataflow Gen2 supports event-driven, streaming-style execution via the Execute Query API. But the core division — orchestration vs. transformation — still holds as the mental model.
Whether you give the crisp move-vs-change framing and then acknowledge the 2026 blurring at the edges — not just the textbook answer without current context.
Q4
Spark for TB-scale complex transformations, unstructured data processing, or Python/Scala library dependencies. Dataflow Gen2 for low-code/no-code transformations, especially with the Modern Query Evaluator narrowing the performance gap significantly — for small-to-medium datasets, Dataflow Gen2 is now a legitimate production option, not just a prototyping tool.
The 2026 nuance: with Mapping Data Flows now native inside Dataflow Gen2, some Spark-scale transformation logic that previously required a notebook can now be authored visually. Evaluate whether the transformation complexity genuinely needs code (custom Python libraries, ML feature engineering) or whether it’s expressible via MDF/Power Query patterns before defaulting to a notebook.
Whether you’ve updated your mental model given MDF’s arrival in Dataflow Gen2 — candidates who reflexively say “complex logic always means Spark” are missing the 2026 capability shift.
Q5
Yes — a Pipeline includes a “Dataflow” activity. A common pattern: Pipeline copies raw data from an on-prem source into OneLake (via Copy Job or Copy Activity), then triggers a Dataflow Gen2 to clean and load it into a Warehouse or Lakehouse table, with the pipeline handling error alerts and retries around the whole sequence.
For event-driven scenarios, the 2026 addition of the Execute Query API (Streaming) means a Dataflow can also be triggered outside a scheduled Pipeline run — external systems can call specific queries directly for near-real-time transformation without waiting for the next pipeline schedule.
Whether you know the Execute Query API as an alternative trigger path beyond Pipeline-orchestrated scheduling — this is the kind of detail that separates current knowledge from stale documentation.
Q6
The Modern Query Evaluator (GA) is a rebuilt execution engine for Power Query in Dataflow Gen2, optimizing joins, group-bys, type conversions, and complex expressions. Independent benchmarking shows roughly 30% CU savings vs Dataflow Gen1 and ~20% time savings vs standard Gen2 without the optimizer — reversing Dataflow Gen2’s historical reputation as the slowest ingestion option in Fabric.
This matters for architecture decisions made before 2025: teams that avoided Dataflow Gen2 for production workloads due to performance concerns should re-evaluate. The Modern Query Evaluator is enabled by default in current Dataflow Gen2 items; verify it’s active for older dataflows that predate the change.
Whether you can quote the specific benchmark figures (30% CU savings) and know this reverses old guidance. Candidates who still describe Dataflow Gen2 as “slow, use for small datasets only” are working from outdated performance data.
Q7
Yes — dbt jobs run natively inside Fabric as of public preview announced at Ignite 2025. dbt models are authored, orchestrated, and deployed inside Fabric with access to Fabric’s governance and monitoring, combining dbt’s version-controlled, testable SQL approach with Fabric-native CI/CD.
This changes the “Spark vs Dataflow vs SQL” decision for teams already invested in dbt workflows — they no longer need external orchestration (Airflow, dbt Cloud) to run dbt against Fabric Warehouse; the models run and are monitored inside the platform directly.
Whether you know dbt is native, not just “compatible via the T-SQL endpoint” — a distinction that matters for teams evaluating whether to bring their existing dbt project into Fabric’s governance model.
Dataflow Gen2 ETL
Output destinations, staging, the 2-tier CI/CD pricing model, Mapping Data Flows arriving natively, and the AI-powered authoring additions from FabCon 2026.
Q8
Lakehouse, Warehouse, Azure SQL Database, KQL Database, ADLS Gen2 (CSV), SharePoint (via new Site Picker), and Snowflake and Excel Files (both Preview as of FabCon 2026). This is a significant expansion from Gen1, which was largely limited to internal storage for Power BI consumption.
The new ADLS Gen2 CSV destination and Fabric Lakehouse Files (CSV) destination are particularly useful for teams that need raw CSV output for downstream Spark/Python tools rather than Delta-formatted tables — previously this required a workaround.
Whether you know Snowflake and Excel Files are newly added destinations (Preview) — this is a fast-moving area and the specific destination list signals recent hands-on use.
Q9
Staging (enabled by default) loads data into a temporary Lakehouse location before transformation logic applies. This improves performance for folding queries but adds latency for simple pass-through operations. Disable staging for specific queries when direct query folding to the source is preferred and no intermediate materialization is needed.
With the Modern Query Evaluator’s improved optimization, the performance calculus around staging has shifted somewhat — test both configurations for your specific transformation pattern rather than assuming staging is always beneficial, since the evaluator now folds more logic to the source automatically.
Whether you know staging is a tunable setting per query, not an all-or-nothing dataflow property, and that the calculus changed with the Modern Query Evaluator.
Q10
Introduced at FabCon Europe 2025: the first 10 minutes of each query evaluation are billed at 12 CU — a 25% reduction from the previous flat rate. This makes Dataflow Gen2 (CI/CD) significantly more cost-effective for the majority of queries, which complete within that window.
Combined with the Modern Query Evaluator’s execution speed improvements, most queries now both run faster and cost less per minute than before 2025. For cost estimation exercises in interviews, cite both improvements together — the CU-per-minute reduction and the reduced total minutes from faster execution compound.
Whether you can quote the specific pricing change (25% reduction, 12 CU, 10-minute threshold) — generic “it got cheaper” answers don’t demonstrate the currency of knowledge this question is probing for.
Q11
As of April 2026, creating new Dataflow Gen2 items without CI/CD and Git integration support is no longer available. All new Dataflow Gen2 items are created with CI/CD and Git integration by default. Existing classic dataflows continue working; convert them via the “Save As” feature when ready.
This closes a gap that previously required teams to explicitly opt into CI/CD-enabled dataflows — now it’s the only path for new items, which simplifies governance conversations since there’s no longer a “which type did we create” ambiguity for anything built after April 2026.
Whether you know Classic dataflows are deprecated for new creation (not fully retired) — precision matters here since existing items still function.
Q12
As of June 2026, Azure Data Factory’s Mapping Data Flow (MDF) transformations run natively inside Dataflow Gen2 — no rewrite into Power Query or Spark required. A guided Mapping Data Flow migration tool brings existing ADF MDF assets into Fabric Data Factory directly.
This directly reverses the prior Fabric guidance, which required full rewrites of MDF logic. Teams with significant existing MDF investment in Azure Data Factory should re-evaluate migration timelines — what was previously a rewrite project may now be substantially closer to a lift-and-shift with the migration tool, though full expression parity should be verified per-package before committing.
Whether you know this is a June 2026 reversal of prior guidance. This is the single most consequential update in the entire module — candidates repeating “you must rewrite MDF logic” are giving actively wrong guidance as of this date.
Q13
Both reached GA at FabCon Atlanta 2026. Pipeline Expression Copilot converts natural language descriptions into pipeline expression syntax and explains existing expressions in plain language. AI-powered Transform with a prompt lets Dataflow Gen2 users create new columns by describing the needed logic in natural language, using context from other columns in the row.
The practical value: expression syntax in Fabric pipelines (`@concat`, `@pipeline().parameters.X`, nested functions) has historically been a barrier for less experienced pipeline authors. Expression Copilot removes the need to memorize the grammar, while explaining existing expressions helps new team members understand inherited pipelines faster during onboarding or troubleshooting.
Whether you know both are GA (not Preview) as of FabCon 2026 — and can describe both directions (generate expressions, explain expressions), not just one.
Q14
Variable Library (GA since September 2025) is a centralized configuration item referenced by Pipelines, Copy Jobs, and Dataflow Gen2. Dataflows reference variables using Variable.Value and Variable.ValueOrDefault functions in Power Query — enabling dynamic substitution of workspace IDs, Lakehouse IDs, and connection paths without hardcoded values.
// Reference an environment-specific Lakehouse ID // instead of hardcoding it — resolves per deployment stage let LakehouseId = Variable.Value("TargetLakehouseId"), Source = Lakehouse.Contents(LakehouseId) in Source
Whether you know the specific function names (Variable.Value) and can explain why this eliminates manual parameter changes during Dev-to-Prod promotion — a concrete CI/CD maturity signal.
Q15
Usually a schema mismatch or timeout during the staging write. Fixes: check for special characters in column names, reduce batch sizes, or disable staging for the specific query. The 2026 addition of richer Dataflow diagnostic download makes root-cause identification significantly faster than in prior versions.
The new diagnostic download (Preview, FabCon 2026) captures end-to-end execution detail including the on-premises data gateway hop if applicable — previously, gateway-traversing dataflows were the hardest to diagnose because gateway-side logs weren’t correlated with the dataflow’s own error output. See the Dataflow Gen2 Error 20302 fix guide for the step-by-step resolution.
Whether you know about the enhanced diagnostic download capability — a candidate who only describes the old trial-and-error approach (batch size guessing) hasn’t used the current troubleshooting tools.
Q16
Incremental refresh to Lakehouse destinations reached GA (September 2025) — configure natively rather than hand-rolling RangeStart/RangeEnd parameters for that scenario. For Warehouse destinations, Dataflow Gen2 still performs Replace or Append only — no native Upsert. For Upserts, land data in a staging table via Dataflow, then run a T-SQL MERGE via a Pipeline Script Activity.
The MERGE statement itself is now GA in Fabric Warehouse (see the Data Warehouse interview guide), so this staging + MERGE pattern is more reliable than it was when MERGE required separate INSERT/UPDATE/DELETE workarounds.
Whether you know incremental refresh to Lakehouse is now native GA — distinct from the still-manual Warehouse Upsert pattern. Conflating the two reveals imprecise knowledge of destination-specific capabilities.
Pipelines & Activities
Orchestration activities including the new Lakehouse Utility Suite and Materialized Lakeview Refresh, plus Conditional Retries and control flow fundamentals.
Q17
Copy Activity is a pipeline step supporting 100+ connectors, optimized for throughput, moving data from Source to Sink without transformation logic. Unlike the standalone Copy Job, it runs as part of a larger orchestrated pipeline where you need conditional branching, retries chained with other steps, or coordination with non-copy activities.
Whether you consistently distinguish Copy Activity from Copy Job throughout the interview, not just when directly asked to compare them.
Q18
Lookup retrieves actual data rows (e.g., config table values, watermark timestamps). Get Metadata retrieves structural file information (name, size, last modified date) — used to iterate over a folder’s contents with a ForEach loop, without reading the file contents themselves.
Whether you can immediately map each activity to its canonical use case (Lookup → control table read, Get Metadata → file enumeration) without hesitation.
Q19
The Lakehouse Utility Suite (Preview) brings purpose-built pipeline activities for operationalizing Lakehouse maintenance — the Lakehouse Maintenance activity schedules OPTIMIZE, VACUUM, and other upkeep tasks directly inside a pipeline, replacing manual scripts or notebook-based maintenance jobs.
Before this, teams typically wrote a Spark notebook with hard-coded OPTIMIZE/VACUUM calls and scheduled it separately from their main ETL pipelines — creating a maintenance workflow disconnected from the pipeline that actually loads the data. The Lakehouse Maintenance activity brings this into the same pipeline canvas, alongside a Materialized Lakeview Refresh activity for keeping materialized lake views current as part of pipeline runs.
Awareness of this 2026 addition — it’s a signal that a candidate is actively engaged with the platform’s operational tooling, not just its data movement features.
Q20
ForEach iterates over a collection (a list of tables), executing activities per item — parallel or sequential. Switch evaluates an expression and routes execution to one specific branch based on the result, similar to a CASE statement in SQL.
Whether you can immediately identify which pattern fits a described scenario (e.g., “process 50 tables the same way” → ForEach; “route based on file type” → Switch).
Q21
The Notebook activity calls a Fabric Spark Notebook. Dynamic parameters pass from the pipeline into the notebook via the parameters cell toggle in notebook settings — enabling a single notebook to be reused across multiple datasets or environments rather than duplicating notebook code.
Whether you know the parameters cell mechanism specifically, as the pattern for making notebooks reusable across pipeline invocations.
Q22
Invoke Pipeline lets a parent pipeline trigger child pipelines, enabling modular design — a “Master Orchestrator” pipeline calling separate “Ingest” and “Process” child pipelines rather than one monolithic pipeline handling everything. This improves maintainability and allows independent testing/deployment of each child.
Whether you connect Invoke Pipeline to modular architecture principles, not just describe the mechanical trigger relationship.
Q23
Pipeline Conditional Retries (2026, Build) let you retry specific activities based on the actual failure condition, rather than applying a blanket retry policy to every activity regardless of the error type. This avoids wasting retry attempts on non-transient failures (e.g., a permissions error, which won’t resolve on retry) while still retrying genuinely transient ones (network blips, throttling).
Previously, a fixed retry count/interval applied uniformly meant permission errors and network timeouts got the same treatment — burning through retry attempts on failures that would never succeed regardless of retry count, delaying the eventual failure notification.
Whether you understand the operational cost of blanket retry policies (wasted time before failure notification on permanent errors) and know Fabric now addresses this with conditional logic.
Connectivity & Gateways – Fabric Data Factory Interview Questions
On-prem and VNet gateways, credential management, and the new Outbound Access Protection security model for regulated environments.
Q24
On-Premises Gateway: installed software on a local VM/server, bridges your local network to Fabric without opening inbound ports. VNet Data Gateway: a managed service injected into your Azure Virtual Network — used to connect to Azure PaaS services (SQL Managed Instance, etc.) secured behind Private Endpoints.
Whether you can immediately map “on-premises source” to On-Prem Gateway and “Azure PaaS behind Private Endpoint” to VNet Gateway without hesitation.
Q25
Install the gateway on multiple servers clustered together. This removes the single point of failure — if one gateway node goes down, Fabric routes traffic to remaining active nodes in the cluster automatically. Also applies to gateway support for Invoke Pipeline and Semantic Model Refresh activities (September 2025), broadening which activity types benefit from gateway HA.
Whether you know gateway HA now extends to Invoke Pipeline and Semantic Model Refresh, not just Copy activities — a scope expansion from September 2025.
Q26
Credentials are stored in the Fabric “Manage Connections and Gateways” settings, but best practice is storing secrets in Azure Key Vault and referencing them in the connection definition — ensuring automatic password rotation without touching pipeline definitions when credentials change.
Whether you default to Key Vault referencing rather than storing credentials directly in connection settings, even though the latter technically works.
Q27
Outbound Access Protection (OAP) ensures Pipelines, Dataflows Gen2, and Copy Jobs connect only to explicitly approved destinations, reducing the risk of data exfiltration. Combined with private endpoints and workspace-level policies, it lets regulated organizations run sensitive workloads across dev/test/prod with defense-in-depth controls.
Related 2026 addition: secure ingestion into Fabric Data Warehouse via Copy Job and Copy Activity now supports the most locked-down network topologies — Warehouse fronted by a workspace private link, workspaces with OAP enabled, routed through a VNet or on-premises gateway. This closes a gap that previously forced regulated customers into less secure workarounds for warehouse ingestion.
Whether you know OAP by name and can connect it to the broader 2026 secure-ingestion story for private-link-fronted Warehouses — a signal of enterprise security awareness beyond basic gateway setup.
Q28
Common causes: outbound firewall blocking, outdated gateway version. Ensure port 443 (HTTPS) and required Azure Service Bus ports are open outbound. Use the Diagnostics tool in the Gateway app to test connectivity. For Dataflow Gen2 jobs traversing a gateway, the 2026 enhanced OPDG diagnostics provide richer end-to-end logs that correlate gateway-side failures with dataflow errors.
Whether you know about the enhanced OPDG diagnostics specifically for Dataflow Gen2 — previously the hardest failure mode to root-cause because gateway logs and dataflow logs weren’t correlated.
Q29
Native connectors exist for AWS S3 and GCS — no gateway required for these public cloud services. Authentication uses Access Keys or Identity Federation. Multi-cloud support (AWS, GCP, hybrid) is designed to work within the same pipeline, so a single orchestration can span sources across providers.
Whether you know gateways are unnecessary for these specific public cloud connectors — a common point of confusion for candidates who assume all external connections require a gateway.
Advanced Scenarios
Parameterization, metadata-driven ingestion at scale, error handling, and the emerging agentic tooling layer — Data Factory MCP Server and Data Factory Skills.
Q30
Define parameters at the pipeline level (e.g., TableName, DateRange) and reference them via the expression language: @pipeline().parameters.TableName. This lets one pipeline serve multiple tables or environments without duplication.
@concat('landing/', formatDateTime(utcnow(), 'yyyy/MM/dd'), '/file.csv')
With Pipeline Expression Copilot now GA, complex expressions like nested conditionals no longer require memorizing exact syntax — describe the logic in natural language and Copilot generates the expression, though understanding the underlying grammar remains valuable for debugging generated output.
Whether you can write a correct expression from memory and also know Copilot now assists with this — both matter, since Copilot-generated expressions still need to be understood, not just accepted blindly.
Q31
Store the list of tables to copy in a control table. Pipeline does a Lookup to retrieve the list, then a ForEach loop iterates through it, invoking a parameterized Copy Job or Copy Activity per table. This scales to hundreds of tables from a single pipeline definition rather than one pipeline per table.
With Copy Job’s auto-schema-creation, this pattern is now simpler than before — the control table drives which tables to copy, and Copy Job handles destination schema without requiring pre-mapped column definitions per table, reducing the metadata the control table itself needs to carry.
Whether you connect the metadata-driven pattern to Copy Job’s schema auto-creation as a simplification — showing you understand how newer tools reduce the complexity of established patterns.
Q32
Every activity has a retry policy (e.g., 3 attempts, 30-second interval) for transient failures. For failures that need human attention, connect the “On Failure” path to a Web Activity calling a Logic App or Teams webhook. With Pipeline Conditional Retries (2026), differentiate transient errors (worth retrying) from permanent ones (fail fast, alert immediately).
Whether you distinguish blanket retry policies from the newer conditional approach, and know when fail-fast (no retry) is actually the correct behavior.
Q33
CSV files sometimes contain invisible Byte Order Marks (BOM) that corrupt the first column header. Fix: in Copy Activity Source settings, confirm encoding is “UTF-8” (not “UTF-8 with BOM”). See the CSV Column Not Found fix guide for the full resolution walkthrough.
Whether you recognize this specific, easily-missed encoding gotcha rather than treating it as a generic “column not found” error requiring broader debugging.
Q34
The Data Factory MCP Server (Preview) exposes Dataflow Gen2 and pipeline capabilities — dataflow creation, M/Power Query authoring, connection management, query execution, refresh orchestration — as tools AI assistants can call directly from VS Code, Claude, ChatGPT, Gemini, or the command line. Data Factory Skills extend this coverage to Pipelines, Copy Job, and Connections & Gateways. Airflow support was added at Build 2026, extending agent-driven authoring to code-first Airflow DAGs.
This is Fabric’s agentic tooling strategy applied to the ETL layer specifically — an AI assistant with MCP access can author a dataflow, run a diagnostic query, or trigger a refresh without a human navigating the Fabric UI directly, useful for automated pipeline generation or AI-assisted debugging workflows.
Whether you’re aware of the agentic direction of the platform beyond Copilot-in-UI features — the MCP Server represents programmatic AI access, a distinct capability from conversational assistance.
Q35
A pipeline in Workspace A can trigger a Notebook in Workspace B if the calling identity has permissions, but architecturally cleaner is keeping orchestration local to a workspace. For genuine cross-workspace dependency, use Eventstream or a lightweight webhook/Logic App as the coordination layer rather than deeply coupling pipelines across workspace boundaries.
Whether you recognize cross-workspace pipeline coupling as an architectural smell to avoid where possible, not just a mechanically supported capability.
Migration & SSIS/ADF
The most significantly changed module — Mapping Data Flow migration tooling reverses the prior “rewrite everything” guidance for ADF customers.
Q36
SSIS Integration Runtime is still not supported in Fabric — the core answer hasn’t changed, you cannot lift-and-shift SSIS packages directly. What has changed: for the transformation logic portion, Mapping Data Flows (which many SSIS migrations first moved to in ADF) now run natively in Dataflow Gen2, closing part of the gap. Custom .NET script components still require a rewrite, typically to Spark Notebooks.
The realistic migration path in 2026: SSIS → (if not already done) ADF Mapping Data Flows as an intermediate step → native Fabric Dataflow Gen2 via the MDF migration tool. Organizations still on raw SSIS packages face more rewrite work than those who’d already modernized to ADF MDF.
Precision: SSIS itself is still unsupported, but the MDF layer many SSIS migrations pass through is now natively supported. Candidates who say “nothing changed” or “everything’s supported now” are both wrong in different ways.
Q37
Pipeline JSON definitions are similar enough that you can often copy ADF pipeline JSON and paste into the Fabric Pipeline JSON editor, adjusting for unsupported activities. For Mapping Data Flow assets specifically, the guided MDF migration tool (June 2026) provides a much closer to 1-click experience than the general pipeline migration story.
Separate the two migration surfaces in your answer: pipeline orchestration logic (still largely manual JSON porting with adjustments) vs. MDF transformation logic (now has dedicated tooling). Conflating them into a single “migration is easy now” or “migration is still hard” answer misses the nuance.
Whether you separate the pipeline-JSON migration path (still manual) from the MDF-specific tooling (guided, June 2026) rather than treating ADF migration as a single undifferentiated problem.
Q38
Export the .pqt (Power Query Template) file from Gen1 and import into Fabric Dataflow Gen2. Configure the new Output Destination (Lakehouse, Warehouse, or one of the expanded 2026 destinations) since Gen1 typically only wrote to internal Power BI storage.
Whether you know the specific file format (.pqt) and remember the Output Destination configuration step, which Gen1 users often forget since Gen1 didn’t require it.
Q39
Don’t copy everything. Use Shortcuts to expose historical data in OneLake instantly (zero-copy). Build Pipelines (or Copy Jobs) only to move “active” or “new” data going forward. This significantly reduces migration time and storage duplication cost versus copying the full history upfront.
For CDC-based ongoing sync after the initial migration, evaluate Mirroring over custom incremental pipeline logic — Mirroring’s zero-ETL CDC replication (now including “open mirroring” integration with Stelo for streaming change data without pipeline rebuilds) handles the ongoing sync automatically once the historical shortcut is in place.
Whether you sequence the migration correctly: Shortcut for instant historical access, then Mirroring or Copy Job for the ongoing delta — rather than a single “copy everything” approach.
Q40
Connect the workspace to Git (Azure DevOps or GitHub). Commit Pipeline and Dataflow definitions — now default for all new Dataflow Gen2 items since April 2026. Use Deployment Pipelines to promote Dev → Test → Prod, combined with the Variable Library so environment-specific values (workspace IDs, connection strings) swap automatically rather than requiring manual edits per stage.
For pipelines specifically, 2026 added Connection and Item reference support — cleaner, more portable references that simplify dev-to-prod promotion without needing to manually re-point every connection reference after each deployment.
Whether you combine Git + Deployment Pipelines + Variable Library as a complete CI/CD story, and know about the newer Connection/Item reference portability improvements rather than describing an incomplete 2023-era workflow.
ETL done. Eventstreams, KQL Databases, and streaming analytics are next.
Start Real-Time Questions →
