Microsoft Fabric Real-Time Intelligence
Interview Questions

Real-Time Intelligence is a Fabric workload designed for high-volume, low-latency data streams (IoT, logs, telemetry). Specifically, it consists of three main artifacts: Eventstreams for ingestion and routing, KQL Databases (Eventhouse) for storage and querying, and Data Activator for automated actions (Reflex).

Use a KQL Database for time-series data, logs, and telemetry where you need sub-second query performance on billions of rows. In contrast, use a Lakehouse for general-purpose batch analytics and data warehousing optimized for structured analytics with Delta Lake ACID support.

Azure Stream Analytics (ASA) is a PaaS offering primarily for stream processing logic. Fabric Real-Time Intelligence, however, integrates ingestion, storage, and analytics using the Kusto (ADX) engine. Therefore, Fabric is preferred for integrated analytics where data needs to be stored and queried immediately, whereas ASA focuses on in-flight processing.

Specifically, an Eventhouse is a Fabric item that hosts one or more KQL databases and manages their compute and policies, similar to an Azure Data Explorer cluster. Consequently, it allows you to manage capacity and sharing policies efficiently across a group of related databases.

Yes. KQL Databases persist data in OneLake and maintain a hot cache on local SSD compute for fast queries. Furthermore, they expose this data through OneLake in Delta Parquet format. This “OneLogicalCopy” feature allows Spark and Power BI (Direct Lake) to query KQL data without physical duplication.

Fabric Eventstreams and KQL Databases support Streaming Ingestion. This mechanism can make data available for query within seconds (often < 10s) of arrival. As a result, it is significantly faster than the batch ETL processes typically used in the Data Warehouse.

Eventstreams can ingest data from diverse sources, including Azure Event Hubs, Azure IoT Hub, Custom Apps (via API), and Change Data Capture (CDC) streams. It creates a “no-code” pipe to route this data. Consequently, you can connect real-world data sources without writing complex integration code.

You can route data to multiple destinations simultaneously. These include a KQL Database (for analytics), a Lakehouse (for archival), or a Custom App. This fan-out capability is key for Lambda architectures where data needs to be both analyzed immediately and archived for later batch processing.

Eventstreams include a “Processing Editor” (Derived Streams). You can apply lightweight transformations like filtering, aggregation (Group By), or field renaming before the data lands in the destination. This helps reduce storage costs and improves data quality at the source.

Use the Azure Event Hubs source when you already have infrastructure sending data to an existing Hub. In contrast, use the Custom App source when you want Fabric to provide a connection string endpoint. This allows your application to send data directly, eliminating the need to manage a separate Azure Event Hub resource.

Eventstreams natively support JSON, Avro, and CSV formats. JSON is the most common for modern telemetry. However, if data arrives in a binary format, you may need a custom pre-processor before sending it to the Eventstream.

When an Eventstream encounters a malformed message (e.g., bad JSON), it can be configured to drop the message or route it to a “Dead Letter” destination. Configuring this is critical for production pipelines to ensure that one bad message does not block the entire stream.

A KQL Database is a scalable store optimized for streaming data like logs and time-series. It uses the Kusto engine to index every column by default. Consequently, this enables extremely fast text search and aggregation over massive datasets without complex tuning.

Hot Cache stores data on local SSDs of the compute nodes for instant query performance but is more expensive. Cold Storage stores data in OneLake (Blob) for long-term retention, which is cheaper but slower. You define a “Caching Policy” to balance cost vs. speed effectively.

The Retention Policy dictates how long data is kept before permanent deletion. For example, you can set a Soft Delete period of 365 days. After this period, data is removed to free up storage. Importantly, this is distinct from the Caching Policy, which only controls performance.

The dynamic data type in KQL is used to store JSON objects or arrays. Unlike SQL `VARCHAR`, KQL can index the internal properties of a dynamic column. Therefore, you can query nested JSON fields (e.g., `Context.Device.ID`) directly without complex parsing logic.

An Update Policy acts like a trigger. When data lands in a source table (e.g., RawLogs), the policy runs a KQL query to transform the data and insert it into a target table (e.g., ProcessedLogs). This is the primary mechanism for performing ELT transformations inside a KQL Database.

Materialized Views in KQL pre-compute aggregations (e.g., `summarize count() by Hour`) as new data arrives. They are essential for deduplicating data (`arg_max()`) and improving performance for dashboards that query massive historical datasets repeatedly.

SQL is optimized for transactional operations and relational joins. In contrast, KQL is optimized for append-only streams, time-series analysis, and free-text search. KQL syntax is pipe-based (`|`), making it more intuitive for exploratory data analysis than SQL.

KQL Databases automatically expose their data to OneLake. This means you can open a Fabric Notebook and read KQL tables using Spark SQL, or open Power BI and use Direct Lake mode against KQL data without any physical data movement.

Use the pipe operator `|`. Example: `Table | where State == ‘TX’ | sort by Timestamp desc`. This is the most basic pattern you must know. The `where` clause filters rows, and `sort by` orders them.

Project selects specific columns to keep in the output (like SQL SELECT). Summarize aggregates data (like SQL GROUP BY). Example: `| summarize count() by City` groups rows by city and counts them.

The `make_series` operator converts discrete data points into a continuous time series array. It allows you to fill gaps (missing values) with default values (like 0). Consequently, it is essential for rendering charts that need a value for every time bucket.

If a column is stored as a string but contains JSON, use `parse_json()`. Example: `extend Props = parse_json(PropertiesColumn) | project Props.DeviceID`. However, if the column is already `dynamic` type, you can access fields directly using dot notation.

Use the `join` operator. Example: `TableA | join kind=inner (TableB) on ID`. However, in KQL, joins can be expensive. Therefore, for performance, it is often better to denormalize data during ingestion or use the `lookup` operator for small dimension tables.

KQL excels at time analysis. You can use `bin(Timestamp, 1h)` inside a summarize clause to bucket data into 1-hour windows. This is the standard way to generate histograms or trend charts for IoT data.

Data Activator is a “no-code” tool in Fabric designed to automatically take action when specific patterns are detected in data. It creates items called Reflexes that monitor data streams (from Eventstreams or Power BI) and trigger alerts or workflows.

A Reflex consists of Objects (what you monitor, e.g., “Freezers”), Properties (data fields, e.g., “Temperature”), and Triggers (conditions, e.g., “Temp > 30”). When a Trigger condition is met, it automatically fires an Action.

Power BI Alerts are simple thresholds on dashboard tiles. In contrast, Data Activator Triggers are much more powerful; they run on the backend data stream independent of reports, support complex logic (e.g., “value stays high for 15 mins”), and can trigger external systems via Power Automate.

Standard actions include sending an Email (Outlook) or a Teams message. For advanced integration, you can trigger a Custom Action which calls a Power Automate flow. Consequently, this allows you to perform virtually any task (e.g., create a Jira ticket, call an API).

Reflex items have a built-in monitoring tab where you can see the history of trigger evaluations and successful/failed actions. This is crucial for debugging why an alert did not fire as expected during testing.

Data Activator evaluates conditions in near-real-time as data arrives in the Eventstream or Power BI dataset. While fast, it is not a “hard real-time” system (microseconds). It is designed for operational latencies (seconds to minutes).

Scenario: 10,000 sensors sending temperature data every second.
Solution: Ingest via Event Hubs into Fabric Eventstreams. Route raw data to a KQL Database for time-series analysis. Create a Reflex to alert if temperature exceeds 50°C. Use Power BI Direct Lake on KQL for the dashboard.

Scenario: Migrating application logs from Splunk/ELK to Fabric.
Solution: Use KQL Databases. KQL is highly optimized for text search and log patterns. It offers a cost-effective alternative with separate Hot/Cold caching policies compared to expensive index-heavy solutions.

Compute (KQL) and Storage (OneLake) are billed separately. To optimize, reduce the “Hot Cache” retention period (e.g., from 30 days to 7 days) for data that isn’t queried frequently. Additionally, ensure efficient encoding on ingestion to reduce storage size.

Fabric manages availability at the capacity level. However, KQL compute currently runs in a single region per workspace. Therefore, for multi-region HA, you would need to implement dual-ingestion architectures to handle region-wide outages.

KQL Databases support Row-Level Security (RLS) similar to SQL. You can define a policy function that filters data based on the `current_principal()` (user). This is critical when sharing IoT dashboards with different customers.

KQL has built-in plugins for anomaly detection and forecasting (`series_decompose_anomalies`). You can run these functions directly in the query as part of the dashboard. This provides real-time predictive insights without moving data to Spark.