How are the operational aspects of Lakeflow Declarative Pipelines different from Spark Structured Streaming ?

• A. Structured Streaming can process continuous data streams, while Lakeflow Declarative Pipelines cannot.
• B. Lakeflow Declarative Pipelines can write to Delta Lake format, while Structured Streaming cannot.
• C. Lakeflow Declarative Pipelines manage the orchestration of multi-stage pipelines automatically, while Structured Streaming requires external orchestration for complex dependencies.
• D. Lakeflow Declarative Pipelines automatically handle schema evolution, while Structured Streaming always requires manual schema management.

A data engineer is designing a Lakeflow Declarative Pipeline to process streaming order data. The pipeline uses Auto Loader to ingest data and must enforce data quality by ensuring customer_id and amount are greater than zero. Invalid records should be dropped.
Which Lakeflow Declarative Pipelines configurations implement this requirement using Python?

• A. @dlt.table
  @dlt.expect( " valid_customer " , " customer_id IS NOT NULL " )
  @dlt.expect( " valid_amount " , " amount > 0 " )
  def silver_orders():
  return dlt.read_stream( " bronze_orders " )
• B. @dlt.table
  def silver_orders():
  return (
  dlt.read_stream( " bronze_orders " )
  .expect_or_drop( " valid_customer " , " customer_id IS NOT NULL " )
  .expect_or_drop( " valid_amount " , " amount > 0 " )
  )
• C. @dlt.table
  def silver_orders():
  return (
  dlt.read_stream( " bronze_orders " )
  .expect( " valid_customer " , " customer_id IS NOT NULL " )
  .expect( " valid_amount " , " amount > 0 " )
  )
• D. @dlt.table
  @dlt.expect_or_drop( " valid_customer " , " customer_id IS NOT NULL " )
  @dlt.expect_or_drop( " valid_amount " , " amount > 0 " )
  def silver_orders():
  return dlt.read_stream( " bronze_orders " )

The business reporting tem requires that data for their dashboards be updated every hour. The total processing time for the pipeline that extracts transforms and load the data for their pipeline runs in 10 minutes.
Assuming normal operating conditions, which configuration will meet their service-level agreement requirements with the lowest cost?

• A. Configure a job that executes every time new data lands in a given directory.
• B. Schedule a job to execute the pipeline once hour on a new job cluster.
• C. Schedule a jo to execute the pipeline once and hour on a dedicated interactive cluster.
• D. Schedule a Structured Streaming job with a trigger interval of 60 minutes.

A data engineer is building a Lakeflow Declarative Pipelines pipeline to process healthcare claims data. A metadata JSON file defines data quality rules for multiple tables, including:
{
" claims " : [
{ " name " : " valid_patient_id " , " constraint " : " patient_id IS NOT NULL " },
{ " name " : " non_negative_amount " , " constraint " : " claim_amount > = 0 " }
]
}
The pipeline must dynamically apply these rules to the claims table without hardcoding the rules.
How should the data engineer achieve this?

• A. Use a SQL CONSTRAINT block referencing the JSON file path.
• B. Load the JSON metadata, loop through its entries, and apply expectations using dlt.expect_all.
• C. Reference each expectation with @dlt.expect decorators in the table declaration.
• D. Invoke an external API to validate records against the metadata rules.

A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non- overlapping five-minute interval. Incremental state information should be maintained for 10 minutes for late- arriving data.
Streaming DataFrame df has the following schema:
" device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT "
Code block:

Choose the response that correctly fills in the blank within the code block to complete this task.

• A. delayWrite( " event_time " , " 10 minutes " )
• B. awaitArrival( " event_time " , " 10 minutes " )
• C. withWatermark( " event_time " , " 10 minutes " )
• D. slidingWindow( " event_time " , " 10 minutes " )
• E. await( " event_time + '10 minutes ' " )