Data-Engineer-Associate - Troubleshooting, Monitoring, and Optimization (10% of the exam) - Section 6.2

Use the Lakeflow Jobs UI to monitor pipeline health by interpreting job statuses, viewing DAG task graphs to spot upstream blockers, and tracking run times and failure rates.

Interpret job and task statuses, trace a failed or skipped task back to its upstream blocker in the task graph, repair a failed run instead of rerunning everything, and track failure rates with notifications.

job statusestask graphrepair runupstream failuresjob notifications

Practice question for this objective

Free sampleTroubleshooting, Monitoring, and Optimizationmedium

A Lakeflow Job has six tasks arranged as a chain of dependencies. In the run detail for last night's run, the task graph shows the third task in a failed state, while the two tasks below it are greyed out and reported as skipped because an upstream dependency did not succeed. An on-call engineer must decide where to look for the fault. Which reading of that run is correct?

  • AEach skipped task met an error of its own, so the engineer should read the logs of all three tasks together before deciding what broke the run.
  • BThe failed state was propagated up the graph from a task below it, so the fault sits downstream and the third task's own logs will be empty.
  • CThe failed task is the one that ran and errored, so its logs carry the fault, while the skipped tasks were never dispatched and hold no output of their own. Correct
  • DAll six tasks executed, and the two greyed out tasks finished with warnings, so their target tables may hold partly written output from the run.
In a job task graph, a skipped task never started; the failed task upstream of it holds the run's cause. Task state in Lakeflow Jobs flows from a dependency to its dependants. A task whose dependency did not succeed is marked skipped and is never dispatched to compute, so it produces no logs and no output. Only the task that actually ran and errored records the failure, which makes it the single place to read for the cause of the run.

Why A is wrong: Tempting because a run that ends unsuccessfully colours several tasks at once in the graph, but a skipped task was never dispatched to compute, so it met no error and wrote no logs of its own.

Why B is wrong: Tempting because status does travel between tasks, but it travels downstream from a dependency to its dependants, never upstream, and a failed task always has logs from its own attempt.

Why C is correct: Correct: Lakeflow Jobs marks a task skipped when a dependency did not satisfy its Run if condition, so the failed task is the single place the run's cause is recorded.

Why D is wrong: Tempting because greyed styling can read as a degraded finish, but skipped is a distinct terminal state meaning the task never started, so it wrote nothing to its targets.

See more Data-Engineer-Associate practice questions, answers explained.

Exam traps in Troubleshooting, Monitoring, and Optimization

Answers that look right on this material and are not. Each one is a distractor from a different question in the Data-Engineer-Associate bank for this domain.

  • The transform and publish tasks each ran and raised their own errors after reading partial output written by ingest, so the three task logs have to be read together before a cause emerges.

    Why it is wrong: Tempting because a partial write really can poison downstream logic, but the upstream failure status is reported precisely when a task is not launched, so neither downstream task produced an error of its own.

  • Add a notification on run start, so the team can note the start time of each run and work out by hand whether it has overrun.

    Why it is wrong: Tempting because a start notification does arrive during the run, but it carries no information about how long the run has taken and shifts the watching back onto a person.

  • Set a timeout at the duration that counts as slow, since exceeding it raises the notification, and a run is cancelled only when a second and higher timeout value is also configured on the same job.

    Why it is wrong: This is tempting because both settings are configured in the same part of the job definition and both are expressed as a duration, but a job carries one timeout and reaching it cancels the run, so this choice stops exactly the runs the team wants to let finish.

Examworthy is not affiliated with or endorsed by Databricks. Original, blueprint-aligned practice material only.