PDE - Maintaining and Automating Data Workloads (18% of the exam) - Section 5.2

Implement automation and repeatability using Cloud Composer DAGs for scheduling and orchestrating data workloads.

Implement repeatable data workloads by authoring Cloud Composer DAGs to schedule, retry, and chain dependent jobs, and select Google Cloud Workflows for lightweight event-driven orchestration that does not require a full Airflow environment.

Cloud ComposerDAGsWorkflowsJob scheduling

Practice question for this objective

Free sampleMaintaining and Automating Data Workloadsmedium

A platform team must automate two distinct flows. The first chains roughly forty interdependent Spark and BigQuery steps every night, needs per-task retries and visibility into each step, and must back-fill missed dates after an outage. The second reacts to a single file arriving in Cloud Storage by calling three REST APIs in sequence with light branching, finishing in seconds, and the team wants no always-on orchestration cost for it. Which TWO service choices, each matched to the right flow, make both workloads repeatable while fitting their orchestration needs? (Select TWO.)

AUse Cloud Composer for the forty-step nightly pipeline, so the DAG expresses the inter-step dependencies, provides per-task retries, exposes per-step run history, and can back-fill missed logical dates after an outage. Correct
BUse Cloud Composer for the event-driven file flow as well, keeping the environment running continuously so the same orchestrator handles both the nightly pipeline and the quick API calls under one tool.
CUse Cloud Scheduler alone for the forty-step nightly pipeline, defining one cron job per step and ordering them purely by staggered cron times so each step fires after the previous one is expected to finish.
DUse Google Cloud Workflows for the file-triggered flow, invoking it from an Eventarc trigger on the Cloud Storage event so it sequences the three REST calls with light branching and bills only per execution. Correct
EUse Google Cloud Workflows for the forty-step nightly pipeline too, modelling every Spark and BigQuery dependency in a single workflow definition so both flows share one serverless orchestration engine.

Match Cloud Composer to long interdependent scheduled data pipelines and Google Cloud Workflows to short event-driven API sequences so each workload is repeatable at the right cost. Cloud Composer runs managed Airflow, whose DAGs give per-task retries, dependency chaining, per-step history, and catch-up back-filling that a forty-step nightly data pipeline needs to be reproducible. Google Cloud Workflows is serverless and billed per execution, so an Eventarc-triggered Workflows definition sequences a few REST calls with branching in seconds without paying for idle orchestration capacity. Pairing each flow with the orchestrator built for its shape is what makes both repeatable at the right operational cost.

Why A is correct: Cloud Composer runs managed Airflow, whose DAGs model many interdependent steps with per-task retries, a per-step run history, and catch-up back-filling of missed logical dates, which matches every requirement of the long nightly pipeline.

Why B is wrong: Consolidating on one tool is tempting, but a Composer environment runs continuously and bills for that capacity, which is the wrong fit for a seconds-long event-driven flow where the team explicitly wants no always-on orchestration cost.

Why C is wrong: Cloud Scheduler can fire jobs on a cron cadence, but staggering forty cron times does not express real dependencies, retries, or back-fill, so a slow or failed step would let later steps run anyway, which fails the ordered nightly pipeline.

Why D is correct: Workflows is a serverless orchestrator that sequences REST and API calls with conditional branching and charges per step executed rather than for idle capacity, and an Eventarc trigger on the object event starts it, which fits the seconds-long event-driven flow with no always-on cost.

Why E is wrong: Workflows suits short API-style sequences, but it lacks the rich DAG scheduling, catch-up back-fill, and per-task operational tooling that a forty-step data pipeline with missed-date recovery needs, so it is the wrong engine for the heavy nightly flow.

See more PDE practice questions, answers explained.

Exam traps in Maintaining and Automating Data Workloads

Answers that look right on this material and are not. Each one is a distractor from a different question in the PDE bank for this domain.

Workflows uses Python DAG files and runs an Airflow scheduler, while Cloud Composer uses a YAML step definition and is the better fit for long-running batch data pipelines with many operators.
Why it is wrong: Reverses the two products. Workflows uses a YAML or JSON step definition, and Cloud Composer is the Airflow-based offering with Python DAGs that suits multi-operator data pipelines.
Increase the DAG-level dagrun_timeout and rely on Composer to re-run failed DAG runs from the scheduler so that the ingest task is attempted again on the next day's run.
Why it is wrong: Extending dagrun_timeout only changes how long a run may take before being killed, and the next scheduled run is a separate DAG run rather than a retry of the failed task, so the requirement for three retries within the same run is not met.
Set retries to 4 on the Dataflow task with a static retry_delay, and rely on Dataflow's built-in deduplication of Pub/Sub message ids to ensure no duplicate writes to BigQuery.
Why it is wrong: Pub/Sub deduplication only applies inside a single streaming job's lifetime and does not protect a batch job's BigQuery writes across separate task attempts, so duplicate output is still possible if a previous attempt partially succeeded.

Examworthy is not affiliated with or endorsed by Google Cloud. Original, blueprint-aligned practice material only.