ExamworthyExamworthy
Open the appStart free

PDE - Ingesting and Processing Data - Section 2.4

Deploy and operationalise pipelines using Cloud Composer and Workflows for job automation, and implement CI/CD for data pipelines.

Deploy data pipelines using Cloud Composer DAGs and Apache Airflow operators or Google Cloud Workflows for serverless step orchestration, and configure CI/CD pipelines to test and promote pipeline code across environments reliably.

Cloud ComposerWorkflowsCI/CDApache Airflow

Practice question for this objective

Free sampleIngesting and Processing Datamedium

A data platform team is choosing between Cloud Composer and Workflows to orchestrate their data pipelines. They have a daily batch flow with thirty interdependent tasks across BigQuery, Dataflow, and Dataproc, plus a need for backfills, conditional branching on data quality results, and rich operator libraries maintained by an open-source community. Which statement best characterises the role boundary between the two services for this workload?

  • AWorkflows is the better fit because it provides native operators for BigQuery, Dataflow, and Dataproc and supports backfills with a DAG scheduler maintained by the open-source community.
  • BCloud Composer and Workflows are interchangeable for batch data pipelines because both expose DAGs, backfills, and Airflow operators behind a managed runtime.
  • CWorkflows is the better fit because long-running, stateful data pipelines with thirty tasks are its primary use case, while Cloud Composer is intended for short HTTP service orchestrations.
  • DCloud Composer is the better fit because it runs managed Apache Airflow, which provides the operator ecosystem, DAG backfills, and branching primitives such as BranchPythonOperator that suit this multi-step data pipeline. Correct
Recognise that Cloud Composer is managed Apache Airflow for data pipeline orchestration, while Workflows targets lighter service and event orchestration. Cloud Composer is a managed Apache Airflow service, so it inherits Airflow's scheduler, DAG model, backfills, branching operators, and the large community library of Google Cloud operators. Workflows is a separate product for orchestrating HTTP services and Google Cloud connectors via a steps-based definition; it does not expose Airflow concepts such as DAG runs or backfills, so a thirty-task interdependent batch pipeline maps more naturally to Composer.

Why A is wrong: Workflows does have connectors for Google Cloud services, but it is not built on Apache Airflow, it has no concept of DAG backfills, and its connector catalogue is not an open-source community library. The candidate may be confusing Workflows connectors with Airflow operators.

Why B is wrong: The services are not interchangeable. Only Cloud Composer runs Airflow and supports DAG backfills and Airflow operators. Workflows uses YAML or JSON step definitions and a different execution model, so this conflates the two products.

Why C is wrong: This inverts the positioning of the two services. Workflows is optimised for lightweight, event-driven service orchestration with HTTP and connector steps, while Cloud Composer is the data-pipeline orchestrator built on Airflow for richer multi-task DAGs.

Why D is correct: Cloud Composer is managed Apache Airflow, and Airflow's strengths are exactly the requirements stated: a large community-maintained operator catalogue for Google Cloud services, first-class backfills via the scheduler, and conditional branching through operators such as BranchPythonOperator.

See more PDE practice questions, answers explained.

More in this domain

Back to all Ingesting and Processing Data objectives, or the PDE cert hub.

Examworthy is not affiliated with or endorsed by Google Cloud. Original, blueprint-aligned practice material only.