PDE - Designing Data Processing Systems (22% of the exam) - Section 1.2

Design for reliability and fidelity, including data preparation with Dataform and Dataflow, pipeline orchestration, disaster recovery, ACID compliance, and data validation.

Design reliable data pipelines using Dataform for SQL-based data preparation, Dataflow for stream and batch processing, and Cloud Data Fusion for visual integration, and define disaster recovery strategies that maintain ACID compliance and validate data fidelity after failures.

DataformDataflowCloud Data FusionDisaster recovery

Practice question for this objective

Free sampleDesigning Data Processing Systemsmedium

A government open-data team uses Cloud Data Fusion to onboard 40 source systems into BigQuery, each with its own schema, light cleansing rules, and sensitive fields that must be masked. Citizen analysts maintain the pipelines through a visual interface, while a separate platform team manages shared connections, credentials, and pipeline templates that all analyst pipelines reuse. Which Cloud Data Fusion capability lets the platform team package the shared connections and cleansing logic as reusable building blocks that analysts can drop into pipelines without writing code?

  • AWrangler directives saved as ad hoc transformation scripts inside each individual pipeline by every analyst as they build it.
  • BDataform repositories referenced from inside Cloud Data Fusion pipelines so each analyst pipeline calls a shared SQLX script for cleansing.
  • CCloud Functions wrapped behind HTTP triggers that each pipeline must call for masking, registered as external services in the Data Fusion environment.
  • DPlugins published from the Hub, combined with system-managed connections and reusable pipeline templates that analysts import into their own pipelines. Correct
Use Cloud Data Fusion plugins, system connections, and pipeline templates to share governed building blocks across many visual pipelines. Cloud Data Fusion is designed around a visual pipeline canvas where each node is a plugin. The Hub lets platform teams publish plugins and pipeline templates centrally, and system-managed connections let analysts reference shared credentials and endpoints without seeing the secrets. Together these features deliver reusable, governed building blocks for no-code pipeline authoring, which neither inline Wrangler scripts, Dataform, nor ad hoc Cloud Functions provide.

Why A is wrong: Wrangler directives are powerful for interactive cleansing, but saving them inside each pipeline means logic is duplicated and cannot be governed centrally by the platform team; this misses the reuse requirement.

Why B is wrong: Dataform is a separate service that orchestrates SQL inside BigQuery; Cloud Data Fusion pipelines do not natively call SQLX scripts, so this is a misleading mix of services.

Why C is wrong: Calling Cloud Functions adds latency, cost, and operational overhead, and is not the idiomatic Data Fusion mechanism for reusable transforms; the visual-first model points to plugins and templates instead.

Why D is correct: Cloud Data Fusion plugins extend the visual palette with custom sources, transforms, and sinks; combined with named system connections and saved pipeline templates, they give analysts pre-blessed building blocks they drag into pipelines without writing code, which matches the team's requirements.

See more PDE practice questions, answers explained.

Exam traps in Designing Data Processing Systems

Answers that look right on this material and are not. Each one is a distractor from a different question in the PDE bank for this domain.

  • Store source data in a regional bucket in europe-west2 and rely on Cloud Storage's built-in cross-region failover, which automatically serves objects from any other European region when the primary fails.

    Why it is wrong: Regional buckets do not automatically replicate to other regions, and Cloud Storage does not silently fail over a regional bucket to another region; this is a misconception that sounds reassuring but does not exist as a feature.

  • Dataform and Dataflow both schedule SQL transformations inside BigQuery, while Cloud Data Fusion is a managed Apache Beam service for streaming pipelines built with Java or Python SDKs.

    Why it is wrong: This swaps the runners. Dataflow runs Apache Beam, not SQL inside BigQuery, and Cloud Data Fusion is not a Beam runner: it is a visual ETL service based on the open source CDAP project that targets Dataproc Spark.

  • Keep the Dataform repository in a single primary workspace tied to us-central1 and rely on a manual code copy into a us-east1 workspace if the primary region fails, accepting that the SQLX may diverge until the copy is performed.

    Why it is wrong: A manual copy during an incident delays recovery and risks divergence between the active and standby SQLX, which puts the 2 hour RTO at risk and makes ongoing maintenance error prone.

Examworthy is not affiliated with or endorsed by Google Cloud. Original, blueprint-aligned practice material only.