PDE domain - 18% of the exam

Maintaining and Automating Data Workloads

Maintaining and Automating Data Workloads is 18% of the PDE exam. These are the objectives it covers, each with practice questions, with every answer explained.

The domain in numbers

61
Practice questions
5
Objectives
48
Medium
13
Hard

Where people go wrong

  • Expecting an autoscaled Dataproc cluster to stop billing when idle

    The wrong answers lean on autoscaling, preemptible secondary workers or fair scheduling to cut idle spend on a persistent Dataproc cluster. Autoscaling does not scale primary workers to zero, so the master and minimum primary workers keep billing all day, and preemptible secondaries do nothing about idle primaries. The correct answers remove the cluster itself: an ephemeral job-based cluster torn down on completion, scheduled deletion after an idle timeout, or Dataproc Serverless for Spark batch workloads.

    See it in a sample question
  • Answering slot contention with capacity instead of batch priority

    When delay-tolerant ELT displaces analyst queries in BigQuery, the wrong answers submit it as interactive jobs, move it to on-demand pricing, add a zero-baseline reservation or raise the baseline. Interactive jobs compete immediately for the same slots, and none of those changes reclassifies the work. Submitting the jobs with batch priority makes BigQuery queue them until slots free up, so they yield to interactive demand without extra baseline slots and without being cancelled.

    See it in a sample question
  • Treating regional HA as cross-region protection

    The wrong answers stretch Cloud SQL regional HA, backups or extra replicas across a region failure. Regional HA keeps a synchronous standby in a second zone of the same region, so a full regional outage takes both down. Surviving it needs a cross-region read replica, which is asynchronous, promoted by an operator, and accepts some lost writes bounded by replication lag. Memorystore for Redis Cluster does not replicate across regions either, so a standby cluster must be provisioned elsewhere.

    See it in a sample question
  • Mixing up Airflow retry, timeout and SLA settings

    The wrong answers give a job to the wrong Airflow setting. The schedule_interval and catchup parameters govern when runs are created, not how a failed task recovers. The dagrun_timeout and execution_timeout settings kill the run or task, so they fail a requirement to alert and keep partial output. An sla raises a miss notification through sla_miss_callback while the task keeps running, and it never retries anything. Automatic re-execution after a transient failure belongs to retries, with retry_delay and retry_exponential_backoff, and a downstream task starts only after its upstream task succeeds.

    See it in a sample question
  • Expecting one shared reservation to isolate teams

    The wrong answers expect a single reservation, an assignment, a higher autoscaling ceiling or on-demand pricing to give each team a guaranteed share of slots. Assignments only route projects to a reservation, and autoscaling adds burst capacity without dividing it, so a heavy query can still starve another team. Isolation comes from one reservation per team, each with its own baseline, with idle slot sharing left on so quiet teams lend unused slots within the same administration project.

    See it in a sample question

Objectives in this domain

What to study

The official documents this domain's practice questions cite most.

Sample questions from this domain

Free sampleMaintaining and Automating Data Workloadsmedium

A platform team operates a persistent Dataproc cluster shared by several Spark jobs that mostly run between 09:00 and 18:00. Outside those hours the cluster sits idle for 14 hours per day but is occasionally used for an unscheduled rerun. Leadership has asked for an automatic way to stop paying for that idle period without breaking the rerun pattern. Which Dataproc capability most directly addresses this requirement?

  • AConfigure a Dataproc autoscaling policy that scales the primary worker pool down to zero workers between 18:00 and 09:00 and back up in the morning.
  • BSwitch the cluster to use only preemptible secondary workers overnight so that Google can reclaim them and effectively pause the cluster outside business hours.
  • CStop the cluster manually each evening through a scheduled script and rely on operators to restart it before the morning workload begins.
  • DEnable cluster scheduled deletion with an idle deletion timeout, so the cluster is removed automatically once it has been idle for a set period and a fresh one is created on demand for reruns. Correct
Use Dataproc scheduled deletion with an idle timeout to eliminate cost during long idle periods on a persistent cluster. Scheduled deletion lets a cluster be removed after a maximum age, after a maximum idle duration, or at a specific time. For a workload that is idle overnight, idle deletion ends the meter entirely, and the create-on-demand pattern handles the occasional out-of-hours rerun cheaply.

Why A is wrong: Dataproc autoscaling operates on the secondary worker pool, not the primary, and it does not let the cluster scale to zero, so the master and minimum primary workers continue to bill through the night.

Why B is wrong: Preemption is unpredictable and does not stop master and primary worker charges; it also leaves any rerun exposed to reclamation, so it does not deliver a controlled idle shutdown.

Why C is wrong: Dataproc does not support a true stop/start of a cluster the way Compute Engine VMs do; a scheduled manual workflow is fragile and still leaves the unscheduled-rerun case to human response time.

Why D is correct: Idle deletion (cluster scheduled deletion) tears the cluster down after a configurable idle period, ending all charges; an unscheduled rerun simply provisions a new cluster on demand, which fits a pattern where idle time dominates the day.

Free sampleMaintaining and Automating Data Workloadsmedium

A data engineering team runs nightly transformation jobs that materialise three large reporting tables in BigQuery. The jobs are tolerant of multi-hour delays but must not be cancelled, and the team wants them to wait in a queue when their reservation is fully utilised by interactive analyst queries rather than competing for slots. Which job classification and submission approach best fits these requirements?

  • ASubmit the jobs with the batch priority so that BigQuery queues them when shared slots are saturated and starts them as capacity frees up, without affecting the daily quota for concurrent interactive queries. Correct
  • BSubmit the jobs with the interactive priority so that they receive slots immediately and complete predictably alongside analyst queries.
  • CSubmit the jobs as interactive but cap their slot usage with a separate workload management policy so that they yield to analyst queries automatically during peak hours.
  • DSubmit the jobs as interactive and rely on BigQuery to automatically downgrade them to batch priority once the reservation is fully saturated by analyst workloads.
Classify delay-tolerant transformation jobs correctly so that BigQuery queues them rather than competing for interactive slot capacity. BigQuery supports two job priorities: interactive jobs run as soon as resources are available and count against an interactive concurrency quota, while batch jobs are queued and started when capacity frees up, counting against a separate batch quota. Submitting delay-tolerant work as batch keeps interactive headroom for analysts and avoids cancellations, because BigQuery waits rather than failing the job.

Why A is correct: Batch jobs are explicitly designed for delay-tolerant workloads: BigQuery queues them when shared resources are busy, schedules them when capacity is available, and counts them against a separate batch quota, leaving the interactive concurrency budget free for analysts.

Why B is wrong: Interactive jobs do start immediately when slots are available, but they compete with analyst queries for the same slot pool, which is the opposite of the requested behaviour of waiting in a queue when slots are busy.

Why C is wrong: BigQuery does not expose a per-job slot cap that throttles interactive queries based on time of day; reservations and assignments allocate slots, but interactive jobs within an assignment still compete on a fair-share basis.

Why D is wrong: It is tempting because BigQuery does manage slot sharing dynamically, but priority is set at job submission time and is not automatically downgraded; if the team wants queue-on-busy behaviour, they must submit with the batch priority.

Free sampleMaintaining and Automating Data Workloadshard

A media subscriptions company runs a Cloud SQL for MySQL primary in us-east4 with regional HA and a single cross-region read replica in us-west1. During a regional outage of us-east4 the team promotes the us-west1 replica and points the application at it, but after recovery they discover that 11 minutes of writes that were committed to the original primary before the outage never reached the promoted replica. The team wants to plan for the next event so that lost-write exposure is bounded to seconds rather than minutes, while keeping a single primary topology. Which approach is the most appropriate next step?

  • AConvert the cross-region replica to a regional HA standby, which provides synchronous replication into the secondary region and reduces lost-write exposure to zero.
  • BTune the workload and instance to keep replication lag low under normal load and monitor it with alerting, accepting that asynchronous cross-region replication makes some lag unavoidable. Correct
  • CSchedule more frequent on-demand backups in us-east4 and use point-in-time recovery into us-west1 during the outage, which bounds data loss to the backup interval.
  • DAdd additional cross-region read replicas in further regions to spread replication load, which will reduce replication lag on the us-west1 replica.
Recognise that cross-region Cloud SQL replication is asynchronous and that lost-write exposure must be managed through lag monitoring. Cloud SQL has no synchronous cross-region replication option; the standby in regional HA is in a paired zone of the same region. Cross-region replicas use asynchronous replication, so the RPO during an unplanned region failure equals the replication lag at the moment of failure. Bounding exposure therefore requires keeping lag low and alerting on it, not switching to a non-existent synchronous mode.

Why A is wrong: Cloud SQL regional HA places the synchronous standby in a different zone of the same region, not in another region. The option misrepresents the product and would not provide synchronous cross-region writes.

Why B is correct: Cloud SQL cross-region replication is asynchronous, so some lag is inherent and lost writes during an unplanned region failure are bounded by current lag at the time of failure. The realistic mitigation is to keep lag low through workload tuning, monitoring, and alerting rather than expecting a synchronous cross-region option that does not exist.

Why C is wrong: Point-in-time recovery from backups has an RTO of many minutes and requires the source region to be reachable for transaction log retrieval, so it is unsuitable as a primary cross-region failover strategy.

Why D is wrong: Adding more replicas does not reduce the lag on any individual asynchronous replica; replication lag is bounded by network latency and write volume, not by replica count. The change is plausible-sounding but does not address the root cause.

Free sampleMaintaining and Automating Data Workloadsmedium

An on-call engineer is reviewing how a critical Cloud Composer DAG handles transient failures. The DAG includes a task that calls an external HTTP endpoint, and the team wants to understand exactly which Airflow scheduling primitive controls how many times a failed task is re-executed before the run is marked failed. Which Airflow task parameter governs this behaviour?

  • AThe schedule_interval setting on the DAG, which determines both the cadence of new runs and the number of times each task within a run may be re-executed on failure.
  • BThe retries parameter on the task, optionally combined with retry_delay and retry_exponential_backoff, which together specify how many additional attempts Airflow makes after a task instance fails. Correct
  • CThe catchup flag on the DAG, which when set to true causes any failed task instances to be retried automatically across all backfilled intervals.
  • DThe depends_on_past flag on the task, which causes the task to retry until the previous DAG run for the same task has succeeded.
Identify the Airflow task parameters that control how transient failures are retried before a task is marked failed. Per-task retry behaviour in Airflow is controlled by the retries parameter, which sets how many additional attempts the scheduler makes after the first failure of a task instance. retry_delay and retry_exponential_backoff shape the wait between attempts. Only after all configured retries fail does the task move to a failed state and propagate failure through the DAG, which makes this the correct primitive for handling transient errors against an external HTTP endpoint.

Why A is wrong: Plausible because schedule_interval is a real DAG-level parameter, but it controls only how often new DAG runs are created and has no effect on per-task retry behaviour.

Why B is correct: Correct. The retries parameter sets the number of additional attempts after the first failure. retry_delay and retry_exponential_backoff configure the wait between attempts. The DAG run is marked failed only once retries are exhausted.

Why C is wrong: Tempting because catchup is a real and important DAG flag, but it controls whether missed intervals are backfilled, not how individual tasks recover from transient failures within a single run.

Why D is wrong: Misreads depends_on_past. That flag controls whether the current task waits on the success of the same task in the previous run; it does not configure a retry count for the current task.

Free sampleMaintaining and Automating Data Workloadsmedium

A multinational organisation manages BigQuery workloads for analytics, machine learning, and data engineering teams in a single administration project. They want each team to have a guaranteed minimum capacity, share unused capacity opportunistically when their own workload is light, and isolate one team's heavy job from starving the others. Which combination of BigQuery features best supports these goals?

  • ACreate a single large reservation in the administration project and assign all team projects to it, relying on fair-share scheduling within the reservation to balance team workloads.
  • BCreate one reservation per team in separate administration projects and rely on cross-project query federation to redistribute capacity when one team is idle, while autoscaling provides isolation.
  • CAssign all team projects to on-demand pricing and use INFORMATION_SCHEMA to monitor per-team slot consumption, applying quotas to limit any team that exceeds a fair share of capacity.
  • DCreate one reservation per team with its own baseline and assignment, and enable idle slot sharing so that unused baseline slots from quiet reservations are made available to busy reservations in the same administration project. Correct
Combine per-team reservations and idle slot sharing in a shared administration project to deliver minimum capacity, elasticity, and workload isolation. BigQuery reservations live in an administration project and bind to billing projects via assignments. Giving each team its own reservation with a baseline provides a guaranteed minimum, while idle slot sharing redistributes unused baseline slots between reservations in the same administration project so quiet teams temporarily lend capacity to busy ones without giving up isolation when their own work returns.

Why A is wrong: A single reservation gives every team an equal slot of the shared pool with no minimum guarantee, so a heavy job from one team can crowd out the others; this does not provide the per-team minimum capacity the organisation needs.

Why B is wrong: Idle slot sharing operates between reservations within the same administration project, not across separate administration projects, so this setup would not allow capacity to flow between teams; autoscaling sizes a single reservation and does not provide cross-team isolation.

Why C is wrong: On-demand pricing does not offer per-team minimum capacity or isolation; quotas can cap usage but cannot guarantee a baseline, which is exactly the requirement the organisation has stated.

Why D is correct: Per-team reservations with their own baselines give each team a guaranteed minimum capacity, the assignment binds team projects to that reservation, and idle slot sharing lets unused baseline slots from quiet reservations flow to busy ones, achieving both isolation and elasticity.

Other domains in this exam

See also the PDE cert hub, the study guide, and the cheat sheet.

Examworthy is not affiliated with or endorsed by Google Cloud. Original, blueprint-aligned practice material only.