Data-Engineer-Associate domain - 6% of the exam

Databricks Intelligence Platform

Databricks Intelligence Platform is 6% of the Data-Engineer-Associate exam. These are the objectives it covers, each with practice questions, with every answer explained.

The domain in numbers

18
Practice questions
2
Objectives
18
Medium

Where people go wrong

  • Resizing all-purpose compute instead of changing how it bills

    The wrong answers reach for autoscaling, a lower maximum worker count or the type of task that triggers the run, and expect the all-purpose bill to come down. The DBU rate follows the compute resource, so none of these changes it, and an all-purpose cluster with auto termination switched off bills through every idle hour whatever its size. Jobs compute is created for a run, billed at the lower jobs DBU rate and terminated when the run finishes, while an idle interactive cluster needs an auto termination window.

    See it in a sample question
  • Swapping serverless and classic compute

    The wrong answers swap which compute starts quickly. A classic SQL warehouse provisions virtual machines in the customer's cloud account and takes minutes, while a serverless SQL warehouse draws on capacity managed in the Databricks account and starts within seconds. Serverless compute is not limited to warehouses, because it also serves notebooks and Lakeflow Jobs tasks, and it removes instance type and worker count choices rather than relying on an instance pool.

    See it in a sample question
  • Giving managed and external tables the same DROP behaviour

    The wrong answers give a managed table and an external table the same drop behaviour, or assume Unity Catalog deletes files it only governs. Unity Catalog owns the storage lifecycle of a managed table, so its files become eligible for deletion. An external table registered over a path through a storage credential and external location loses only its metastore entry, and the files stay in place for other readers.

    See it in a sample question
  • Expecting locks between Delta readers and writers

    The wrong answers import relational locking: readers blocking writers, writers making readers wait, or a recovery pass after a crash. Delta Lake uses optimistic concurrency control, so writers commit without blocking and conflict only when the files they touch overlap. A reader resolves one version of the transaction log when the query starts and reads only the files listed in that version, so a concurrent overwrite does not change its result.

    See it in a sample question
  • Treating log retention or VACUUM as history control

    The wrong answers assume a surviving log entry keeps a version readable, or that VACUUM can reverse a bad change. Time travel needs both the log entry and the data files it points to, and VACUUM deletes unreferenced files older than the retention threshold regardless of log retention. That is why an old version can fail after VACUUM, while RESTORE TABLE TO VERSION AS OF 412 works when no VACUUM has run.

    See it in a sample question

Objectives in this domain

Sample questions from this domain

Free sampleDatabricks Intelligence Platformmedium

A workspace admin reviewing usage finds one all-purpose cluster accounted for a large share of last month's DBUs while its notebooks executed cells on roughly two days in that month. The cluster was created for a proof of concept, auto termination was switched off at creation, and no scheduled task is attached to it. The admin wants the underlying cause of the charge addressed rather than the symptom. What should the admin do?

  • ASet an auto termination window on the cluster so that it shuts down after a defined idle period, which stops it billing through the stretches when nobody is running a cell. Correct
  • BReduce the cluster's maximum worker count so that the same idle period consumes fewer DBUs, and ask the team to report any performance problem that follows.
  • CEnable autoscaling on the cluster with a minimum of one worker, so that the cluster releases capacity by itself during the long stretches without activity.
  • DMigrate the proof of concept notebooks to a serverless SQL warehouse with a short auto stop window, which removes the charge by retiring the cluster in favour of warehouse compute.
All-purpose compute bills for uptime rather than activity, so auto termination is the setting that prevents an idle cluster accruing DBUs. DBU consumption on all-purpose compute is a function of how long the driver and workers stay up, not of how many commands run on them. A cluster created with auto termination disabled therefore bills continuously until somebody stops it by hand, and setting an idle termination window returns that decision to the platform.

Why A is correct: All-purpose compute bills while it runs regardless of whether commands are executing, so the cause of the charge is a cluster left up with auto termination disabled, and restoring that setting stops the idle accrual.

Why B is wrong: A lower maximum trims the size of the bill without touching its cause: an idle cluster with fewer workers is still a running cluster billed at the all-purpose rate for weeks of doing nothing.

Why C is wrong: Autoscaling does release underutilised workers, which sounds like the fix, but it holds the minimum workers and the driver running indefinitely, so a cluster idle for most of a month keeps billing.

Why D is wrong: A serverless warehouse does stop when idle, but it runs SQL statements rather than proof of concept notebooks, so this replaces the workload's compute service instead of correcting the setting that caused the charge.

Free sampleDatabricks Intelligence Platformmedium

A platform team currently sizes every cluster by hand, choosing instance types and worker counts for each workload, and is evaluating serverless compute for notebooks and Lakeflow Jobs tasks. Which statement correctly describes what serverless compute changes for them?

  • AAn administrator has to keep a pool of virtual machines warm in advance, and any serverless workload waits for that pool to be refilled whenever it has been drained by earlier work.
  • BThe same cluster configuration page still applies, so the team keeps choosing instance types and worker counts but is charged one flat hourly rate whatever size it happens to pick.
  • CServerless compute applies to SQL warehouses alone, so notebooks and Lakeflow Jobs tasks have to stay on classic clusters that the platform team configures for itself.
  • DDatabricks provisions and sizes the compute, so the team stops choosing instance types and worker counts, and capacity for a workload becomes available within seconds. Correct
Serverless compute is provisioned and sized by Databricks, removing instance and worker choices and starting workloads in seconds. With serverless compute the workspace no longer describes the machines it wants. Databricks holds ready capacity and allocates it to the workload, so the instance type, worker count and autoscaling bounds disappear from the configuration and start-up drops from minutes to seconds. Consumption is still tied to the resources the workload uses, so the change is one of who manages the infrastructure rather than a flat fee, and it covers notebooks and Lakeflow Jobs tasks as well as SQL warehouses.

Why A is wrong: This describes instance pools attached to classic compute; serverless capacity is managed by Databricks and needs no pool maintained by the workspace administrator.

Why B is wrong: It sounds like a simple billing change, but serverless removes the sizing choices entirely and consumption still varies with the resources a workload uses.

Why C is wrong: Serverless SQL warehouses came first, which makes this plausible, but serverless compute is also available for notebooks and for Lakeflow Jobs tasks.

Why D is correct: Serverless compute is managed by Databricks from capacity that is already ready, which removes the instance type and worker count decisions and gives near-immediate start-up.

Free sampleDatabricks Intelligence Platformmedium

A data engineer drops two Unity Catalog tables that sit in the same schema. One is a managed table. The other is an external table registered over a path inside a working external location. Neither table has been cloned. What happens to the underlying data files in each case?

  • AThe managed table's files become eligible for deletion by Databricks, while the external table's files stay untouched in the external location. Correct
  • BThe files behind both tables are deleted straight away, because Unity Catalog owns the storage lifecycle of every table it governs.
  • CThe files behind both tables stay in place, because DROP TABLE in Unity Catalog removes the metastore entry and nothing else.
  • DThe managed table's files stay in place for a retention period, while the external table's files are removed together with its metastore entry.
DROP TABLE deletes the data of a Unity Catalog managed table but only the metadata of an external table. The difference is who owns the storage. A managed table lives in storage that Unity Catalog controls, so dropping it hands the data back to the platform for deletion. An external table is a governed reference to a path that the customer owns through an external location and storage credential, so dropping it removes the catalogue entry and leaves every file exactly where it was.

Why A is correct: Correct, because Unity Catalog owns the storage lifecycle of a managed table and only owns the metadata of an external table, whose files remain the responsibility of the storage account.

Why B is wrong: This is tempting because Unity Catalog does govern both tables, but governing access is not the same as owning storage, and deleting external files would destroy data other systems may still read.

Why C is wrong: This generalises external table behaviour to every table, which is a common carry-over from the legacy Hive metastore, and it misses that a managed table's data is deleted with it.

Why D is wrong: This inverts the two behaviours, and although managed table data is retained briefly before permanent deletion, the external files are never deleted by the DROP statement.

Free sampleDatabricks Intelligence Platformmedium

A streaming job appends micro-batches to a Delta table for several minutes while an analyst runs a long aggregation over the same table from a SQL warehouse. The aggregation returns a result consistent with one point in time rather than a mixture of old and new rows. Which mechanism produces that behaviour?

  • AThe SQL warehouse takes a read lock over the table for the duration of the aggregation, which holds back the streaming job's commits until the analyst's query has finished.
  • BThe streaming job stages each micro-batch in a temporary directory, and its rows only join the table once every open reader has disconnected from it.
  • CThe aggregation re-reads the storage directory listing as each task begins, and discards any file whose modification time falls after the query started.
  • DThe aggregation resolves the table to a single version of the transaction log when it starts, and reads only the data files that this version lists. Correct
Delta Lake readers pin one transaction log version at query start, which gives snapshot isolation without blocking concurrent writers. The transaction log, not the storage directory, is the authoritative list of files that make up a table at a given version. A reader resolves the table once, at the version current when it starts, and then reads exactly the files that version names. Later commits create new versions with new file lists and are therefore invisible to the running query, so writers and readers proceed at the same time without either blocking the other.

Why A is wrong: This assumes readers block writers, which is the model in many relational databases, but on Delta Lake a reader never delays a commit and the streaming job keeps writing throughout.

Why B is wrong: Micro-batches are committed as soon as they are written rather than held for readers, so tying visibility to reader disconnection invents a coordination step that does not exist.

Why C is wrong: Delta Lake reads the file list from the transaction log rather than by listing storage, and relying on modification times would be fragile because file timestamps do not record commit order.

Why D is correct: Correct, because snapshot isolation comes from pinning one log version at query start, so files committed afterwards are simply not part of the file list being read.

Free sampleDatabricks Intelligence Platformmedium

A Delta table sets delta.logRetentionDuration to 30 days and keeps the default file retention for VACUUM. After a routine VACUUM run, an analyst queries the table with VERSION AS OF set to a version that was committed twenty days ago. What is the expected outcome?

VACUUM prod.sales.orders;
SELECT * FROM prod.sales.orders VERSION AS OF 118;
  • AThe query returns the rows as they stood at that version, because the log entry describing the version is retained for the full thirty days.
  • BThe query is rejected before execution, because VERSION AS OF can only address versions committed within the past seven days on any Delta table.
  • CThe query returns an empty result set, because VACUUM rewrites each historical version of the table so that it contains no rows at all.
  • DThe query fails, because VACUUM has removed data files that the twenty-day-old version still references, even though the log entry for that version survives. Correct
Delta Lake time travel needs the data files as well as the log entry, so VACUUM retention limits how far back a query can reach. A Delta table version is a list of data files recorded in the transaction log. Log retention governs how long that list is kept, while VACUUM governs how long the files themselves are kept once no current version references them. When the two settings disagree, the shorter file retention wins in practice, because a version whose files have been deleted cannot be reconstructed and the read fails.

Why A is wrong: This is tempting because the log retention setting really is thirty days, but a log entry only lists the data files a version needs; it does not keep those files alive once VACUUM has removed them.

Why B is wrong: Seven days is the default VACUUM file retention rather than a fixed limit on time travel syntax, so this invents a hard rule that Delta Lake does not enforce on the query itself.

Why C is wrong: VACUUM only deletes files that the current version no longer references and never rewrites historical versions, so an empty result would misrepresent what the command does.

Why D is correct: Correct, because time travel needs both the log entry and the data files it points to, and VACUUM deletes unreferenced files older than the retention threshold regardless of log retention.

Other domains in this exam

See also the Data-Engineer-Associate cert hub, the study guide, and the cheat sheet.

Examworthy is not affiliated with or endorsed by Databricks. Original, blueprint-aligned practice material only.