Data-Engineer-Associate - Data Ingestion and Loading (21% of the exam) - Section 2.1

Enable and detail data ingestion patterns, including batch, streaming, and incremental loading, from local files and Lakeflow Connect standard and managed connectors.

Distinguish batch, streaming and incremental ingestion and the sources each suits, from uploaded local files to Lakeflow Connect standard connectors and fully managed connectors. Explain how checkpointing lets an incremental pattern process only new data.

batch ingestionincremental loadingStructured StreamingLakeflow Connectcheckpoints

Practice question for this objective

Free sampleData Ingestion and Loadingmedium

A data team must pull tables from a Salesforce account and from a self-managed PostgreSQL database into Unity Catalog, keeping both refreshed incrementally without writing custom extraction code. Which statement correctly describes how Lakeflow Connect covers these two sources?

ABoth sources are handled by Auto Loader, which discovers rows through the cloudFiles source once each system has been registered as an external location in Unity Catalog.
BBoth sources are handled by COPY INTO with a source connection specified in FORMAT_OPTIONS, since that statement is the supported entry point for connector-based ingestion.
CThe Salesforce application alone is in scope, because Lakeflow Connect covers SaaS applications and databases must instead be exported to files and read with a file source.
DManaged connectors handle both, with the Salesforce application ingested end to end and database sources such as PostgreSQL ingested through a configured ingestion gateway that captures changes. Correct

Lakeflow Connect offers managed connectors for SaaS applications and standard connectors, using an ingestion gateway, for database sources. Lakeflow Connect exists so that ingestion from applications and operational databases does not require hand-written extraction jobs. Managed connectors cover SaaS applications end to end, and the managed database connectors place an ingestion gateway close to the source to capture changes and land them incrementally in Unity Catalog.

Why A is wrong: Auto Loader is genuinely the incremental ingestion tool for files, which makes it a tempting answer. It reads objects from cloud storage paths and has no ability to connect to an application API or a database.

Why B is wrong: COPY INTO does load data into Delta tables, so it looks relevant, but it reads files of a given format from a storage path. It cannot take an application or database connection as its source.

Why C is wrong: This reflects an earlier and narrower view of the product. Lakeflow Connect covers database sources as well as SaaS applications, so a manual file export is unnecessary.

Why D is correct: Lakeflow Connect provides fully managed connectors for SaaS applications and for operational databases, and the database connectors deploy an ingestion gateway close to the source to capture its changes.

See more Data-Engineer-Associate practice questions, answers explained.

Exam traps in Data Ingestion and Loading

Answers that look right on this material and are not. Each one is a distractor from a different question in the Data-Engineer-Associate bank for this domain.

Configure a Lakeflow Connect managed connector for both sources, supplying the internal service's base URL and authentication token in its Unity Catalog connection so that the same connector handles it.
Why it is wrong: This is tempting because a connection does hold endpoint and authentication details for the sources it supports. It is wrong because a managed connector implements the schema and change tracking of one specific product, so it cannot be pointed at an arbitrary in-house service by supplying a URL.
Both feeds require only a Unity Catalog connection holding the source credentials, because every managed connector reaches its source from Databricks managed compute.
Why it is wrong: Tempting because a Unity Catalog connection is genuinely the credential object for both feeds, but it is incomplete: a database connector also deploys an ingestion gateway that has to sit where it can reach the database.
A single pipeline runs entirely on serverless compute and reaches the database without any component needing network access to it, because the source is polled through a public managed service.
Why it is wrong: Tempting because much of Lakeflow Connect does run on serverless compute, but a self managed database sits inside the customer network. Something has to be able to reach it, so a component with network access to the source is unavoidable.

Examworthy is not affiliated with or endorsed by Databricks. Original, blueprint-aligned practice material only.