Data-Engineer-Associate - Data Transformation and Modeling (22% of the exam) - Section 3.1

Implement data cleaning by reading bronze tables with PySpark or SQL, cleaning nulls, standardising data types, and writing to silver tables.

Read a bronze table with PySpark or SQL, handle nulls with fill, drop and coalesce, cast columns to consistent types, and write the result to a silver table. Place each step correctly in the medallion architecture.

medallion architecturebronze and silver tablesnull handlingtype castingPySpark DataFrames

Practice question for this objective

Free sampleData Transformation and Modelingmedium

A bronze table holds quantity as a STRING, because the source files were ingested without type inference. A data engineer runs the code below to replace missing quantities with zero before writing the silver table. The job succeeds, but a query counting the rows where quantity IS NULL in the silver table returns the same figure as before. What explains the result?

bronze = spark.read.table("prod.bronze.line_items")
cleaned = bronze.na.fill(0)
cleaned.write.mode("overwrite").saveAsTable("prod.silver.line_items")

AThe fill reaches only columns carrying a NOT NULL constraint in the bronze table definition, and quantity was declared as accepting nulls when the table was created.
BThe affected rows hold an empty string rather than a null, and a fill replaces only true nulls, so those values were left in place by the transformation.
CA fill requires an explicit subset of column names, and a call that omits the subset argument is treated as a no operation across every column in the frame.
DA numeric fill value is applied only to numeric columns, so a STRING column is skipped; casting quantity to an integer type first, or filling with a string, populates it. Correct

A null fill is matched by data type, so a numeric fill value silently skips string columns and leaves their nulls in place. The fill applies the supplied value only to columns whose type is compatible with it, so an integer reaches integer, long and double columns and passes over string columns without comment. Standardising the type first, by casting the column to an integer type as part of the bronze to silver step, makes the fill land and puts the column into the shape the silver contract promises.

Why A is wrong: Tempting because Delta tables do support column constraints, but a fill on a DataFrame is a transformation over values and pays no attention to the constraints declared on the source table.

Why B is wrong: A realistic bronze problem in general, but the check in the stem counts rows where quantity IS NULL, so the unchanged figure proves the values really are null and not empty text.

Why C is wrong: Tempting by analogy with the drop call, where a subset changes the outcome, but omitting the subset on a fill widens it to all eligible columns rather than disabling it.

Why D is correct: The fill is matched by type, so passing an integer restricts it to numeric columns and a STRING column is left exactly as it was, which is why the null count is unchanged.

See more Data-Engineer-Associate practice questions, answers explained.

Exam traps in Data Transformation and Modeling

Answers that look right on this material and are not. Each one is a distractor from a different question in the Data-Engineer-Associate bank for this domain.

It replaces the nulls in all three columns, converting the replacement value to an empty string for region so that the string column receives a value of a compatible type.
Why it is wrong: Tempting because implicit conversion happens elsewhere in Spark SQL. The fill value is not coerced across type families here, so a string column is left alone rather than filled with a converted form of the number.
The subset list is ignored once how is supplied, so the call inspected every column in the DataFrame; removing the subset argument makes the two key columns govern the drop.
Why it is wrong: Tempting because the two arguments do interact. They are independent: subset restricts which columns are inspected and how decides the threshold, and dropping subset would widen the inspection to every column rather than narrow it.
Silver holds the payloads exactly as they were received together with ingestion metadata such as source file name and load time, so any transformation can be replayed from it once a defect is found.
Why it is wrong: Tempting because replayability is a genuine goal of the architecture. That description belongs to the bronze layer, which is the immutable landing zone; silver sits downstream of it and holds transformed records.

Examworthy is not affiliated with or endorsed by Databricks. Original, blueprint-aligned practice material only.