A Microsoft Fabric Spark notebook reads a very large Delta fact table partitioned by event_date, joins it to a small currency lookup, and aggregates revenue. The job re-reads the same lookup on every cell, scans far more partitions than the date filter needs, and produces uneven task durations from skewed keys. The team wants to cut wasted scanning, avoid recomputing the lookup, and let the engine adapt to skew without re-partitioning the table. Which THREE changes together best improve performance? Select THREE.
- AApply a literal predicate on the partition column so Spark performs partition pruning and reads only the relevant event_date partitions instead of the whole table. Correct
- BCache the small currency lookup DataFrame in memory so repeated cell executions reuse the materialised result rather than recomputing the source read each time. Correct
- CEnable Adaptive Query Execution so Spark can coalesce shuffle partitions and split skewed partitions at run time based on the observed data statistics. Correct
- DDisable predicate pushdown on the Delta source so the engine loads every column and row first, then filters in memory for more predictable scan behaviour.
Why A is correct: A literal filter on the partition column lets Spark prune partitions at plan time, so only the matching files are listed and read, directly cutting wasted scanning.
Why B is correct: Caching the reused lookup keeps it materialised across cells, removing the repeated source read, which is the stated recompute cost on every cell.
Why C is correct: Adaptive Query Execution reacts to runtime statistics to handle skew and right-size shuffle partitions, addressing the uneven task durations without re-partitioning.
Why D is wrong: This is tempting as a uniformity argument, but disabling pushdown forces full reads and removes the file skipping that pruning provides, making the scanning worse.