A team is ingesting a large corpus of unstructured product images and their tabular metadata to train a multimodal model with Vertex AI. The image files live in Cloud Storage and the metadata, including labels and feature columns, sits in BigQuery. They want the training job to read the structured features efficiently while still pulling the raw image bytes, and they want to avoid exporting and duplicating the multi-terabyte image set. How should they organise ingestion?
- AExport every image into BigQuery as a BYTES column joined to the metadata so the entire dataset can be read with a single query.
- BCopy the BigQuery metadata into per-image JSON sidecar files in Cloud Storage and ignore BigQuery during training.
- CRead the structured features directly from BigQuery and resolve each row's Cloud Storage URI to stream the image bytes from Cloud Storage at training time. Correct
- DLoad the images into a Cloud SQL table as binary blobs and have the pipeline join them to the BigQuery metadata at training time.
Why A is wrong: Co-locating everything in one table looks convenient, but materialising multi-terabyte image bytes into BigQuery duplicates the data, inflates storage cost, and pushes large blobs through a query engine designed for analytical columns rather than bulk binary streaming.
Why B is wrong: Sidecar files keep everything in one storage service, but flattening structured features into thousands of tiny JSON objects discards BigQuery's efficient columnar reads and recreates the small-file overhead the team is trying to avoid.
Why C is correct: Keeping images in Cloud Storage and features in BigQuery lets each service do what it is built for, and storing the object URI in the metadata row lets the pipeline stream bytes on demand without duplicating the corpus.
Why D is wrong: A managed database feels like a tidy home for both, but Cloud SQL is a transactional store ill-suited to multi-terabyte binary blobs and high-throughput training reads, and a cross-service join per batch adds latency without removing duplication.