Work with common deep learning data types and architectures

Free sampleSoftware Developmentmedium

A data engineer is building a preprocessing pipeline for a text-to-image diffusion model. The pipeline must handle floating-point pixel values, tokenised text IDs, and binary attention masks in a single batched forward pass. Which statement best describes how these three data types are typically managed within the model's input layer?

AAll three inputs are cast to float32 at ingestion because neural network layers require a single unified numeric precision throughout the entire network.
BPixel values and attention masks are combined into one tensor before the first layer to reduce the number of separate inputs the model graph must manage.
CToken IDs are one-hot encoded into float vectors before embedding, because transformer attention cannot directly process integer indices.
DPixel values use float32 or float16, token IDs use integer tensors fed into an embedding layer, and attention masks use boolean or integer tensors applied inside the attention computation. Correct

Describe how floating-point image data, integer token IDs, and binary attention masks are handled as distinct data types within a deep learning model's input pipeline. Deep learning pipelines for multimodal models handle heterogeneous data types concurrently. Pixel values are floating-point tensors (float32 or float16) representing continuous intensity values. Text token IDs are integer tensors that serve as indices into a learned embedding lookup table, which converts them to dense float vectors before attention is applied. Attention masks are boolean or integer tensors with values 0 or 1 that are applied inside the attention layer to mask padding or future tokens; they are never merged with image tensors. Using the correct dtype for each input avoids unnecessary casting, preserves semantic meaning, and is standard practice in frameworks such as PyTorch and JAX.

Why A is wrong: Mixed-precision training and inference is standard practice; text token IDs remain as integer tensors (int32 or int64) until they are passed through an embedding lookup that converts them to float. Casting token IDs directly to float32 before embedding is semantically incorrect and would bypass the embedding table entirely.

Why B is wrong: Merging spatially structured image data with binary mask vectors is not a valid operation because they have different shapes and semantics. Attention masks are applied multiplicatively inside attention layers, not concatenated with pixel tensors at the input stage.

Why C is wrong: One-hot encoding was used in earlier NLP systems but is computationally wasteful at large vocabulary sizes. Modern transformer implementations pass integer token IDs directly to a learnable embedding matrix (an efficient table lookup), which is both faster and more memory-efficient than materialising a full one-hot vector for each token.

Why D is correct: Each data type retains its appropriate dtype: continuous pixel intensities are represented as floating-point values, discrete token IDs are integers indexed into an embedding table to produce float vectors, and binary attention masks are boolean or 0/1 integer tensors multiplied into the attention score matrix to suppress padding positions. Using the correct dtype for each input is essential for numerical correctness and memory efficiency.

Work with common deep learning data types and architectures.

Practice question for this objective

More in this domain