Software Development

An engineering team serves a single ONNX model on Triton Inference Server. Under load, the GPU sits at roughly 30 percent utilisation while many requests queue, because each request is processed one at a time even though the model accepts variable batch sizes. The team cannot change the client, which still sends one inference per call. Which model configuration change most directly raises throughput by forming larger batches from these separate in-flight requests?

• AAdd a dynamic_batching section to config.pbtxt so Triton groups individual incoming requests into larger batches on the server side before running the model.check_circle Correct
• BRaise max_batch_size in config.pbtxt to a larger number, which alone causes Triton to merge separate requests into one batch automatically.
• CSet instance_group count to a higher value so multiple model copies run, which merges the queued requests into a single larger batch.
• DSwitch the model to the sequence_batching scheduler so correlated requests are gathered into one batch.

Why A is correct: Dynamic batching queues separate single-request inferences for a short window and combines them into one batched execution, which is exactly the server-side aggregation needed to lift GPU utilisation without touching the client.

Why B is wrong: max_batch_size only sets the upper bound a batch may reach; it is tempting because it mentions batching, but without dynamic_batching enabled Triton still runs each single request on its own and never combines them.

Why C is wrong: Adding instances increases concurrency by running more copies in parallel, which sounds like it would help the queue, but each instance still processes one request at a time and no request gets combined into a larger batch.

Why D is wrong: Sequence batching is for stateful streams where requests share a correlation ID and must hit the same instance; it is plausible because it contains the word batching, but it does not aggregate independent stateless requests for throughput.

A developer fine-tunes a large pre-trained vision-language model on a proprietary dataset of 2,000 annotated medical images. After fine-tuning, validation accuracy on the medical task is high, but performance on standard image benchmarks drops sharply. What best explains this outcome?

• AThe pre-trained model was too small to retain general visual features alongside the new medical task knowledge.
• BFine-tuning all layers with a high learning rate on a small specialised dataset caused catastrophic forgetting of the general representations learned during pre-training.check_circle Correct
• CThe 2,000 medical images introduced distribution shift, which inherently makes generalisation to non-medical benchmarks impossible regardless of training strategy.
• DThe vision-language pre-training objective does not align with discriminative classification, so any fine-tuning on image tasks will degrade general benchmark scores.

Why A is wrong: Tempting because model capacity sounds relevant, but a model that achieved high scores on standard benchmarks before fine-tuning clearly had sufficient capacity. The drop is caused by how training was conducted, not by the model being too small.

Why B is correct: Catastrophic forgetting occurs when unconstrained gradient updates overwrite broadly useful weight configurations. A high learning rate applied to all layers on a narrow dataset shifts the weights strongly toward the new distribution, erasing the general-purpose features that supported benchmark performance. Transfer learning efficiency depends on preserving those features through selective freezing or low learning rates.

Why C is wrong: Distribution shift does reduce benchmark performance somewhat, but it does not make generalisation impossible. Strategies such as freezing lower layers or using a very low learning rate routinely preserve general-purpose representations even when fine-tuning on narrow domains, so the outcome is not inevitable.

Why D is wrong: Many state-of-the-art classifiers are initialised from vision-language pre-trained models and do not suffer benchmark degradation after careful fine-tuning. The objective mismatch is a real consideration but is not the primary cause of the sharp drop described; the fine-tuning strategy is the dominant factor.

A practitioner is choosing a transformer architecture family to build a sentence-pair semantic similarity system, where each input pair must be encoded into a single fixed-length representation that summarises the whole pair bidirectionally. Which architecture family is best suited in principle for producing such bidirectional, whole-input representations?

• AA decoder-only architecture, because its causal self-attention lets each token aggregate information from all earlier and later tokens to form a balanced bidirectional summary.
• BAn encoder-decoder architecture, because the cross-attention bridge is required to compute a single similarity score between two separately encoded sentences.
• CA decoder-only architecture with a prepended classification token, because the final token's hidden state already aggregates the entire sequence under causal attention.
• DAn encoder-only architecture, because every token attends to all other tokens in both directions, yielding representations that integrate full left and right context for classification-style tasks.check_circle Correct

Why A is wrong: Decoder-only models are tempting because they dominate generation, but causal masking blocks attention to later tokens, so they cannot build genuinely bidirectional representations.

Why B is wrong: Encoder-decoder models suit sequence-to-sequence mapping, but cross-attention is for conditioning generation on a source, not a requirement for producing one bidirectional representation of a pair.

Why C is wrong: The final hidden state under causal attention does see all prior tokens, which tempts candidates, but it still lacks the future-to-past attention that defines bidirectional encoding.

Why D is correct: Encoder-only transformers such as BERT use unmasked self-attention so each token sees the entire sequence, producing bidirectional representations ideal for encoding a pair into one fixed-length vector.

A developer gives an LLM a tool definition for a get_weather function and sends a user message asking for the forecast. The model returns a response whose finish reason indicates a tool call, with arguments naming a city. Which sequence correctly completes the function-calling loop so the user receives a natural-language answer?

• AReturn the tool call's raw arguments JSON directly to the user, since the model has already decided the city and no further model call is needed.
• BRe-send the original user message with the temperature raised so the model abandons the tool call and answers the weather question from its own parameters.
• CExecute get_weather, then append the result as a new user-role message and stop, treating that message as the final answer shown to the user.
• DExecute get_weather locally with the supplied arguments, append a tool-role message containing the result, then call the model again so it can phrase the final reply.check_circle Correct

Why A is wrong: This is tempting because the arguments look like an answer, but they are only the model's request to run code; the user expects prose and the weather data has not yet been fetched or summarised.

Why B is wrong: Raising temperature does not make the model resolve real-time weather, and discarding the tool call defeats the purpose of function calling, since the model lacks current data without executing the tool.

Why C is wrong: Using the user role for tool output misrepresents the conversation structure, and stopping there skips the model call that turns raw data into a reply, so the user sees a data blob rather than an answer.

Why D is correct: The model only emits a tool call; the application runs the function, feeds the result back as a tool-role message, and re-invokes the model so it can compose a grounded natural-language answer from the returned data.

A team feeds variable-length sentences into a transformer encoder as one batched tensor. Short sentences are padded with a special token so every row reaches the same length. During the forward pass, the padded positions still influence the attention scores of real tokens, slightly corrupting the encoder outputs. Which input alongside the token-id tensor should they supply to stop this?

• AA token-type id tensor that labels each position with a segment number so the model separates the padded region from the real tokens.
• BAn attention mask tensor of the same shape that marks real positions as 1 and padded positions as 0, so padded keys are excluded from the softmax.check_circle Correct
• CA position-id tensor that restarts the positional index at zero for every padded span so the encoder ignores those offsets.
• DA second token-id tensor holding the reversed sequence so the model can average the forward and reversed encodings and cancel the padding.

Why A is wrong: Token-type (segment) ids distinguish sentence A from sentence B in pair tasks; they do not zero out attention to padding, so the corruption persists.

Why B is correct: The attention mask sets padded key positions to a large negative value before the softmax, driving their attention weight to zero so padding cannot influence real-token representations.

Why C is wrong: Position ids only supply ordering information to the embeddings; restarting them changes positional signal but still lets padded keys contribute to the attention softmax.

Why D is wrong: Averaging a reversed copy is not a standard transformer input and does nothing to remove padded keys from the attention computation, so the corruption remains.