Core Machine Learning and AI Knowledge

A production system must answer questions that may require either a code interpreter, a retrieval tool, or an image captioner depending on the query. A large language model decides at runtime which tool to invoke, inspects the result, and may call additional tools before generating the final answer. This design is best described as which type of orchestration?

• AModality orchestration, because different input types are routed to modality-specific models before a response is assembled
• BLate fusion, because each specialist tool returns a partial output that is aggregated into one final response
• CEarly fusion, because all potential tool outputs are pre-encoded and concatenated into a shared context before the model responds
• DAgent orchestration, because an LLM acts as a planner that selects tools, inspects intermediate results, and iterates before producing a final answercheck_circle Correct

Why A is wrong: Modality orchestration routes data to models specialised by input type using mostly static or rule-based dispatch. It does not imply a planning loop, tool inspection, or multi-step reasoning by the model itself.

Why B is wrong: Late fusion is a model architecture concept where independent predictions from separate branches are combined. It describes a fixed differentiable pipeline, not a dynamic runtime reasoning loop driven by an LLM.

Why C is wrong: Early fusion joins raw inputs before encoding, and applies to training-time architecture choices. Pre-fetching every possible tool result before inference would be impractical and is not what the described system does.

Why D is correct: Agent orchestration is characterised by a model that autonomously plans, dispatches tool calls, observes results, and decides whether further actions are needed. This is distinct from modality orchestration, which uses static routing without iterative reasoning.

A team is building a model that classifies short clips by combining a video stream and a sensor time-series. They want the two modalities to influence each other's intermediate representations during encoding, not just at the input layer or after each modality has produced its own final prediction. Which fusion approach satisfies this requirement?

• AIntermediate fusion, in which each modality is first processed by its own encoder and the resulting feature maps are exchanged through cross-attention or fusion layers partway through the network.check_circle Correct
• BEarly fusion, in which the raw video tensor and the raw sensor tensor are concatenated into a single input vector that one shared encoder processes from the first layer onward.
• CLate fusion, in which each modality is encoded and classified independently and only the two final class-probability vectors are combined by averaging or a small learned head.
• DDecision-level voting, in which each modality casts a discrete predicted label and the final answer is chosen by majority count across the two modality votes.

Why A is correct: Intermediate (hybrid) fusion keeps modality-specific encoders yet allows their hidden representations to interact through cross-attention or fusion layers, which is exactly the mutual mid-network influence the team wants.

Why B is wrong: Early fusion is tempting because it lets the modalities mix, but it joins them only at the raw input layer with no modality-specific encoding, so it does not provide cross-influence between separately encoded intermediate representations as required.

Why C is wrong: Late fusion is attractive for its modularity, but it combines only the final predictions, so the modalities never influence each other's intermediate representations, failing the stated requirement.

Why D is wrong: Decision-level voting feels like a simple way to use both modalities, but it operates purely on final discrete labels and discards all intermediate features, so no representational cross-influence occurs.

A transformer block consists of a multi-head self-attention sub-layer followed by a position-wise feed-forward sub-layer. A candidate argues the feed-forward sub-layer is redundant because attention already mixes information across tokens. What is the most accurate description of the distinct role the feed-forward sub-layer plays?

• AIt mixes information across token positions a second time using a different set of attention weights, doubling the effective receptive field of the transformer block.
• BIt computes the positional encodings for each token so that sequence order information is refreshed and reinjected at every successive layer of the network.
• CIt applies the same two-layer non-linear transformation independently to each token position, adding representational capacity that attention's largely linear mixing does not provide.check_circle Correct
• DIt normalises the attention output across the feature dimension to keep activations within a stable numerical range as signals pass between successive blocks.

Why A is wrong: Tempting because stacking sub-layers does extend modelling power, but the feed-forward sub-layer operates on each position independently and performs no cross-token mixing; only the attention sub-layer moves information between positions.

Why B is wrong: Tempting because position matters in transformers, but positional information is added once at the input via positional encodings; the feed-forward sub-layer contains learnable transformation weights, not a positional-encoding generator.

Why C is correct: Correct: the feed-forward sub-layer is a per-position multilayer perceptron with a non-linearity, applied identically to every token. It transforms each token's representation independently, supplying non-linear capacity that the attention-weighted averaging of value vectors does not by itself contribute.

Why D is wrong: Tempting because normalisation is part of a transformer block, but that job belongs to the layer-normalisation step, not the feed-forward sub-layer, which is a non-linear projection rather than a normaliser.

An engineer trains a vision-language model and plots the training loss against step count. The curve descends smoothly for the first few thousand steps, then oscillates up and down with large amplitude without settling, even though the average level keeps drifting down slowly. The batch size and data are unchanged throughout. Which hyperparameter adjustment most directly addresses this oscillation?

• ARaise the dropout probability in every layer to add more regularisation to the network.
• BApply a learning-rate schedule that decays the rate as training progresses, for example cosine decay.check_circle Correct
• CIncrease the weight-decay coefficient so the optimiser penalises large weights more heavily.
• DSwap the activation function from ReLU to GELU throughout the network.

Why A is wrong: More dropout targets overfitting on held-out data and would add training noise rather than remove it, so it does not explain or fix loss oscillation driven by step-size dynamics.

Why B is correct: Large late-stage oscillation around a slowly improving mean is the signature of a learning rate that is too high near the minimum; decaying it lets the optimiser take smaller steps and settle into the basin.

Why C is wrong: Weight decay shrinks parameter magnitudes to curb overfitting, which is a plausible-sounding knob, but it does not govern the step size that produces overshoot-driven oscillation near the minimum.

Why D is wrong: Changing the activation alters representational behaviour and might be tried for accuracy, but it does not control the optimiser step size, so the oscillation caused by an over-large learning rate would persist.

A team is deciding how to represent a categorical feature with only three unordered values (red, green, blue) as input to a small feed-forward neural network. They want a representation that does not imply any ordinal relationship between the values. Which encoding best satisfies this requirement, and why?

• AOne-hot encoding, producing a three-element binary vector per value, because each category occupies an independent dimension with no implied ordering or magnitude.check_circle Correct
• BInteger encoding, mapping red to 0, green to 1, and blue to 2, because it produces the most compact single-column tensor for the network to consume.
• CMin-max normalisation of the integer codes into the range 0 to 1, because scaling the codes removes the ordinal information carried by the raw integers.
• DA learned dense embedding of width two, because embeddings always outperform sparse encodings on categorical inputs regardless of cardinality.

Why A is correct: One-hot encoding places each category on its own orthogonal axis, so no value is numerically larger than another and the network receives no spurious ordinal signal, which is exactly what an unordered low-cardinality feature needs.

Why B is wrong: Integer encoding is tempting because it is compact and simple, but a single ordered column implies that blue (2) is numerically greater than red (0), injecting a false ordinal relationship the network can latch onto for an unordered feature.

Why C is wrong: This looks reasonable because normalisation is a standard preprocessing step, but rescaling 0, 1, 2 to 0, 0.5, 1 preserves the relative ordering and spacing, so the false ordinal relationship remains intact.

Why D is wrong: Embeddings are powerful for high-cardinality features, but the claim that they always outperform sparse encodings is false, and for only three categories a learned embedding adds parameters and training cost with no representational benefit over one-hot.