PMLE domain - 20% of the exam

Serving and Scaling Models

Serving and Scaling Models is 20% of the PMLE exam. These are the objectives it covers, each with practice questions, with every answer explained.

The domain in numbers

57
Practice questions
2
Objectives
28
Medium
29
Hard

Where people go wrong

  • Raising a target to cut tail latency

    The wrong answers raise a number to protect tail latency: a higher autoscaling target, a higher maximum-concurrent-requests setting, or a higher scaling threshold. Each one lets replicas run hotter or queue more requests before the autoscaler adds capacity, so queueing and p99 latency get worse. The correct answers lower the target utilisation or the concurrency limit, so replicas saturate sooner and scale-out starts while spare capacity remains.

    See it in a sample question
  • Sizing for peak instead of autoscaling from a low floor

    The wrong answers pin the minimum replica count to peak, run a fixed pool sized for peak load, or pre-warm on a fixed schedule. Peak-sized capacity is billed through idle periods, which the questions rule out, and a nightly schedule cannot cover a burst that arrives at an unannounced time. The correct answers keep autoscaling with a low floor: a small minimum replica or instance count where the first burst must not wait for a cold start, or scale-to-zero on Cloud Run where sporadic traffic makes idle cost the priority.

    See it in a sample question
  • Moving transforms to the client or the wrong layer

    The wrong answers place preprocessing or postprocessing anywhere except the serving container: on each client, in a front proxy, or inside the model itself. Client-side logic is duplicated across callers, which invites inconsistent thresholds and training-serving skew, and a threshold baked into the model forces a retrain. The correct answers run tokenisation, resizing, softmax and thresholds in the server handler, or in a custom prediction routine's preprocess and postprocess steps, so every client receives one consistent response.

    See it in a sample question

Objectives in this domain

Sample questions from this domain

Free sampleServing and Scaling Modelsmedium

An online recommendation endpoint autoscales on a target requests-per-second metric. Each replica is configured with a high maximum-concurrent-requests setting to maximise throughput per replica. During a traffic spike, the throughput numbers look healthy and few new replicas are added, yet the 95th-percentile latency climbs far above the agreed service objective. The team wants to bring tail latency back under control during spikes without permanently over-provisioning idle replicas. Which change best addresses the latency regression?

  • ALower each replica's maximum-concurrent-requests setting so replicas saturate sooner, triggering the autoscaler to add capacity earlier when requests start queuing. Correct
  • BRaise the autoscaler's target requests-per-second threshold so replicas are added later, keeping each replica fully utilised and reducing overall cost during spikes.
  • CRemove the autoscaler and run a fixed large pool of replicas sized for peak traffic to guarantee headroom at all times.
  • DMove feature lookups to a larger cache so each request spends less time fetching features, eliminating the queuing that drives tail latency during spikes.
Understand that a high per-replica concurrency limit masks queuing, so tuning it down lets the autoscaler react to latency pressure during spikes. When throughput is the scaling signal but each replica accepts many concurrent requests, a replica can hold a deep queue while its measured requests-per-second still looks fine, so the autoscaler stays idle and tail latency grows. Lowering the concurrency limit forces earlier saturation and triggers scaling sooner.

Why A is correct: A high concurrency limit lets one replica queue many in-flight requests while throughput still looks healthy, hiding the backlog; lowering it makes replicas saturate sooner so the autoscaler adds capacity before queuing inflates tail latency.

Why B is wrong: Higher thresholds do reduce replica count and cost, which sounds efficient, but they let each replica absorb even more load before scaling, deepening the per-request queuing that is already inflating tail latency.

Why C is wrong: A fixed peak-sized pool would hold latency down, making it tempting, but it permanently over-provisions idle replicas during normal traffic, which the team explicitly wants to avoid.

Why D is wrong: Faster feature lookups can reduce per-request work and are a real serving optimisation, but the bottleneck here is requests piling up behind a high concurrency limit, not slow feature reads, so caching does not address the queuing.

Free sampleServing and Scaling Modelsmedium

A team serves a fraud-scoring model on a Vertex AI dedicated endpoint. Traffic is near zero overnight but spikes sharply at 09:00 each weekday, and the first burst of requests times out before capacity catches up. They want the endpoint to absorb the morning spike without paying for idle replicas overnight. Which configuration change best meets both goals?

  • ASet minReplicaCount to a higher fixed value equal to the peak so replicas are always provisioned for the spike.
  • BDisable autoscaling and run a single large machine type so one replica handles all traffic at any hour.
  • CLeave the defaults but add a client-side retry loop so timed-out morning requests are resubmitted until replicas are ready.
  • DKeep a small minReplicaCount above zero, raise maxReplicaCount well above peak, and lower the target utilisation so scale-out triggers earlier. Correct
Tune Vertex AI endpoint autoscaling by combining a warm minimum floor, a high ceiling, and an earlier scale-out trigger to absorb spikes economically. Autoscaling reacts to a utilisation signal, so a non-zero minimum keeps warm replicas that answer the first burst while a lower utilisation target makes the scaler add replicas before the request queue saturates, and a high maximum lets it reach peak; together they cover the spike without holding peak capacity overnight.

Why A is wrong: Pinning the floor to peak capacity does remove the cold-start delay, but it keeps every peak replica running through the idle overnight window, which directly contradicts the requirement to avoid paying for idle capacity.

Why B is wrong: A single oversized replica looks simpler and avoids scaling lag, but one replica cannot scale out for a sharp concurrent spike and becomes a throughput bottleneck during the 09:00 burst.

Why C is wrong: Client retries can mask occasional failures, but they add load to an already saturating endpoint during scale-out and do nothing to provision capacity earlier, so the morning timeouts persist.

Why D is correct: A small warm floor avoids the cold-start timeouts on the first burst, the high ceiling lets the endpoint scale out for the spike, and a lower utilisation target makes the autoscaler add replicas before the queue saturates.

Free sampleServing and Scaling Modelshard

A PyTorch image-classification model is deployed in a custom container on Cloud Run for online inference. The raw model outputs logits over 1,000 classes, but the client application expects a single top label with a confidence between 0 and 1, and labels below 0.6 confidence must be reported as 'uncertain'. Where should the softmax, top-label selection, and threshold logic be implemented?

  • AIn the container's postprocessing step after the model forward pass, converting logits to probabilities, selecting the top label, and applying the 0.6 threshold before responding. Correct
  • BInside the training script, by retraining the model so its final layer emits a single calibrated label and the threshold rule directly.
  • CIn the client application, which should receive the 1,000 logits and apply softmax, argmax, and the 0.6 threshold itself before display.
  • DIn a separate batch job that periodically rewrites stored predictions, so the online endpoint can return precomputed labels instead of running the rule live.
Inference postprocessing such as softmax, label selection, and thresholds belongs in the serving container so all clients share one consistent response contract. The serving container wraps the model forward pass and is the single point that every request passes through, so converting logits to probabilities, choosing the top label, and applying the uncertainty threshold there guarantees a uniform response shape without duplicating logic in clients or coupling rules to retraining.

Why A is correct: Postprocessing inside the serving container turns raw logits into the contracted response shape in one place, so every client receives a consistent top label, confidence, and uncertainty rule.

Why B is wrong: Baking business rules into the model architecture couples a tunable threshold to a costly retrain, so although it removes serving code it is the wrong place for postprocessing that may change with product requirements.

Why C is wrong: Pushing softmax and thresholds to every client duplicates logic across callers and ships large logit payloads over the network, which is tempting for thinness but fragments the serving contract.

Why D is wrong: Batch rewriting suits offline scoring, but an online endpoint must transform each fresh request synchronously, so precomputed labels cannot serve novel images arriving in real time.

Other domains in this exam

See also the PMLE cert hub, the study guide, and the cheat sheet.

Examworthy is not affiliated with or endorsed by Google Cloud. Original, blueprint-aligned practice material only.