PMLE - Serving and Scaling Models (20% of the exam) - Section 4.1

Serve models for batch and online inference using Agent Platform, Model Garden, Cloud Run, and GKE, packaging models from frameworks such as PyTorch and XGBoost with prebuilt and custom containers, versioning in the Model Registry, implementing rollout strategies such as A/B testing and canary deployments, and handling inference pre- and postprocessing.

Deploy models for online and batch inference on Agent Platform, Cloud Run, and GKE, packaging frameworks such as PyTorch and XGBoost in prebuilt or custom containers and versioning them in the Model Registry. Design rollout strategies such as canary deployments and A/B testing to reduce risk, and implement inference pre- and postprocessing to handle input transformation and output formatting.

Online and batch inferenceCustom containersModel RegistryCanary deployments

Practice question for this objective

Free sampleServing and Scaling Modelshard

A PyTorch model relies on a custom C++ operator and a SentencePiece tokenizer that are not present in any Vertex AI prebuilt serving container, and the team's inference handler must run tokenisation before the forward pass. They need online serving on Vertex AI. Which packaging approach lets them deploy this model successfully?

AUse the prebuilt PyTorch serving container and pass the tokeniser and custom operator as extra files in the model artifact directory at deploy time.
BConvert the model to the XGBoost format so the prebuilt XGBoost container can run the custom operator and tokeniser without extra dependencies.
CBuild a custom serving container that installs the C++ operator and tokeniser dependencies and implements the prediction route, then deploy it to a Vertex AI online endpoint. Correct
DDeploy the model to a batch prediction job, because batch jobs automatically install any missing system libraries the handler imports.

Choose a custom serving container when a model needs system libraries or custom operators that prebuilt containers do not include. Prebuilt containers ship a fixed dependency set and cannot gain unlisted system libraries or compiled custom operators at deploy time, so a model needing those plus custom preprocessing must run in a custom container that installs them and implements the prediction route.

Why A is wrong: Bundling extra files seems to extend the prebuilt container, but that container only installs its fixed dependency set, so an unlisted system library and a compiled custom operator will not be available at runtime and serving fails.

Why B is wrong: Switching to the XGBoost container sounds like a way to dodge missing PyTorch dependencies, but XGBoost is a gradient-boosting runtime that cannot execute a PyTorch graph or a custom C++ operator, so the model could not run at all.

Why C is correct: A custom container lets the team install the exact system libraries, compile the custom operator, and implement the preprocessing prediction route, which is the supported path when prebuilt containers lack required dependencies.

Why D is wrong: Choosing batch prediction appears to sidestep container limits, but batch jobs use the same container constraints and do not auto-install arbitrary system libraries, and the requirement is online serving rather than offline scoring.

See more PMLE practice questions, answers explained.

Exam traps in Serving and Scaling Models

Answers that look right on this material and are not. Each one is a distractor from a different question in the PMLE bank for this domain.

Google Kubernetes Engine with a custom autoscaler and an in-cluster ingress that the team configures to route a percentage of traffic to the new model version's pods.
Why it is wrong: GKE can achieve canary routing and GPU autoscaling, but it requires the team to build and operate node pools, autoscalers, and traffic-splitting ingress, which directly conflicts with their stated lack of Kubernetes operations capacity.
Build a custom container with a Flask server that loads the booster, register the image in Artifact Registry, and deploy it to a Vertex AI endpoint.
Why it is wrong: A custom Flask container does serve XGBoost, but it forces the team to write and maintain serving code and an HTTP layer, which contradicts the requirement of least operational overhead when a prebuilt container already exists.
Splitting traffic by percentage between two endpoints so that requests are distributed across model iterations automatically.
Why it is wrong: Percentage traffic splitting is a real rollout technique at the endpoint, but it governs how live traffic is routed and is not the registry mechanism that catalogues and lets you pin or revert to specific stored iterations.

Examworthy is not affiliated with or endorsed by Google Cloud. Original, blueprint-aligned practice material only.