PMLE - Collaborating Within and Across Teams to Manage Data and Models (16% of the exam) - Section 2.3

Track and run ML experiments, choosing the right environment among Experiments on the Agent Platform, Agent Platform Pipelines, and Kubeflow Pipelines, evaluating predictive and gen AI solutions including LLM-as-a-judge, and tracking artifacts, versions, and lineage with ML Metadata.

Track runs, metrics, and hyperparameter combinations using Agent Platform Experiments, and choose between Agent Platform Pipelines and Kubeflow Pipelines to orchestrate multi-step workflows at the right abstraction level. Apply LLM-as-a-judge to evaluate generative AI outputs, and use ML Metadata to record artifact versions and lineage across experiments.

Agent Platform ExperimentsKubeflow PipelinesLLM-as-a-judgeML Metadata

Practice question for this objective

Free sampleCollaborating Within and Across Teams to Manage Data and Modelsmedium

A team trains three versions of a churn model in separate runs under one Experiment on the Agent Platform. Each run logs the model artifact plus AUC, precision and recall as parameters and metrics. The lead now wants to pick the run with the best AUC, see exactly which dataset version and hyperparameters produced it, and promote that specific artifact to staging. Which capability lets them rank the runs and trace the winning artifact back to its inputs without re-running anything?

  • AExporting all three runs to a notebook and recomputing AUC locally so the artifacts can then be compared side by side outside the Experiment.
  • BComparing the logged runs in the Experiment to sort by AUC, then following the selected run's tracked artifact and parameters back to its dataset version and configuration. Correct
  • CEnabling autologging on a fresh run so the framework captures AUC again and overwrites the earlier runs with a single comparable record.
  • DDeploying each artifact to a separate endpoint and reading AUC from the serving logs to decide which version performed best at training time.
Use Experiments on the Agent Platform to compare runs by metric and trace the winning artifact back to its dataset version and parameters. Each run within an Experiment stores its metrics, parameters and artifact references, so the platform can sort runs by a chosen metric and the lineage recorded for the selected run links its artifact to the exact dataset version and hyperparameters that produced it, with no re-execution needed.

Why A is wrong: Recomputing locally is tempting because notebooks feel flexible, but the metrics are already logged in the Experiment, so recomputation wastes effort and discards the tracked lineage linking artifact to inputs.

Why B is correct: Experiments on the Agent Platform record each run's metrics, parameters and artifacts, so the team can sort runs by AUC and trace the chosen run's logged inputs and artifact lineage directly.

Why C is wrong: Autologging sounds like the right tracking feature, but it captures a new run rather than ranking existing ones, and it would not overwrite or consolidate the three runs already recorded.

Why D is wrong: Serving metrics seem like a measurement source, but endpoint logs report serving behaviour, not the training AUC already logged, so this neither ranks the runs nor recovers their training inputs.

See more PMLE practice questions, answers explained.

Exam traps in Collaborating Within and Across Teams to Manage Data and Models

Answers that look right on this material and are not. Each one is a distractor from a different question in the PMLE bank for this domain.

  • The pipeline's component caching, which stores past step outputs so the evaluation artifact can be matched to its model by reusing the cached result on demand.

    Why it is wrong: Caching is appealing because it persists step outputs, but it exists to skip redundant recomputation, not to record which artifact fed which step, so it cannot answer the lineage query.

  • Compute accuracy by matching each generated summary exactly against a reference summary and rejecting any output that differs from the reference text.

    Why it is wrong: Exact-match accuracy is tempting because it is automated and familiar, but free-form summaries legitimately vary in wording, so exact matching penalises valid outputs and fails to measure relevance or faithfulness.

  • An Experiments run dashboard, because the metric and parameter charts for each run fully reconstruct the upstream dataset and preprocessing lineage of a model.

    Why it is wrong: An Experiments dashboard is tempting because it stores per-run parameters and metrics, but it focuses on run comparison rather than linking artifacts to the executions that produced them, so it does not fully capture lineage.

Examworthy is not affiliated with or endorsed by Google Cloud. Original, blueprint-aligned practice material only.