After customising an ASR model, a team needs to deploy it as a low-latency GPU inference service that several applications will call concurrently. They want NVIDIA's recommended path for serving the optimised model rather than running the training framework in production. Which deployment step aligns with this goal?
- AKeep the training script running and have each application import it directly to call the model object in memory.
- BExport the fine-tuned model and serve it through an optimised inference server so requests are handled without loading the training framework. Correct
- CConvert the model to run on the CPU only so it can be deployed onto generic nodes without GPU scheduling complexity.
- DDistribute the raw checkpoint file to each application and let them load and run it independently on their own hardware.
Why A is wrong: Reusing the training code feels simple because the model is already loaded there, but the training stack is not built for concurrent low-latency serving and ties every client to that process, so it does not scale as a service.
Why B is correct: Production serving expects an exported, inference-optimised artefact behind a dedicated inference server, which removes the training framework overhead and supports concurrent GPU-backed requests with low latency.
Why C is wrong: Avoiding GPU scheduling reduces operational complexity, which is the temptation, but CPU-only ASR inference is far slower and contradicts the stated low-latency, GPU-served requirement.
Why D is wrong: Shipping the checkpoint looks self-contained, but it duplicates model loading across clients, leaves optimisation and batching to each one, and provides no shared low-latency serving layer, so it is not the recommended production path.