NCA-AIIO - AI Infrastructure - Section 2.4

Articulate the advantages, challenges, and considerations of on-premise versus cloud infrastructure.

Describe the trade-offs between on-premise GPU infrastructure - with predictable costs, data sovereignty, and low-latency access - and cloud-based GPU instances that offer elasticity and lower upfront capital expenditure. Weigh these factors against workload characteristics such as continuous long-running training jobs versus sporadic or burst inference demand.

Practice question for this objective

Free sampleAI Infrastructuremedium

An AI team runs a large language model training workload that consumes a consistent, predictable level of GPU compute every day of the year. Which financial characteristic of on-premise infrastructure makes it the more cost-effective choice for this scenario?

  • AOn-premise hardware is purchased as a capital expenditure, so high steady-state utilisation spreads the fixed cost over time, yielding a lower effective cost per GPU-hour than cloud rental. Correct
  • BOn-premise infrastructure is billed as an operational expenditure, so the team avoids large upfront payments and benefits from elastic scaling during the steady workload.
  • CCloud providers charge a flat annual fee for reserved GPU instances, making cloud equivalent to on-premise capex and removing any cost advantage for steady workloads.
  • DOn-premise infrastructure incurs zero operational costs because the hardware is owned outright, so the total cost of ownership is simply the purchase price.
Explain why high steady-state utilisation favours on-premise capital expenditure over cloud operational expenditure for AI workloads. On-premise GPU infrastructure is acquired as a capital expenditure: a large upfront outlay that is then depreciated over the asset lifetime. When a workload runs at consistently high utilisation, the amortised cost per GPU-hour falls well below the equivalent cloud rental rate, because the cloud provider's margin, redundancy, and multi-tenancy overhead are all embedded in the hourly price. The break-even point typically falls within one to three years, after which the on-premise cost is essentially the ongoing operational cost only. This capex-versus-opex dynamic is the primary financial justification for private AI infrastructure at scale.

Why A is correct: When utilisation is consistently high and predictable, the upfront capital expenditure amortises across every workload hour, undercutting the per-hour cloud rate over a multi-year horizon.

Why B is wrong: On-premise hardware is a capital expenditure, not an operational one. Describing it as opex conflates the financial model with cloud billing, which is the opposite arrangement.

Why C is wrong: Reserved cloud instances do reduce per-hour rates, but they are still opex payments and typically remain more expensive than owned hardware at very high steady-state utilisation, so the premise is incorrect.

Why D is wrong: Owned hardware carries ongoing operational costs including power, cooling, networking, maintenance, and staff, so zero operational cost is factually wrong and overstates the capex advantage.

See more NCA-AIIO practice questions, answers explained.

More in this domain

Back to all AI Infrastructure objectives, or the NCA-AIIO cert hub.

Examworthy is not affiliated with or endorsed by NVIDIA. Original, blueprint-aligned practice material only.