NCA-AIIO - AI Infrastructure (40% of the exam) - Section 2.4
Articulate the advantages, challenges, and considerations of on-premise versus cloud infrastructure.
Describe the trade-offs between on-premise GPU infrastructure - with predictable costs, data sovereignty, and low-latency access - and cloud-based GPU instances that offer elasticity and lower upfront capital expenditure. Weigh these factors against workload characteristics such as continuous long-running training jobs versus sporadic or burst inference demand.
Practice question for this objective
lock_openFree sampleAI Infrastructuremedium
An AI team runs a large language model training workload that consumes a consistent, predictable level of GPU compute every day of the year. Which financial characteristic of on-premise infrastructure makes it the more cost-effective choice for this scenario?
AOn-premise hardware is purchased as a capital expenditure, so high steady-state utilisation spreads the fixed cost over time, yielding a lower effective cost per GPU-hour than cloud rental.check_circle Correct
BOn-premise infrastructure is billed as an operational expenditure, so the team avoids large upfront payments and benefits from elastic scaling during the steady workload.
CCloud providers charge a flat annual fee for reserved GPU instances, making cloud equivalent to on-premise capex and removing any cost advantage for steady workloads.
DOn-premise infrastructure incurs zero operational costs because the hardware is owned outright, so the total cost of ownership is simply the purchase price.
Explain why high steady-state utilisation favours on-premise capital expenditure over cloud operational expenditure for AI workloads. On-premise GPU infrastructure is acquired as a capital expenditure: a large upfront outlay that is then depreciated over the asset lifetime. When a workload runs at consistently high utilisation, the amortised cost per GPU-hour falls well below the equivalent cloud rental rate, because the cloud provider's margin, redundancy, and multi-tenancy overhead are all embedded in the hourly price. The break-even point typically falls within one to three years, after which the on-premise cost is essentially the ongoing operational cost only. This capex-versus-opex dynamic is the primary financial justification for private AI infrastructure at scale.
Why A is correct: When utilisation is consistently high and predictable, the upfront capital expenditure amortises across every workload hour, undercutting the per-hour cloud rate over a multi-year horizon.
Why B is wrong: On-premise hardware is a capital expenditure, not an operational one. Describing it as opex conflates the financial model with cloud billing, which is the opposite arrangement.
Why C is wrong: Reserved cloud instances do reduce per-hour rates, but they are still opex payments and typically remain more expensive than owned hardware at very high steady-state utilisation, so the premise is incorrect.
Why D is wrong: Owned hardware carries ongoing operational costs including power, cooling, networking, maintenance, and staff, so zero operational cost is factually wrong and overstates the capex advantage.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-AIIO bank for this domain.
Burst scaling: provisioning extra GPU instances during peak encoding periods adds unexpected charges that compound over a six-month observation window.
errorWhy it is wrong: Burst scaling costs relate to compute instance pricing during demand spikes, not to the cost of transferring data out of the cloud. The scenario specifically identifies data transfer as the dominant expense, so compute elasticity charges are not the explanation.
Dedicated single-tenant bare-metal GPU nodes reserved for the full year to guarantee a fixed slice of capacity
errorWhy it is wrong: Year-long reserved bare-metal capacity is a genuine cloud purchasing option and tempting for guaranteed performance, but reserving peak-sized capacity for a whole year contradicts the goal of paying only for the few spike hours, leaving the reservation idle and paid for most of the time.
On-premise hardware delivers higher GPU throughput than equivalent cloud instances, reducing the time sensitive patient data spends in the processing pipeline.
errorWhy it is wrong: Processing throughput is a performance consideration, not a data residency or sovereignty one. High throughput does not address the regulatory requirement that data remain within the organisation's physical control.