Public vs Private Cloud costs

The Hidden Price of the Public Cloud: Why Private Cloud Wins Long-Term?

Introduction

Artificial Intelligence (AI) is driving the next wave of innovation, and businesses across every industry are racing to integrate it into their products and operations. But as the demand for AI workloads grows, so does the cost, especially for those relying on the public cloud. Training large models and running inference at scale on public infrastructure quickly becomes unsustainable for companies investing heavily in AI. This has sparked a growing shift: forward-thinking organisations are turning to private cloud solutions to regain control over performance, security, and cost efficiency. For those aiming at medium to long-term growth, private cloud isn't just a viable alternative, it's a strategic advantage.

Public Cloud cost example

Consider a scenario where your organisation wants to invest in AI initiatives. This will require access to a platform equipped with graphics processing units (GPU’s).

The type of GPU required depends on the type of AI workloads. We will look at an example whereby we are wanting to run several medium sized LLMs and RAG workloads which require a substantial amount of GPU memory. 

For this example, we will look at using H100 NVL GPUs which have 94GB GPU memory per GPU.

In Microsoft Azure, there are several H100 options. If we take the following:

GPU optimised compute:

Running six of these virtual machines in Microsoft Azure would cost approximately £449,593 per year. Over a typical 3 year period, commonly aligned with standard support contract durations, the total cost would amount to £1,348,779.

The total resources available for the 6 Azure VMs are:

CPU = 240
RAM = 1,920GB
GPU memory = 564GB
Storage = 6TB premium SSD v2

You get only the above resources and there are no additional features with that.

If we now take a few Private Cloud scenarios using Broadcom’s flagship VMware platform and HPE infrastructure, let’s see how the cost compares.

To understand the different licensing models for VMware, please refer to the following blog.

Private Cloud cost examples

The below examples have been put together to get as close as possible to the public cloud offering, however a like for like is impossible as private cloud in general offers a multitude of additional features which public cloud does not.

Hardware requirements 1 - VVF

3 HPE DL380 G11 servers
32 physical cores per server
10TB vSAN ESA
256GB RAM per server
2 H100 GPU per server
VVF licensing
*Network infra (physical firewalls, 2 TOR switches)
*DC costs (rack, power, IP transit)

*not everyone requires datacenter costs and network Infrastructure as most businesses will already have this in place.

 Total initial one-off costs: £216,881
Total monthly costs for 3 years: £950 + (£1500 + £2500) = £4950

The monthly cost amounts to £59,400 per year and £178,200 over 3 years. This amounts to a grand total of £395,081 which is 28% of the cost of public cloud.

From this private cloud, the resources that are available are as follows:

vCPU = ~480
RAM = 768GB
GPU memory = 564GB
Storage = 10TB all flash

In this private cloud, you can build as many virtual machines as it can hold based on the above resources without any additional costs other than the above fixed costs. This also includes sharing GPU resources between VMs.

In addition, you get the following features:

• vSphere Enterprise Plus (ESXi license)

• vCenter Server Standard (vCenter license)

• vSphere with Tanzu (includes TKG Runtime) 

• vSAN Enterprise (*includes 250GiB per CPU Core per host)

• Aria Suite Standard

• Aria Suite Lifecycle

• Aria Automation Orchestrator (Basic)

• Aria Operations

• Aria Operations for Logs

While the initial investment in a private cloud is higher than that of a public cloud solution, the public cloud incurs a recurring cost of £37,466 per month, totalling approximately £224,796 over just six months. Beyond this point, a private cloud becomes the more cost-effective option. 

For organisations seeking advanced capabilities such as automation, software-defined networking, and streamlined lifecycle management, Broadcom’s flagship solution, VMware Cloud Foundation, offers a comprehensive and feature rich platform to support and scale your private cloud infrastructure.

See the breakdown below.

Hardware requirements 2 - VCF

Mgmt. domain

4 HPE DL380 G11 servers
16 physical cores per server
5TB vSAN ESA
64GB RAM per server

Workload domain

3 HPE DL380 G11 servers

32 physical cores per server

10TB vSAN ESA

256GB RAM per server

2 H100 GPU per server

Licensing, network, DC

VCF licensing with NSX

*Network infra (physical firewalls, 2 TOR switches)

*DC costs (rack, power, IP transit)

*not everyone requires datacenter costs and network Infrastructure as most businesses will already have this in place.

Total initial one-off costs: £251,208
Total monthly costs for 3 years: £3877 + (£2500 + £1500) = £7877

The monthly cost amounts to £85,524 per year and £256,572 over 3 years. This amounts to a grand total of £507,780 which is only around one third of the cost of public cloud.

From this private cloud, the resources that are available are as follows:

Mgmt domain:

vCPU = 64
RAM = 256GB
Storage = 5TB all flash

This cluster is designated for standard virtual machine workloads that do not require GPU resources. It will also host all management virtual machines (VMs) associated with the VMware Cloud Foundation software-defined data center.

Workload domain:

vCPU = ~480
RAM = 768GB
GPU memory = 564GB
Storage = 10TB all flash

In addition, you get the following features:

• SDDC Manager

• vSphere Enterprise Plus (ESXi license)

  • vCenter Server Standard  (vCenter license)

  • vSphere with Tanzu (includes TKG Runtime)

• vSAN Enterprise (includes 1TiB per CPU Core)

• NSX Enterprise Plus

• Aria Suite Enterprise

  • Aria Automation

  • Aria Automation Orchestrator

  • Aria Operations

  • Aria Operations for Logs

• Aria Operations for Networks Enterprise

• HCX Enterprise

• VMware Data Services Manager (DSM)

 Final Thoughts

The comparison above clearly highlights that private cloud is the more efficient and cost-effective choice for environments requiring high-performance computing or a medium to large number of virtual machines. 

Public cloud offerings are well-suited for quick deployments, short-term testing, or initial proof-of-concept phases due to their flexibility and ease of setup. 

However, once requirements are defined and a growth strategy is in place, transitioning to a private cloud provides greater control, scalability, and long-term financial advantage.

Below is a summary comparison of the above scenarios side by side.