Sovereign AI vs Cloud AI: Which Is Right for Your Business?
Cloud AI suits most Australian businesses; sovereign AI on your own GPU servers is worth it when data must stay under your control or volume is high and steady. A 2026 comparison of data location, cost, control and hardware, with sources.
For most Australian businesses, cloud AI is the right starting point: it is faster to set up, billed per use and gives access to the most capable models, and the major platforms let you keep data in an Australian region if you configure them to. Sovereign AI, meaning open-weight models running on GPU servers you control in an Australian facility, is the better choice when data must not leave your control (government, health, legal and some financial work), when you need to customise models on your own data, or when AI use is high and steady enough to keep dedicated hardware busy. Many businesses end up with both.
This guide compares the two on data location, cost, control and hardware, using sources checked in November 2026. Nublock's sovereign AI infrastructure service covers the build side.
What is sovereign AI?
Sovereign AI means the location of the data, access to it, ownership of the infrastructure and responsibility for running it all sit with your organisation, or a trusted provider in your jurisdiction. In practice it means running open-weight models on GPU servers you own or lease, on-premises, in a co-location facility or in private hosting in Australia, so prompts and documents never go to a shared public AI service.
What is cloud AI?
Cloud AI means calling models hosted by a provider, such as Azure OpenAI and other models in Microsoft Foundry, Amazon Bedrock or Google Cloud Vertex AI, or using an AI product like Microsoft 365 Copilot. You send a prompt to the provider's API, it runs on their GPUs and you pay per token or per seat. You do not manage the hardware, and the largest proprietary models are only available this way.
The key differences
Where the data is processed
With sovereign AI, you know exactly where data is processed: on your hardware, in your facility. With cloud AI, it depends on how the service is deployed. Microsoft's Foundry deployment types documentation is a good example of the trade-off:
- Global deployments may process prompts and responses in any Azure region. Microsoft recommends starting here because new models launch first and pricing is lowest.
- Data Zone deployments process data within a Microsoft-defined zone, such as Asia Pacific, which covers several countries.
- Standard and Regional Provisioned deployments process prompts within the Azure geography you choose. Microsoft says these geography-based types arrive last, with no guaranteed availability date.
In all cases Microsoft says data stored at rest stays in the chosen geography. So cloud AI can keep data in Australia, but only if someone picks the right deployment type and region, and the model you want is offered there. Check the equivalent settings on any other platform before sending it personal or confidential information.
Australian rules that affect the choice
- Government data. The Hosting Certification Framework applies to Australian Government customers hosting sensitive government data, whole-of-government systems and PROTECTED systems. New and extended contracts must use certified providers. The certified list includes the Australian regions of Microsoft Azure, Amazon Web Services and Google Cloud at the Strategic level, along with Australian data centre facilities. The framework paused new certification registrations from 3 November 2025 while it is reformed; existing certifications are not affected.
- Personal information sent overseas. APP 8 of the Privacy Act applies when personal information is disclosed to a recipient outside Australia, and the OAIC recommends, as a matter of best practice, not entering personal information into publicly available generative AI tools (OAIC guidance, updated 17 January 2025).
- Health information in Victoria. Health Privacy Principle 9 of the Health Records Act 2001 (Vic) allows health information to be transferred outside Victoria only in listed circumstances, and the Act has no small business exemption. For a Melbourne clinic, that is a reason to look closely at where any AI service processes patient information.
Whichever way you run AI, staff need to know which tools and accounts are approved for which information. Our free AI acceptable use policy template covers approved tools, banned information and these privacy clauses.
Cost structure
Cloud AI is operating expense: you pay per token or per seat, so cost follows use and a pilot costs little. Sovereign AI is mostly capital expense: GPU servers, networking, storage and hosting are paid for up front, and power, cooling and the people to patch and monitor the environment are ongoing. The cost per request only drops below cloud pricing when the hardware is kept busy most of the time.
There is no general break-even point, so do the comparison with your own numbers: estimate a year of token or seat costs at your expected volume, then compare it with the hardware cost spread over its useful life, plus hosting, power and support. If the cloud figure is small or your volume is uncertain, start in the cloud.
Control and customisation
Sovereign AI lets you choose the model, fine-tune it on your own data, pin a version for as long as you like and set your own access controls and logging. Open-weight models make this practical: OpenAI's gpt-oss-120b and gpt-oss-20b (August 2025) are released under the Apache 2.0 licence, and OpenAI says gpt-oss-120b runs on a single 80 GB GPU while gpt-oss-20b needs 16 GB of memory. Cloud AI limits you to the models and settings the provider offers, and providers retire model versions on their own schedule.
Performance and capability
The most capable proprietary models are only available as cloud services, so for the hardest reasoning or writing tasks, cloud AI usually wins. Self-hosted models give you predictable latency and throughput because the hardware is not shared, which suits steady internal workloads such as document extraction, classification and search over your own files.
When to choose sovereign AI
- Data that must stay under your control: government, defence, health, legal or financial information where contracts or regulation limit where it can be processed.
- High, predictable volume: workloads that would keep GPUs busy, where per-token costs would add up.
- Custom models: fine-tuning on proprietary data, or running a specialised open-weight model.
- Isolated environments: requirements for network-segmented or air-gapped AI.
When to choose cloud AI
- Low or variable volume: pilots, occasional use, or work that grows unpredictably.
- Speed: you need something working in weeks, without buying hardware.
- The most capable models: tasks that need models only available as hosted services.
- A small IT team: no capacity to run GPU servers, patching and monitoring.
If you choose cloud AI, set it up so data stays where it should: use business accounts signed in with your work identity, and choose regional deployments where the data requires it. Our secure Claude setup is one example of keeping cloud AI data in Australia.
The hybrid approach
Most organisations that run sovereign AI also use cloud AI. Sensitive data and steady workloads run on owned hardware, and everything else goes to cloud services. The routing rule should follow your data classification: the same categories your acceptable use policy uses to say which information may go into which tool.
Hardware for sovereign AI in 2026
The first sizing question is how much GPU memory the model needs. Lenovo's ThinkSystem GPU summary (updated 12 September 2026) shows the current range of NVIDIA options, for example:
| Scale | Example NVIDIA GPU on ThinkSystem | GPU memory | Suits |
|---|---|---|---|
| Single server, PCIe cards | RTX PRO 6000 Blackwell Server Edition | 96 GB per card | Inference with mid-sized open-weight models, search over internal documents |
| Single server, PCIe cards | H200 NVL, H100 NVL, L40S | Varies by card | Inference and smaller fine-tuning jobs |
| 8-GPU server | SR680a V3 with eight B200 (or B300 on the SR680a V4) | 180 GB HBM3e per B200 | Large models, training and fine-tuning, many concurrent users |
Beyond the GPUs, plan for:
- Networking: high-bandwidth Ethernet or InfiniBand where several GPU servers work together.
- Storage: NVMe for model weights and the documents the model searches.
- Power and cooling: a B200 is rated at 1,000 W per GPU, so an 8-GPU server needs far more power and cooling than a standard rack.
- Hosting: on-premises, co-location or private hosting in Australia.
- Export controls: Lenovo marks many of these GPUs as controlled, meaning US Government rules limit the markets they are sold into, so confirm availability early.
How Nublock helps
Nublock Digital is a Melbourne-based Lenovo hardware partner. Its sovereign AI work uses Lenovo ThinkSystem servers with NVIDIA accelerators, and for AI work, data stays in Australia, client data is not used for training and nothing is routed through Nublock accounts. To talk about which approach fits your business, get in touch.
Sources
- Microsoft Learn, Understanding deployment types in Microsoft Foundry Models
- Department of Home Affairs, Hosting Certification Framework and Certified Service Providers
- OAIC, Guidance on privacy and the use of commercially available AI products, 21 October 2024, updated 17 January 2025
- Victorian Department of Health, Health Records Act
- OpenAI, Introducing gpt-oss, 5 August 2025
- Lenovo Press, ThinkSystem and ThinkAgile GPU Summary, updated 12 September 2026
- Lenovo Press, ThinkSystem SR680a V3 with B200 product guide, updated 29 September 2026
- Lenovo Press, NVIDIA RTX PRO 6000 Blackwell Server Edition product guide, updated 28 July 2026
All sources checked on 16 November 2026.
What is sovereign AI?
Sovereign AI means running AI so that where the data is processed, who can access it, who owns the infrastructure and who operates it all sit within your control or your jurisdiction. For a business, that usually means running open-weight models on GPU servers you own or lease in an Australian facility, rather than sending prompts to a shared public AI service.
Is sovereign AI cheaper than cloud AI?
Only at high, steady volume. Cloud AI is billed per token, so it is cheaper for low or variable use. Sovereign AI has a large upfront cost for GPU servers, hosting, power and cooling, plus the people to run it, and the cost per request only falls once the hardware is busy most of the time. Compare a year of token costs at your real volume with the hardware cost spread over its life plus hosting and support.
Does Australia have sovereign AI or data hosting requirements?
For Australian Government data, yes: the Hosting Certification Framework requires new and extended hosting contracts for sensitive government data to use certified providers, and its certified list includes the Australian regions of Microsoft Azure, AWS and Google Cloud. The framework paused new certifications from 3 November 2025 while it is reformed. Private businesses have no general rule that AI must run in Australia, but APP 8 of the Privacy Act applies when personal information is disclosed overseas, and the Health Records Act 2001 (Vic) restricts transferring health information outside Victoria.
Can cloud AI keep data in Australia?
Sometimes, if you choose the right deployment. On Microsoft Foundry, Global deployment types may process prompts in any Azure region and Data Zone types anywhere in the Asia Pacific zone, while Standard and Regional Provisioned types process prompts within the Azure geography you choose. Microsoft says geography-based deployments get new models last, with no guaranteed availability date, so check that the model you need is offered in an Australian region.
What hardware is needed for sovereign AI?
GPU servers with enough GPU memory for the model, plus fast networking, NVMe storage, power and cooling. As examples, OpenAI says its open-weight gpt-oss-120b runs on a single 80 GB GPU, while Lenovo’s ThinkSystem GPU range runs from single NVIDIA RTX PRO 6000 Blackwell Server Edition cards with 96 GB each up to the SR680a V3 with eight NVIDIA B200 GPUs of 180 GB each.