Not Your AI, Not Your Business
A playbook for Sovereign AI
This is the second part of a series on Sovereign AI. The first part was just to set the scene and provide some context for further analysis. Now, we tackle the core issue: how does this apply to your company, and what should you do about it?
Previously, we defined the four tenets of Sovereign AI, which we will use as a lens to figure out the AI decisions you’re facing in your business.
1: Not your Data, not your Moat
By now, it’s widely known that the major unlock for LLMs was scaling the training data to the entire internet. Most of it legal, some of it less so.
The extent to which your business is isolated and protected from AI depends largely on how much or little your data is represented on the internet. This is data that the AI labs do not have access to currently. Well, unless of course you’re using free versions of ChatGPT, in which case you’re just handing over your data prompt-by-prompt.
Take the example of a law firm. You don’t have that much proprietary data because a lot of it will also be public record. In contrast, if you manufacture obscure industrial components, ChatGPT will not know the secret machine recipes and configurations you have learnt through trial and error over decades. Well, unless your employees used Reddit forums to share best practices.
Your business will hopefully never compete with the AI labs, but to stay relevant in the age of AI, you will need to have your own AI value proposition eventually.
The most obvious asset available is your data, and using it with LLMs. This can mean a few different things.
RAG: You can simply let LLMs access your data to find and summarize information for the user. This makes sense to start, as the data can be unstructured docs across silos, bypassing the typical enterprise data integration challenges.
Fine-tuning: If you do have proprietary data, you can start to experiment with fine-tuning LLMs. This is additional training that forces the LLM to specialize its output for your use case. Thanks to new techniques, frameworks, and platforms, this is increasingly cost-effective to try, especially with smaller open weight models.
Agents: The latest paradigm is to build your own agents. I previously covered a definition for agents, but to put it simply, they are in some ways like apps for LLMs, where you give the AI the ability to interact with your data and take actions across other systems through APIs.
In summary, your first order of business is auditing your data.
Checklist for Data:
What data do we have?
What data could we acquire?
What value can this data unlock?
What new insights might we uncover when combining data across silos?
How might we refine and enrich this data further?
What data is unique to you and not widely available on the internet?
What can you do with this data to build your own AI-enabled products and services?
What legal or technical issues prevent us?
2: Not your Weights, not your Models
Here’s where things start to get spicy. Now, some of you will disagree with my take here. Feel free to argue your case in the comments. I’m not against OpenAI as a company or product, but I am against letting companies unwittingly become trapped in their walled garden. If you wish to put all your eggs in Sam Altman’s gold-painted basket, do so at your own peril. Satya Nadella isn’t, and he pretty much owns OpenAI.
“Having competed against Oracle and IBM in client-server, I knew that the buyers will not tolerate winner-take-all. Structurally, hyperscale will never be a winner-take-all because buyers are smart. Consumer markets sometimes can be winner-take-all, but anything where the buyer is a corporation, an enterprise, an IT department, they will want multiple suppliers. And so you got to be one of the multiple suppliers.” — Satya Nadella
I understand the temptation. ChatGPT has become the Google of AI. Not as in search per se, but as in it is the default platform of consumers in AI. OpenAI kind of owns the consumer market for AI at this point. But if you’re reading this, you’re thinking about your business. That’s different from how you use AI as a consumer, or at least you should think about tradeoffs differently. Let’s break it down.
One is none, two is one
This is the famous saying at NASA, in reference to redundancy. If you have one working system, if anything goes wrong, you have zero systems. In space, that can mean complete failure, years of planning and billions of dollars wasted, and potentially a gruesome death for any astronauts involved.
In business, the stakes are rarely literally life or death. But as Satya Nadella says, the B2B does not tolerate the lack of choice any more than NASA does.
Surely, no one can get fired for choosing ChatGPT for their business? It’s like IBM in the 90s. Except that it’s not. IBM was never a closed-walled garden, despite best efforts. You could buy software and hardware from IBM, but they existed in an open architecture where you still had control over each layer separately. IBM servers and laptops also worked with Oracle software, for example.
Apple has successfully built a closed-walled garden on the consumer side, starting with the original iTunes, where purchased music only ran on the iPod. This seems normal now, but at this time the internet was rampant with illegal music downloads and most devices supported the MP3 format. Families that have iPhones tend to have MacBooks, use iCloud, have Apple TVs, and so on. But while MacBooks are the laptop of choice for most developers, the rest of the enterprise stack does not run on Apple software. They could never crack the enterprise market, because customers simply rejected Apple’s closed garden approach.
Obviously, prompts can easily be copy-pasted from one LLM to the next. What isn’t as easy is your entire chat history, the memories extracted from it, and especially agents you build. OpenAI has taken multiple stabs at the “apps for AI” concept with GPTs, marketplace, and so on, but the latest flavor is their Agent Builder. I was actually quite surprised that they decided to limit this new tool to OpenAI models. It’s understandable, but in my view unfortunate. It limits the viability of OpenAI as an enterprise platform provider.
ChatGPT is a walled garden for consumers, and so far, their strategy is to do the same for businesses. Everything you create cannot be exported and moved to Claude or Perplexity, for example. OpenAI controls the entire intelligence stack; you don’t. Don’t fall for it.
Open Models
If we want to be strict here, the only way to control the weights of a model is by training your own (not likely), or by using an open weight model, sometimes just called open source models. This is a critical distinction, as most open models you find on Huggingface do NOT come with the full source code, hence open weight is the more accurate description.
To train a model, you need two things: data and a training algorithm. Weights are the output or result of the training run. It’s like the difference between a recipe and a dish. If you don’t have the recipe, you can’t be 100% sure what you’re actually eating just by looking at it.
The major downside of open models is that, so far, they are always behind closed frontier models from major AI Labs. So if you need the best model, it won’t be an open model. Then again, you may not often need the very best model. If you’re doing simple tasks like summarization, you may get away with much smaller open models, which are also faster and cheaper to run.
The best part is that you remove the risk of this happening. Below is an example of how a judge in America has ordered OpenAI to hand over 20 million logs from ChatGPT users. Yes, that could be you.
When you run open models, the best way to do it is to run them yourself, either on-premise or one of the many cloud providers. That way, you can absolutely guarantee that no one else has access to your prompts, outputs, or logs.
Chinese AI
Let me cut to the chase here: do NOT use Chinese AI models!
DeepSeek has become the poster boy of the open model movement. The Chinese have “won” open models thanks to DeepSeek R1. On one hand, they seem incredibly open compared to US AI labs like OpenAI who publish nothing but blogs. DeepSeek publishes academic papers that tell you how they built R1 and other models.
The only thing is, they left out the parts that talk about CCP censorship.
Turns out, those were pretty important. According to this study by the cybersecurity firm CrowdStrike, DeepSeek will literally add vulnerabilities to your code.
“However, we found that when DeepSeek-R1 receives prompts containing topics the Chinese Communist Party (CCP) likely considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%.” — Crowdstrike
How can this be? Remember “open weights” does not include source code, or data! For all you know, there is a whole set of safeguards and carefully curated training data all serving the CCPs aims, not your’s!
When you look at benchmarks, the latest DeepSeek V3.2 models look incredible. Performance on par with frontier models at a tiny fraction of the cost.
Just think twice. For most of the things you want open models for, you can just as well use the latest models from Mistral (French), for example.
Shadow AI
Something that I see happening at every company is that AI is quickly creeping into their existing software tools. Suddenly, your video calls are being transcribed and summarized. There’s a copilot sidebar in your calendar. Where did this AI come from? Which model is it using? Where is my data being sent? Do they train their models on my business data? What warranties do they provide on the model outputs?
Yeah, you might want to look into that. It’s easier if you first set up your own AI policy, which you can then benchmark these tools against to validate that they are safe to use.
The issue your CFO will face with AI is that everything costs an additional $20. ChatGPT Teams is $20. Salesforce Einstein is another $20. Want to add AI to Notion? $20. Need AI inside Teams? That’ll be $20. You get the idea.
Suddenly, you’re paying hundreds of dollars per user per month, on top of your existing software licenses! That can’t work.
Pay per token instead of subscriptions
The alternative is to set up your own LLM server and pay per token via API. Without getting overly technical here, instead of paying each provider separately, you try to build agents yourself that can access that data across silos, and then just pay for every LLM API call separately. Just ask ChatGPT, it can explain it to you, or scroll down to the “AI OS” part at the end of this essay.
This approach also allows you to decide the cost vs. performance tradeoff for each use case. For simple summarization tasks, you can get away with lower-tier models, while agent orchestration likely calls for the big guns.
BYOAI?
If you’re old enough to remember BYOD, then you know what I’m talking about. When smartphones became a thing, the question every IT department faced was whether to buy everyone a phone or let employees access corporate data via unsecured personal devices. This is called Bring Your Own Device.
What is the AI equivalent? Many consumers will be paid users of one main AI tool they use, like ChatGPT, Gemini, Claude, or Perplexity. While ChatGPT dominates market share, the numbers are trending down as the others are catching up.
So this is the question: can we let consumers use their paid ChatGPT subscriptions at work? Rumors are that OpenAI are working on a feature like social login, where you can log in via ChatGPT, and use your own subscription to process requests while accessing data in other systems, such as corporate email and so on. I can see the CIO and CISO whincing already.
This may not be absolutely necessary, though, as Model Context Protocol (MCP) has emerged as a potential solution that allows users to authenticate to third-party platforms, giving their AI tool of choice access to data from popular services like Google, Figma, Salesforce, Atlassian, and so on using their own authentication details, whether private or work related.
The fact that OpenAI and Anthropic recently joined forces in setting up the Agentic AI Foundation under the Linux Foundation shows some early signs that AI Labs might choose to collaborate on an open enterprise-friendly AI stack, but the jury is still out.
Checklist for Models:
What AI tools are we using now? Have we reviewed their T&Cs?
Do we really even need frontier models for what we want to do?
What is the smallest open model that can do the task accurately?
Can we use our data to fine-tune open models to perform better?
How important is it whether 3rd parties can access our chat logs?
What happens if our current LLM provider changes terms or pricing?
Are we comfortable using Chinese vs. European vs. US models?
How do current or upcoming regulations limit your choices?
Can you separate above decisions between critical and peripheral use cases?
3: Not your Chips, not your Intelligence
As a business, you (probably) don’t need GPUs. I mean, unless you’re running super duper sensitive workloads and are already allergic to cloud. If you really cannot accept even a private cloud level of privacy, then I guess you’re going Nvidia shopping.
The problem is that to run even moderately sized open models at any scale of inference usage, you’re already spending millions. Potentially tens of millions if serving customers, not just employees. As we’ve seen, the depreciation on GPUs is also pretty brutal. Your CFO may not like what the spreadsheet is saying.
Instead, you can ride the current AI capex wave and enjoy subsidies by tech giants and neoclouds competing for market share. You can lease GPUs or just pay for tokens.
Checklist for Chips:
Do we absolutely need to run AI workloads on-premise?
What level of training & inference performance do we require?
What level of depreciation can our CFO live with?
Do we need to build in the Nvidia ecosystem or not?
4: Not your Equity, not your Technology
The importance of IP to your business depends, well, on your business. Most startups, whether wrappers or not, live and die by IP. Investors want to see that you’re doing something more than cool prompts.
Whether this means proprietary data, fine-tuned open models, custom agents, or even wrapper apps, it’s all IP of some kind. Prompts, not so much.
If you did end up just going with ChatGPT for Business, then you have zero IP. All the stuff you built on their platform, including GPT apps and agents, is effectively the IP of OpenAI. You cannot transfer it. You absolutely cannot sell it. You also cannot license it as part of a SaaS offering. You are limited to whatever monetization mechanisms OpenAI provides, and will have zero say in the terms and conditions imposed.
The solution is to build your own modular AI platform. While this has been technically challenging up to now, there are a lot of companies, including several of the tech giants, working tirelessly to make this approach easier even for startups and small businesses. More on that next.
Checklist for Equity:
What is our role in the AI value chain? End user, reseller, integrator, platform, or advisor?
Do we intend to license or sell our AI-related IP?
Do we need to own our means of intelligence in the future?
How will investors value our AI-related IP?
AI OS: A Playbook for Sovereign AI
So what’s the game plan to work though this maze of thorny choices? It’s what businesses have always done: open architecture. You separate the main components and have multiple providers for each of them. If you have technical or legal concerns with one, you swap them out (relatively) painlessly.
Due to the special strategic nature of AI, I’m calling this the AI Operating System (AI OS), as your future company will literally run on this stack.
Currently, as of late 2025, this is what an AI OS looks like.
Compute
The base layer where everything runs. If you need frontier models from closed labs like OpenAI or Anthropic, you are limited to their partner ecosystems, namely Microsoft Azure, AWS Bedrock, or Google Cloud. These are still much better options than going directly to a single lab. With these hosted cloud options, you have other model providers mixed in, and can choose to run your models in specific regions for data sovereignty and privacy compliance. As you might expect, they are also building a suite of frameworks and tools that help you manage your AI usage.
Beyond the usual suspects, there is the emerging category of “neocloud” providers such as Coreweave, Nscale, and Nebius. Currently, a lot of their capacity is going directly back to the AI labs themselves, but they are also making vertical plays into the software layer to increase their attractiveness to startup, enterprise, and government clients looking to run their own open models efficiently.
Models
The important thing with AI models is to separate them from AI applications. Instead of just choosing ChatGPT, Copilot, Gemini, or Claude as your one-stop shop for AI, you need your own LLM gateway. This sits between the LLM API providers and your applications initiating those prompts. This also means you can swap them out easily.
Right now, the models themselves are more of a commodity. There are many flavors, but they more or less do the same thing. Some are better at specific tasks than others, and there is a range of prices to choose from.
This may not always be the case. We could either have one lab make a major breakthrough, where if you aren’t using that model, you are at risk of falling behind immediately. For example, if one lab gets to AGI first, others might struggle to catch up. This is why we need the ability to switch models rapidly.
The most basic setup would be a web app chatbot, a backend server, and at least two different LLM provider API keys. You just need one backend, and can start connecting a multitude of apps, agents, and tools to your choice of LLMs under central control.
On the other hand, as we’ve moved past base LLMs into fine-tuning with Reinforcement Learning (RL), it may become increasingly relevant to train application or use-case-specific models. Here, the natural approach is to start from an open weights base model and add your proprietary data to the training loop, or even set up your own RL environments.
Finally, and perhaps most importantly, if you have mission-critical AI applications, you should control access to the model. The only way to guarantee this is to self-host the model, which therefore limits you to open-weight models. Oh, and avoid the temptation of Chinese LLMs, unless you are members of the CCP.
Agents
It’s all about the agents, now. The reason that agents are necessary for AI diffusion is that they can actually interact with the real world and other systems. Unlike a chatbot, the agent can use tools such as APIs to decide what to do, and then do it independently.
Here, OpenAI was one of the first to introduce their Agents SDK, which I think is not only fine but great. Unlike most things OpenAI is building, the Agents SDK is model-agnostic — you can use any model via the LiteLLM integration. Another popular choice would be AWS with AgentCore, in case you are already running on AWS. But there are many alternatives from the main cloud providers, and lots of open-source projects. Frankly, here I would rather pick something that has strong community support than build something on my own.
These agents live as some additional code in the backend server you already set up for your models. Depending on the specific framework you choose, agents include definitions for functions, tools, or skills that allow them to carry out tasks that often interact with 3rd party systems. This allows your applications to now trigger agents to carry out tasks instead of just responding to prompts.
Workflows
The next big thing to help your agents interact with complex business processes is to add workflows. OpenAI just released their agent builder called AgentKit, which I do not recommend. Why? Because it’s closed, and only works with OpenAI models. Complete non-starter in my book.
Instead, opt for well-established alternatives like n8n, which you can self-host for free, or Dify. What I’m seeing is that in many use cases, what you really need is automation, not even AI. You can use simple rules instead of burning LLM tokens for fixed deterministic processes, and just use LLMs to do things like summarization or classification as part of the workflow where useful.
As AI and agents become more integrated into your business and software systems, the myriad of exceptions and edge cases might be easier maintained as visual worklows than code or prompts.
Data
None of this is particularly useful, if all you’re doing is prompting the LLM directly. What makes AI useful for businesses is mostly to connect to your proprietary data on operations, human resources, sales, marketing, and so on. In practice, you’re faced with two options.
Either connect these existing systems via API to your agents and let the agents find data themselves, or index their data in a vector database for LLMs to search directly. For unstructured content like documents, you would just index them, potentially syncing your Google Drive, SharePoint, and Dropbox folders.
The long-term opportunity is to gradually move data into this new “brain, exploring more advanced data models like knowledge graphs to model your business data, while potentially getting rid of many expensive SaaS tools in the process (“The Death of SaaS”).
Summary
At the end of the day, you want optionality. Don’t paint yourself into a corner unnecessarily. The AI market has been incredibly volatile, from capability leaps between labs to corporate drama and geopolitical escalation. With the right strategic approach, you maintain flexibility and control over how AI is used in your business, allowing you to ride the ups and downs without losing sleep.
By building your own modular AI platform, you are leaning into a growing trend, and the early investments will compound over time and accelerate your trajectory toward AI sovereignty.
What choices and decisions have you made with your business, whether startup or enterprise? If you’re building your own AI OS, please share in the comments!