Every “open versus closed AI” argument I have sat through has quietly been two arguments at once. One is about which model you use: open-weight or proprietary. The other is about where it runs: your own network or somebody’s cloud. They are independent. You can run an open model on a rented cloud GPU, and you can, with the right enterprise agreement, run a proprietary model inside your own data center. I wrote the hosting decision up in On-Premise AI vs Cloud AI; this chapter is the other one, the model decision, and it is chapter 7 of the Open Source AI Hub.
The two axes, made explicit
Put the two questions together and you get four positions. An open model on your own hardware gives you full control of the weights and the data, and this is where most of my private AI clients end up. An open model on cloud hardware gives you control of the weights and freedom from per-token pricing, but your data still travels to the provider’s machines. A proprietary model through an API gives you the least control and, today, the most capability, and it is the right answer more often than my line of work would suggest. A proprietary model on your own premises is rare and expensive, and mostly the province of very large enterprises. Most private AI deployments I build are the first position, with the third held in reserve for the hardest requests.
The case for open models
Privacy and data control
An open model can run where the data already is, so nothing has to leave. That is not a policy promise or a contract clause; it is the architecture. For code, customer records, contracts and anything under a regulator’s eye, this is usually the deciding argument, and it is the one that brought most of my private AI clients to the door.
Cost shape
Open models change the cost from a per-token bill that scales with usage to a capital cost for hardware plus operations. At low, spiky volume the API is cheaper. At steady, high volume the math flips, often sooner than people expect, and a single well-used GPU server can replace an API bill that was growing every month. The point is not that open is cheaper; it is that open lets you choose the cost shape that fits your usage.
No lock-in and no rug-pulls
The weights you have downloaded are yours to run forever, at a pinned version, regardless of what the maker does next. API models get deprecated on the provider’s schedule, change behavior between versions, and change price. With an open model the upgrade happens when you decide, after you have tested it, and never before.
Auditability and reproducibility
You can inspect an open model, test it exhaustively, pin its exact revision, and reproduce a result a year later. For anything that will be audited, that alone can justify the choice. Fully open models such as OLMo go further and let you see what the model learned from, as chapter 2 explains.
Sovereignty
For an Indian company, or a European one, an open model means the capability sits inside the country and inside the organization, not behind a foreign API that a policy change or a sanction could switch off. I made the wider argument in Who Cannot Use AI, and Why?: open models are what keep AI available to people and countries that are not at the front of the queue.
Customization
You can fine-tune an open model on your own material, distill it into something smaller, quantize it to fit your hardware, and change how it is served. The proprietary equivalents exist, but on the vendor’s terms and at the vendor’s price.
When not to: the honest trade-offs
The capability gap
On the hardest tasks, the frontier proprietary models are ahead, and every few months the gap narrows and then reopens with the next release. For everyday workloads, summarization, extraction, classification, retrieval-augmented answers and most coding, the best open models are good enough that the difference rarely matters. For research-grade reasoning and the most demanding agentic work, it still can. Be honest about which of these your workload is.
The operations burden
Someone has to buy or rent the GPUs, keep the drivers and runtimes current, watch the memory, plan the upgrades and answer the pager. An API hides all of that behind a bill. If you have no one to run the system, the bill is the cheaper option.
Support and accountability
There is no vendor to call when an open model does something strange at two in the morning. You or a partner are the support organization. That is a real cost, and it is exactly the work I do, so I will not pretend it is free.
License diligence
Open-weight is not open source, and the restricted licenses carry obligations that you must actually read. Chapter 3 is the hour that saves you the year.
The safety and tuning work you inherit
Proprietary models arrive with guardrails, evaluations and red-teaming already done by the vendor. With an open model you inherit that work: system prompts, content filters, your own evaluation suite, and the testing to prove the thing behaves. It is not optional, and it is not small.
Talent
Running models well is a skill, and it is still scarce. If nobody on the team has done it, budget for learning it or for someone who has.
A decision rule you can defend
Five questions, answered honestly for one workload at a time, not for the company as a whole.
- Does the data have to stay inside our control, by law, by contract or by prudence?
- Is the volume steady enough that owning capacity beats renting it by the token?
- Is the task inside what a 30B to 70B class open model does well today?
- Do we have, or can we hire or contract, someone to run it?
- Can we tolerate being a step behind the frontier for this workload?
Two or more yes answers, and you should start with an open model and prove it on your own evaluation set before you decide anything else. Fewer than two, and an API is probably the right call for that workload, whatever your feelings about openness. Most organizations have some of each, and the mature answer is a written decision per workload, revisited twice a year as the models move.
Where I land
Open models by default for anything that touches client data or client code, run inside the client’s network, which is the private AI work I do. Proprietary APIs for exploration, for the genuinely frontier-dependent tasks, and for teams that have no one to run a system. And a one-page decision record per workload, so that when a new model lands, the question is “does this change the answer?” rather than “what was the question?”. Practice is the product of theory: the definitions in chapter 2 and the licenses in chapter 3 are what let you make this decision with your eyes open, and the architecture checklist is what you run once you have made it.
Next steps
If you have read the hub in order, you are done with the theory; the FAQ collects the short answers, and the hub page links the operational guides for running all of this. For the hosting half of the decision, read On-Premise AI vs Cloud AI. And if you would like help making the decision for your own workloads, tell me about your situation.