Open or Closed: Choosing the Model Underneath Your Product

Early in a build, a question arrives that feels technical but is really strategic: do we call a closed model through an API, or run an open-weight model ourselves? Teams often answer with whichever is familiar, or whichever the loudest vendor was selling that month. The better answer starts from a quieter question, which is what the business genuinely cannot afford to lose. Control, cost, speed, and compliance all pull in different directions, and the right call depends on which of them is load-bearing for you.

What open and closed actually mean

The labels get thrown around loosely, so it helps to be precise. A closed model is one you can only reach through a vendor’s API. You send text in, you get text back, and the weights that make up the model stay on the vendor’s servers where you never see them. An open-weight model is one whose trained parameters you can download and run on hardware you control, inside your own cloud account or data center.

Note the careful phrase open-weight rather than open-source. Most of the popular downloadable models release their weights and a licence, but not the training data or the full recipe that produced them. True open-source, where everything needed to reproduce the model is public, is rarer. For most product decisions, open-weight is the category that matters, because it is the one that lets you run the thing yourself.

Two different kinds of freedom

A closed API gives you the frontier with no infrastructure to manage. It is fast to start, always updated, and someone else’s problem to run at three in the morning. An open-weight model gives you a different freedom. It runs in your environment, your data never leaves your perimeter, and no vendor can change the terms, raise the price, or deprecate the model out from under you.

Closed models sell you capability. Open models sell you control. Decide which one your product cannot live without.

Neither freedom is strictly better. A startup chasing product-market fit usually needs capability and speed far more than it needs control. A regulated bank handling customer records usually needs control far more than it needs the last five percent of capability. The mistake is treating this as a question of taste rather than a question of what your specific situation demands.

How the two compare in practice

Beyond philosophy, the two paths differ on concrete dimensions that show up in your roadmap and your invoices.

• Capability: closed frontier models still tend to lead on the hardest reasoning and broadest tasks, though open models have closed much of the gap for everyday work.
• Data control: open and self-hosted keeps every token inside your network; closed means your prompts travel to a third party under their data policy.
• Cost shape: closed is pay-per-token with near-zero setup; open is pay-for-hardware, which is cheaper at high, steady volume and more expensive when traffic is light.
• Operational burden: closed hands running the model to the vendor; open puts serving, scaling, and uptime on your team.
• Customization: open weights let you fine-tune deeply and inspect behavior; closed limits you to the knobs the vendor exposes.
• Longevity: an open model you have downloaded cannot be taken away; a closed model can be retired or changed on the vendor’s schedule.

The cost question nobody frames correctly

Teams often compare a per-token API price against a per-hour GPU price and declare one cheaper. That comparison is almost always wrong, because it ignores volume and utilization. A closed API costs you nothing when idle and scales smoothly, which is ideal for spiky or low traffic. A self-hosted model costs the same whether you send it one request or ten thousand, which only pays off when you keep the hardware busy.

The honest number is total cost of ownership. For open models that includes GPUs or cloud instances, the engineers who keep them running, the time spent on optimization, and the opportunity cost of all of it. For closed models it includes the per-token bill at your real projected scale, plus the business risk of a price change you cannot control. Run both numbers at your expected volume before you decide, not at a toy volume that flatters whichever answer you already preferred.

Risk, compliance, and the questions that force the answer

For many organizations the decision is not made on cost or capability at all. It is made on where data is allowed to live and who is allowed to touch it. If a contract, a regulator, or a customer forbids sending data to an external service, that single constraint can settle the whole debate before the engineering starts.

Questions worth asking before you commit

• Residency: are you contractually or legally required to keep data inside a specific boundary?
• Volume: is your traffic high and steady enough that owned hardware beats per-token pricing?
• Team: do you have the people to run inference reliably, or would that burden slow everything else down?
• Capability: does your use case actually need the frontier, or is a smaller open model already good enough?
• Exit: if your chosen vendor doubled its price or shut down next quarter, how badly would it hurt?

Why the smartest answer is often both

The framing of open versus closed implies a permanent marriage, and that is the deepest mistake of all. The strongest teams treat the model as a replaceable component, not a foundation. They put a thin abstraction layer between their product and whichever model answers the call, so they can route simple requests to a cheap open model and hard ones to a closed frontier model, and swap either side as the market shifts.

This matters because the market shifts constantly. The leading model changes every few months, prices fall, and new open releases leapfrog last year’s closed champion. A team married to one provider has to rebuild to move. A team that kept the model layer loosely coupled just changes a configuration line.

Common pitfalls

• Choosing open for ideology, then drowning in operational work your team was never staffed for.
• Choosing closed for speed, then discovering a compliance rule that forbids it after you have shipped.
• Comparing prices at a tiny demo scale instead of your real projected volume.
• Hard-wiring one provider’s API everywhere, so switching later means a painful rewrite.
• Assuming open models cannot keep up, when a smaller tuned model often beats a giant one on your narrow task.

Frequently asked questions

Open source vs closed LLM: which is cheaper?

It depends entirely on volume. Closed APIs are cheaper at low or spiky traffic because you pay only for what you use. Self-hosted open models get cheaper per request as you keep the hardware busy, so they win at high, steady scale. Compare total cost of ownership at your real volume, not a demo volume.

Are open-weight models good enough for production?

For a great many tasks, yes. Open models have closed much of the gap on everyday reasoning, summarization, extraction, and classification, and a smaller model tuned on your task often beats a giant general one. The closed frontier still tends to lead on the hardest, broadest problems, so match the model to the difficulty of your work.

Does using a closed API mean the vendor trains on my data?

Not necessarily, but you must read the terms. Many providers now offer business tiers that contractually exclude your data from training and set retention limits. If no acceptable guarantee exists, or your data cannot leave your boundary at all, that points you toward self-hosting an open model.

How do I avoid vendor lock-in?

Put an abstraction layer between your application and the model so your code does not depend on one provider’s API shape. Keep prompts, routing, and evaluation in your own control, and test regularly against an alternative model. Then switching becomes a configuration change rather than a rebuild.

Key takeaways

• Open versus closed is a strategic bet about control, cost, compliance, and speed, not a matter of taste.
• Closed APIs sell capability and convenience; open weights sell control, data residency, and freedom from vendor terms.
• Compare total cost of ownership at your real volume, since each path wins in a different traffic pattern.
• Data residency and compliance often decide the question before engineering begins.
• Keep the model layer loosely coupled so you can mix both and switch as the market moves.

This decision is rarely permanent, and it should not be treated as one. Architect so the model is a part you can swap, not a foundation you are married to, and the open-versus-closed question loses most of its danger. The real moat was never the model. It was everything you built around it. If you want help designing that model layer and running the cost and compliance trade-offs for your own product, that is exactly the kind of applied work we do at Vesper Labs.