Early in most AI projects someone asks whether you should use a hosted model through an API or run your own. The question often turns into a debate about technology preferences, when really it's a product and business decision.
At WiZR, building Frontline for enterprise clients, this came up more than once, for different parts of the system. The answer wasn't the same every time, and I think that's normal. Different jobs inside one product can have different right answers.
The basic options
| Option | What it means | Good for | Watch out for |
|---|---|---|---|
| Hosted API | Call a provider's model over the internet | Fast start, strong models, no infrastructure | Per-request cost, data leaving your systems, provider changes |
| Hosted open-weight | Run an open model on a cloud provider's managed service | More control, often lower cost at scale | More setup, still some dependency |
| Self-hosted open-weight | Run the model on your own infrastructure | Full control of data and deployment | Real engineering and operations effort |
| Fine-tuned on any of these | Adjust a model for your task | Specific behaviour or format needs | Training data, maintenance, evaluation |
Start with quality
The first question is whether a given option can do the job well enough. That depends entirely on the task. For some tasks, a smaller open model is perfectly good. For others, only the strongest available models are reliable.
Test the options on your own evaluation set before arguing about anything else. If an option can't meet the quality bar, its other advantages don't matter. The AI evaluation checklist covers what to put in that set.
Then look at cost at your scale
Hosted APIs are usually cheapest at low volume, because you pay only for what you use. At high volume, running your own model can become cheaper, but only once you count the infrastructure and the people needed to run it.
I'd model both at a few different usage levels. Today's volume, what you expect in a year, and an optimistic case. I wrote about working out LLM cost per user separately, and the same approach applies here. The answer also feeds straight into how you price the feature.
Data control often decides it
For enterprise customers, where data goes can matter more than cost. Some customers won't accept their data being sent to an external provider. Others are fine with it under the right agreements.
Ask your customers, or your sales team, early. If a big segment needs data to stay inside a certain environment, that might settle the decision before you even compare costs.
Be honest about team capacity
Running your own model means someone has to deploy it, monitor it, scale it, update it and fix it when it breaks. That's real ongoing work. If your team is small and stretched, a hosted API might be the better choice even if it costs more per request.
Saving money on inference while slowing down every product improvement is usually a bad trade early on.
Plan for switching
Whatever you choose, avoid locking the whole product to one provider's quirks. Keep prompts and evaluation sets separate from provider-specific code where you can, so testing an alternative isn't a huge project.
Models improve and prices change quickly. The best choice today might not be the best in a year, and being able to switch gives you options.
A rough way to decide
- Which options meet the quality bar on your evaluation set?
- Which of those meet your customers' data requirements?
- Which can your team realistically run?
- Among the rest, which costs least at your expected scale?
Most teams I know start with a hosted API to learn quickly, then move specific high-volume or sensitive parts to other options once they understand their usage. That's a reasonable path, as long as the move is planned rather than forced by a surprise bill.