Which AI Setup Should You Use?

Think before you act

AI has been one of the most talked-about topics since OpenAI released ChatGPT in November 2022. Nearly four years later, a lot has changed. Models have become dramatically more capable, and while most people still see AI as an advanced chatbot, many now work with tools such as Claude Code, Claude Desktop, GitHub Copilot, OpenAI Codex and Cursor. Managers use it to capture meeting notes, assistants to create presentations, business analysts to analyse data and developers to build software. Alongside the closed models, there’s a growing range of open-weight models such as Qwen, Gemma and Mistral, which you can run yourself, for instance by combining Ollama and OpenCode.

With so many options, one question keeps coming up:

“When should I use what?”

The options: from convenience to control

Every option sits somewhere between convenience and control. The more someone else runs for you, the less effort it takes, and the less say you have over your data. I group the options into three options.

Note: privacy is just an assumption from me. You might disagree

1: Managed services

Someone else runs both the model and the infrastructure. You just use it.

Free chat (e.g. ChatGPT)

The easiest way in, with no cost and no commitment. It’s fine for quick, occasional questions.
But features are capped, and heavy use often drops you to a weaker model. Your conversations may be used for training unless you opt out, so don’t expect any confidentiality.

Paid subscriptions (e.g. ChatGPT Plus, Claude Pro)

Flagship models, more features and a predictable monthly cost. Training can usually be switched off, and business tiers add stronger contractual terms.
On the downside, session limits can stop you mid-task, and it isn’t built for automation. The provider’s terms decide how long your data is stored and whether it’s used for training, and you can’t verify either.

API access

Pay per use and build it into anything. You can go direct to the model provider (Anthropic, OpenAI, Google), through a cloud platform (Amazon Bedrock, Azure AI Foundry, Google Vertex AI), or through an aggregator such as OpenRouter, which gives you one key for many models. You get any model, no session limits, and API traffic is usually excluded from training by default. Model providers and cloud platforms alike offer business terms and data processing agreements, and a cloud platform mainly saves you admin: one bill and access controls you already manage.
On the other hand, costs grow with usage, and you need developer skills to set it up. Many providers keep API data for a while, for example for abuse monitoring, though some offer zero data retention. With aggregators, privacy depends on which provider handles each request, so restrict the routing.

2: Rented compute

You run the model yourself, usually an open-weight one, on hardware someone else owns.

GPU rental marketplaces (e.g. Vast.ai, RunPod)

Spare GPU capacity, often far cheaper than the big cloud providers. It’s good for experiments and batch jobs, with full control over the model and its settings.
But hosts are often individuals or small operators, so uptime and security vary, and the host could, in principle, access your data.

Self-managed cloud hosting

GPU instances or servers from an established cloud provider, set up by you. You get full control without buying hardware, it scales on demand, and there’s an accountable provider behind it.
But you need real expertise in networking, access control and patching. A misconfigured setup leaks data as easily as a bad SaaS tool, and costs add up with sustained use.

3: Your own hardware

Local network hosting

Models on your organisation’s own servers. Data never leaves your organisation, and you control security and access yourself.
The hardware costs money up front and needs maintaining, though. The models you can run may lag behind the cloud, and privacy is only as good as your own security.

Local computer / on-device

The model runs on your own machine. It offers maximum privacy, works offline and costs nothing beyond your hardware.
But your hardware limits you to smaller, slower models, and your device itself still needs to be secure (disk encryption, physical access).

Where does it run, who controls it, and whom do you trust?

Whichever option you pick, check three things.

Where is your data processed? Many providers let you choose a region, for example the EU. This goes for subscriptions, APIs, cloud platforms and self-managed hosting, and it makes GDPR compliance a lot easier. The downsides are fewer providers, varying prices, and the newest models aren’t always available everywhere.

Who controls the provider? Where the data is processed isn’t the same as who controls it. A service labelled “European” may run on European servers while the company behind it is based elsewhere. US companies, for instance, can be forced under laws such as the CLOUD Act to hand over data, even if it’s stored in the EU. That applies to OpenAI and Anthropic just as much as to AWS, Azure and Google Cloud. So check who owns and operates the service, not just where the servers are.

Whose promise are you relying on? Subscriptions and APIs from US providers can be approved, but usually not for every kind of data. Internal code might be fine, while customer data or anything under an NDA might not. So don’t just ask “is AI allowed?”, but also “is this tool approved for this data?” And even when a tool is approved, switching off training gets you a promise, not a guarantee. Providers have good reasons to keep it, but you can’t check it, and once data has left your machine, it’s gone.

Rules of thumb

Knowing all that, which option you actually use comes down to cost, your technical skills and, at work, your company’s AI and data policy. Within those limits, I look at the data first and the workload second. These are my defaults:

• Public information, one-off task → free chat. Your subscription would work too, but there’s no point spending its limits on this.

• Everyday work with data the tool is approved for → a company subscription with training switched off.

• Recurring, automated or part of a product → an API. Going direct to the model provider or through your cloud platform is a practical choice.

• Highly sensitive data → check first whether you’re allowed to use your own network or a regional provider at all. If so, run an open-weight model on your network, or use a provider that is regional in ownership, not just in server location.

• Data your tools aren’t approved for, or you’re unsure → a local model, or describe the structure of the data instead of sharing it.

• Experimenting with open-weight models → a GPU marketplace, but only with public data or data you’re explicitly allowed to use there.

And if you’re unsure how sensitive something is, treat it as highly sensitive.

Putting it into practice

Imagine you have access to the free version of ChatGPT, a Claude Code subscription, a Gemini API key from your employer and Qwen running locally on your laptop. As the company’s AI expert, your supervisor gives you five tasks:

1. Create content for a beginners’ Linux training.

2. Review a few lines of code from your newest application.

3. Build a tool that summarises open tasks from a ticket system every day.

4. Analyse a dataset containing user data. Source unknown.

5. Give your opinion on some financial data from a client.

1. Linux training content: free chat.
All you need is a set of basic commands with explanations, and there’s nothing to protect. Only the Gemini API costs money per call, the other three cost you nothing extra, so it comes down to speed and limits. Qwen on a laptop is noticeably slower. Claude would do a fine job, but every prompt uses up session limits you’d rather keep for coding. Free ChatGPT is fast and more than good enough for this. You’ll still have to build the slides yourself, though, which other tools could do for you.

2. Code review: company subscription.
This is fine as long as three conditions hold: the code isn’t particularly sensitive, your company has approved the tool for code, and training is switched off. Bear in mind that the approval covers code, not everything. The same subscription may not be approved for customer data. And the provider’s promise not to train on your code remains a promise. Providers have a strong incentive to honour it, but your code has still left your machine.

3. Ticket summary tool: subscription to build it, API to run it.
This task has two parts. To build the tool, I use Claude Code with a strong model, usually the latest Opus with the effort level set to xhigh. Whenever Claude calls on a skill, that skill applies its own preset model and effort level. I’ve written my own skills for this, and I’d recommend you do the same: /skill-creator lets Claude help you build them. I also use a few extra tools to improve results and save tokens, which I cover in a separate article. Running the tool is a different matter. New tickets arrive every day and need summarising automatically, which is a job for the API, not a subscription.

4. Dataset with user data: share the question, not the data.
The dataset is too large to check manually, so you don’t really know what’s in it. That’s reason enough to be cautious and treat it as highly sensitive. Running Qwen locally would keep the data on your machine, but without strong hardware it’s far too slow for a dataset this size. The Gemini API key doesn’t help either: whether it’s approved depends on the kind of data, and that’s what you don’t know. The good news is that no model needs to see the data. You only want to know how many users you have. So describe what you want, share only the relevant column names, load the dataset into a database and run the query the AI gives you.

5. Client financial data: local model.
If the client hasn’t explicitly allowed AI, or simply hasn’t said anything about it, a local model is your only option, as everything stays on your own computer. There are good open-weight models out there, but don’t expect miracles. Without powerful hardware, which you rarely find in a laptop, things will be considerably slower. A tool such as llmfit helps you work out which model suits your hardware.

llmfit showing suitable models

A note on model choice

Choosing the setup is half the decision. Choosing the model is the other half.

For serious coding, I use Opus rather than Sonnet. Sonnet too often gave me code I had to throw away, and every restart burns tokens, so the cheaper model ended up costing more. I also skip Fable for day-to-day coding. I haven’t seen a real advantage over Opus, and it uses tokens noticeably faster.

The same goes for your own skills. Simple tasks such as formatting run fine on a lighter model, while architecture decisions or complex refactoring earn a stronger one.

That’s what works for my tasks. In my experience. Yours may differ, so test it yourself.

Takeaways

• Decide by data first, workload second, price third.

• Approved tools mean trusted, not verified.

• Where data is processed and who controls the provider are two different questions.

• Subscriptions are for people, APIs are for processes.

• When in doubt, keep the data local, or keep it out.

• The cheapest model per token isn’t always the cheapest per result.

Before asking which AI is best, ask what you’re about to send it.

Choosing the setup and the model is only part of the story. The tools around them can save tokens and improve results, and that’s what my next article is about.

This Blog Article is originally published on Medium.com https://medium.com/datamindedbe/which-ai-setup-should-you-use-b475ad286eb5