Local or cloud AI: how to choose for your needs and budget

In short

• There is no single right answer: there are three options (cloud API, subscription, open model on your own hardware) and each one wins on a different criterion.
• For raw quality on hard tasks, the large cloud models are still ahead. For privacy and offline use, local is unbeatable.
• API cost can be calculated: tokens × price × volume. Do the maths before deciding, with the AI API cost calculator.
• "Free" local AI is misleading: hardware, electricity, maintenance and security all have a real cost.
• For most small businesses, the best architecture is hybrid: sensitive data stays local, the rest goes to the cloud.
• An inference server exposed to the Internet without authentication is a security hole, not a detail.

Why this guide

Every week a client asks me a version of the same question: "Can we use ChatGPT with our customer data?", "Shouldn't we install our own AI instead?", "How much will it cost per month?".

The answers you find online are often clear-cut for the wrong reasons: cloud vendors tell you local is a hobby, open-model enthusiasts tell you the cloud is a prison. I have nothing to sell on either side: I build websites, apps and automations, and I use both depending on the case. This guide is the grid I actually apply before recommending a solution.

If you are looking for offline AI on a phone, I wrote a dedicated guide: AI on your phone, without the Internet. Here we are talking about professional use: business, development, automation.

The three real options

1. The cloud API

You send a request (text, image, document) to a provider, it returns an answer, and you pay per use, usually per token (pieces of words) in and out.

• Who it is for: developers and companies that build AI into a product, a website or an automation.
• Strengths: access to the most capable models, no machine to manage, instant scaling, pay-as-you-go.
• Weaknesses: your data leaves your infrastructure, the bill depends on volume (and can surprise you), dependence on a provider and on its model or price changes.

2. The subscription (consumer or business)

This is the online assistant with a monthly per-user plan: the chat interface, possibly a "team" or "enterprise" plan.

• Who it is for: people who work with AI every day (writing, analysis, code, research) without building it into software.
• Strengths: predictable price, nothing to develop, built-in tools (files, web search, projects).
• Weaknesses: it is not a building block you can plug into your automations; the terms on how your data is used vary a lot between free, individual and business plans. Read them.

3. The open model, local or on your own server

You download the weights of an openly published model and run it on a computer, a workstation with a graphics card or a server you control.

• Who it is for: highly sensitive data, offline use, high and steady volumes, technical teams that want full control.
• Strengths: data never leaves, no per-request bill, works offline, no dependence on a provider that changes its rules.
• Weaknesses: quality is usually below the best cloud models on complex tasks, hardware to buy, maintenance and security are on you.

My take. Pitting these three options against each other is a mistake. A well-organised agency or small business often uses two of them, sometimes all three, each for what it does best.

The six criteria that actually decide

1. Privacy and compliance

This is the criterion that most often settles the question, and the one most often handled badly.

• In Europe (GDPR): if you send personal data to an AI provider, it generally acts as a processor. You need a data processing agreement, you need to know where the data is processed, and transfers outside the European Union must be framed.
• In Morocco (Law 09-08): the law on the protection of individuals with regard to the processing of personal data is overseen by the CNDP. Transferring personal data abroad is regulated: it is only allowed to a country that ensures an adequate level of protection, and it goes through a procedure with the CNDP. In practice, sending Moroccan customers' data to an API hosted abroad is not a trivial act.
• In the Gulf: several countries have adopted their own data protection laws, with rules on localisation and transfer. Check the text that applies to your country and sector.

I am not a lawyer and this guide is not legal advice. But my practical rule is simple: personal or confidential data that does not need to leave does not leave. Either it is anonymised before sending, or it is processed locally.

Two reflexes before choosing a cloud provider:

1. Check, in the terms of the exact plan you use, whether your data is used to train models and how long it is kept.
2. Check the processing region and the existence of a data processing agreement.

To go further on what you should never paste into an AI: security, what never to hand over to an AI.

2. Answer quality

Let's be honest: on complex reasoning, hard code, long analysis or less-resourced languages, the best cloud models are still ahead of the models you can run on a reasonable machine.

But the useful question is not "which model is best?". It is "which model is good enough for my task?". Sorting emails, extracting fields from an invoice, summarising meeting notes, rewording a text: a mid-sized open model often does the job very well.

My method. I prepare 20 to 30 real examples of the task, with the expected answer. I run them through a local model and a cloud model. I compare. That half-day of testing is worth more than any published leaderboard.

A point of attention for our markets: quality in Arabic, and even more in Darija, varies enormously from one model to another. Test with your real texts, not with English examples.

3. Latency and user experience

• A cloud API adds a network round trip, but runs on very powerful hardware.
• A local model answers without a network, but its speed depends entirely on your hardware. On a laptop without a dedicated graphics card, a mid-sized model can be much slower than an API.

For a customer chatbot, perceived speed matters a lot. For an overnight batch of 5,000 documents, it matters little.

4. Real cost

This is the criterion that creates the most illusions, in both directions. It gets its own section below.

5. Offline use

A building site with no network, an office with an unstable connection, a field device, an environment isolated for security reasons: if the AI must work without the Internet, the question is settled, it is local.

6. Maintenance and skills

An API is a key and a few lines of code. An inference server is a machine to monitor, updates, models to replace, network security. If nobody on the team has the time or skills, "free" local AI will cost a lot in incidents.

Decision table by use case

Hardware for running a model locally

The rule of thumb that avoids bad surprises

The main constraint is memory: the graphics card's video memory (VRAM), or unified memory on computers whose processor and graphics chip share it.

A model is described by its number of parameters (in billions, written B). Each parameter takes a certain number of bits depending on quantisation (the precision with which the weights are stored):

• 16-bit: about 2 bytes per parameter;
• 8-bit: about 1 byte per parameter;
• 4-bit: about 0.5 byte per parameter.

Which gives a starting estimate:

weights memory (GB) ≈ parameters (in billions) × bits / 8

You then add a margin for the context (the current conversation and documents) and for the software itself. The longer the context, the bigger that margin.

Cautious orders of magnitude

These are estimates to get your bearings, not guarantees: real speed and memory use depend on the software, the context length and the model's architecture.

Two important nuances:

• "Mixture of experts" models (MoE) only activate part of their parameters for each generated word. They are faster than a model of the same total size, but you still need to load all the weights into memory.
• Going too low on quantisation (3-bit, 2-bit) saves memory but degrades quality. 4-bit is the most common compromise for local use.

A concrete example. In 2025 OpenAI published open models, the gpt-oss family, under the Apache 2.0 licence. The smallest (about 20 billion parameters) is presented as able to run with about 16 GB of memory, thanks to native 4-bit quantisation on part of the model. It is a good illustration: a well-optimised "20B" fits on a decent developer machine.

Tools to run a model yourself

To get started, my advice is simple: LM Studio if you want to test without a terminal, Ollama if you want to plug the model into an application or an automation.

An Ollama example, which shows how quick it is to start:

ollama pull <model-name>
ollama run <model-name> "Summarise this text in three points: ..."

Watch out for a recent change: Ollama also offers models that run in its own cloud. Its documentation states that in local mode the company does not see your prompts, and that you can force strictly local operation with the OLLAMA_NO_CLOUD=1 variable. If privacy is your reason for going local, turn it on and check which model you are actually using.

The main open model families

The landscape moves every month, so I will not give you a ranking that would be out of date in three months. These are the families I see most often with clients and in tools:

• Llama (Meta): very widespread, large ecosystem, Meta's own licence.
• Gemma (Google): small and mid-sized models, efficient locally.
• Qwen (Alibaba): wide range of sizes, good multilingual level, several versions under the Apache 2.0 licence.
• Mistral (French company): several open models, but licences differ from model to model.
• DeepSeek: very capable models, often large.
• gpt-oss (OpenAI): open models under Apache 2.0.

The "open" licence trap

"Open weight" does not mean "open source", let alone "do whatever you want with it". This is the trap I see most often with companies moving to local AI.

• Apache 2.0 or MIT: permissive licences, commercial use allowed, with a few attribution obligations.
• Custom licences: some families impose their own terms. The Llama licence, for example, includes an acceptable use policy and a threshold for very large services (more than 700 million monthly active users). The Gemma terms of use also have their own prohibited-use rules.
• Per-model licences: with some publishers, two models from the same family do not have the same licence, and some forbid commercial use without a separate agreement.

The rule: read the licence of the exact model you deploy, not the family's in general, and keep a record of that check in the project documentation.

Calculating cost without fooling yourself

The cost of an API

The formula is always the same:

monthly cost ≈ (input tokens × input price + output tokens × output price) × requests per month

Three remarks that change everything:

1. Input often costs more than expected. If you send a long document or the whole conversation history with every request, input tokens dominate the bill.
2. Output is usually more expensive per token than input. Asking for short, structured answers is a direct saving.
3. The choice of model changes the bill by a very large factor. A small model well chosen for a simple task costs a fraction of a large one.

I deliberately put no prices in this guide: they change too often. I built a tool to do this calculation with dated, editable prices: the AI API cost calculator. Enter your volume, the average size of your requests and answers, and compare several models.

The real cost of local

Local is not free. This is what I include in the calculation:

• Hardware: the machine or graphics card, depreciated over its useful life (often two to four years in this fast-moving field).
• Electricity: a graphics card under load uses a lot; so does a server running 24/7.
• Hosting, if you rent a server with a graphics card: often the heaviest line, and it runs even when nobody is using the AI.
• Human time: installation, updates, monitoring, switching models, troubleshooting. This is the number one hidden cost.
• Security: firewall, authentication, backups, logging.

The tipping point

The general logic is:

• Low or irregular volume: the API is almost always cheaper. You only pay for what you use.
• High, steady and predictable volume, on a task an open model handles well: local can become profitable, because the hardware pays for itself.
• Data that cannot leave: the financial calculation comes second, local is the answer.

To know whether the automation itself is worth the investment, regardless of the technology, use the automation ROI calculator.

The hybrid setup I use in practice

In most small-business projects, I end up proposing a two-tier architecture:

1. A local model, or one on a server you control, receives the sensitive data first. It does the work that touches personal data: extraction, anonymisation, classification, internal summary.
2. A cloud model only receives what can leave: anonymised text, public content, writing or translation tasks where quality matters a lot.

A real-world flow for a firm handling client files:

Client file (PDF)
  → local model: extract fields + replace names, numbers, addresses with codes
  → cloud model: write a clear summary from the anonymised text
  → local model: put the real values back into the summary
  → human review

The automation tool in the middle (n8n, for example) orchestrates the steps. I cover it in detail in the guide automating with n8n and AI.

This setup has three advantages: identifying data never leaves, you get cloud quality where it matters, and you can change cloud provider without touching the sensitive part.

Securing an inference server

This is the point tutorials forget, and the one that worries me most when I audit an installation.

Ollama, for example, listens by default only on the machine itself (127.0.0.1, port 11434). To make it reachable from the network, its documentation explains how to change the address with OLLAMA_HOST. The problem: the API has no built-in authentication. Security researchers have repeatedly found thousands of servers of this kind open on the Internet, usable by anyone.

What I apply every time:

• Never expose the inference server's port directly to the Internet.
• Remote access only through a VPN or a private tunnel, or behind a reverse proxy with authentication.
• A firewall that blocks the port, and beware of Docker: publishing a container port can bypass some firewall rules. Bind the port to 127.0.0.1 in the configuration.
• Updates to the inference software: these are young tools, and flaws are found and fixed regularly.
• Access logs, to spot abnormal use (someone consuming your graphics card instead of you, for example).

The traps I see most often

My decision checklist

Before recommending a solution, I answer these questions in writing:

1. What data goes through the AI? Is there personal, medical or financial data, or proprietary code?
2. Is this data allowed to leave our infrastructure, and under what conditions (GDPR, Law 09-08, client contract)?
3. What quality is good enough? Have I tested on 20 to 30 real examples, in the right languages?
4. What volume per month, and is it steady or in peaks?
5. How much does the API cost at that volume? (calculated with dated prices)
6. How much does local cost, hardware, electricity and human time included?
7. Must the AI work offline?
8. Who maintains the solution, and do they have the time and skills?
9. Does the licence of the open model I am considering allow my use?
10. If I have a server, is it unreachable from the Internet without authentication?
11. What happens if the provider or the server goes down?

Once these eleven answers are written down, the decision is almost always obvious.

FAQ

Is a local model really private?

Yes, if the model really runs on your machine and the software sends nothing elsewhere. Check two things: that you are not using a cloud version offered by the same tool by mistake, and that your server is not reachable from the Internet. Privacy depends on the whole installation, not just the model.

Can I use an AI API with Moroccan customers' data?

It is possible, but not without precautions. Law 09-08 regulates the transfer of personal data abroad and the CNDP is the supervisory authority. The simplest and safest approach is often to anonymise the data before sending it, or to process the sensitive part locally. For a significant project, have the setup validated by a lawyer.

What computer should I buy to start locally?

To test models of 7 to 9 billion parameters, a machine with an 8 GB graphics card, or a computer with at least 16 GB of unified memory, is a good starting point. For bigger models, the amount of memory is the main criterion, ahead of raw power. Test first on your current machine with a small model before buying anything.

Are open models as good as cloud models?

On simple or medium tasks (summarising, classifying, extracting, rewording), good open models are often enough. On complex reasoning, hard code or some languages, the best cloud models usually keep an edge. The only reliable answer is a test on your own examples.

How can I estimate my API bill before starting?

Estimate the number of requests per month, the average size of the text sent and of the answer, then multiply by the chosen model's prices. The AI API cost calculator does this for you and compares several models. Add a margin, because real usage often exceeds estimates.

Is a subscription enough for a small business?

For individual use (writing, analysis, research), yes, it is often the best value. As soon as you want to build AI into a website, an app or an automated process, you need an API or a local model. Also check that the plan you choose does not use your data for training.

What if my API provider changes its prices or retires a model?

Design your application so you can switch models easily: a single layer of code that calls the AI, prompts stored separately, and a test set to quickly check that a new model gives acceptable results. It is also an argument for the hybrid architecture.

---

Unsure between local, cloud or hybrid for your business? I can audit your use cases, cost the options and set up a secure solution. See the AI and automation service or contact me to talk it through.