Local models vs cloud APIs: the real cost comparison

Sep 15, 2026

Maho Team

The argument for local models tends to start with a vibe. Cloud is expensive, local is free, my Mac is sitting there anyway. The argument for cloud APIs starts with a different vibe. Local is slow, the model is small, the real models live in datacenters. Both vibes are partly true. Neither is a number.

This post is a numbers-first comparison. We pick a workload, we work out what local actually costs in electricity, what cloud actually costs in tokens, and where the two curves cross. The conclusion is more nuanced than either vibe predicts. All prices below are public list prices as of 2026-Q2 and are approximate. Rates change. Use them as a frame, not as a quote.

Local vs cloud cost comparison

Pick a benchmark workload

Comparing dollars to watts requires a fixed workload, otherwise we are just trading rhetorical wins. The workload we will use:

1 million input tokens and 200 thousand output tokens of usage per month.
A mix of short interactive prompts (200 to 800 input tokens) and longer ones (3,000 to 8,000 input tokens).
A model class around 7 to 13 billion parameters for the local case.
A frontier-tier model for the cloud case.

This is a heavy daily user who lives inside the AI panel: dozens of prompts a day, occasional long-context summarization, regular tool calls. A casual user does ten percent of this. A team scales it ten times in the other direction. The crossover math we work out below scales linearly, so you can move the workload up or down to fit your real usage.

We are also going to ignore one-time costs on both sides. Hardware depreciation and account setup time are real, but they amortize across years and the comparison gets messy fast. We will note the limit, not bake it in.

Local: hardware assumptions, kWh per 1M tokens

Assume an M3 Pro MacBook with sufficient unified memory to run an 8 to 13 billion parameter model in 4-bit quantization. Numbers:

Idle draw: roughly 5 watts.
Heavy LLM inference: roughly 25 watts under sustained generation.
Tokens per second for an 8B model on this machine: roughly 40 to 60 tokens per second of output.

The relevant figure is energy per token. Worked out:

25 watts for 50 tokens per second is 0.5 joules per token.
A million output tokens at that rate is 500,000 joules, which is 0.139 kilowatt-hours.
At a US average residential electricity rate of roughly $0.16 per kWh in 2026, that is about 2.2 cents per million output tokens.

Input tokens are cheaper because prefill is faster than decode. The order of magnitude is similar, often less than half the decode cost per token. To keep the math simple, we will treat the whole workload as the decode rate and call it 5 to 6 cents per month for our benchmark workload, end to end, in pure electricity. For practical purposes, the marginal cost of running a local model on a laptop you already own is rounding error against most other things in your monthly bill.

Two caveats:

The laptop is doing other things. The 25 watts is incremental load on top of the baseline you would draw anyway, so the real “extra” cost is even smaller. If the laptop would otherwise be sleeping, the cost is the full delta.
Heat and fans. Sustained inference on a laptop heats the chassis and spins fans. Not a dollar cost, but a comfort cost on hot days.

For the setup walkthrough, see Ollama with Maho, step by step.

Cloud: API price per 1M tokens, latency floor

Public list prices for frontier models as of 2026-Q2 (approximate, US dollars, per 1M tokens):

| Provider / model | Input | Output | |---|---|---| | OpenAI GPT-4o | ~$2.50 | ~$10.00 | | Anthropic Claude Sonnet 4 | ~$3.00 | ~$15.00 | | Google Gemini 1.5 Pro | ~$1.25 | ~$5.00 |

Cheaper tiers exist (GPT-4o mini, Claude Haiku, Gemini Flash) at roughly an order of magnitude less. The “frontier” pricing above is what you actually pay when you want a model that is meaningfully better than what runs on your laptop.

For our benchmark workload (1M input, 200K output) on Claude Sonnet 4: $3.00 + $3.00 = roughly $6 per month. On GPT-4o: $2.50 + $2.00 = roughly $4.50 per month. On Gemini 1.5 Pro: $1.25 + $1.00 = roughly $2.25 per month.

Latency is the other axis. Cloud APIs from a US west coast network see a first-token latency of around 300 to 800 milliseconds for frontier models. Local models on the same machine you are typing on can return a first token in under 100 milliseconds. That gap matters more for short interactive prompts than long generations: a 50-millisecond round trip is invisible inside a 30-second response, but it is the whole experience inside a 300-millisecond one.

Energy and dollars per million tokens

Crossover point

For pure dollar cost, frontier cloud already loses to local at modest usage. Six dollars a month for Claude Sonnet 4 versus six cents in electricity. The cloud cost is 100 times higher. Crossover is not at our benchmark workload, it is far below it.

But that is only true if you accept the model swap. The cloud number is for a frontier model. The local number is for an 8B model. They are not the same product. The honest comparison is “which one solves your problem”, not “which one costs less per token in isolation”.

The interesting crossovers are not in dollars. They are in capability:

For chat, summarization, and most code tasks under 8K context, an 8B local model is good enough. Crossover in capability is at “tasks that fit in the smaller model’s bandwidth”. Below the line: local wins on every axis. Above the line: you need cloud.
For long-context work (legal documents, large codebases, research synthesis at 100K+ tokens), local models on a laptop start to choke on memory and on quality. Crossover in capability shifts toward the cloud.
For agentic chains that make many sequential calls, the per-call latency advantage of local compounds, and the per-call dollar cost of cloud compounds in the other direction. A 30-step agent at 500 ms per cloud call is 15 seconds of pure latency. The same agent on a local model can run in a fraction of that.

For our benchmark workload, the math says: if you can tolerate the smaller model, local is dramatically cheaper and faster. If your workload demands frontier capability for a non-trivial fraction of calls, cloud earns its keep.

Where the comparison breaks

The clean numbers above hide a few things you will run into in real use.

Long context. Frontier cloud models routinely accept 200K to 1M context windows. A laptop running an 8B model with a 32K window will try, and fail, and try again on big documents. If your work is long-context heavy, cloud is not optional, it is the only path.

Agentic chains with many tool calls. A long agent run can burn through tokens fast. A 50-step research agent on Claude Sonnet 4 with 4K input plus 1K output per step is roughly 200K input and 50K output tokens, or about $1.35 per run. Run that agent ten times a day and the cloud bill matters. The same chain on a local model is free. But if any step in the chain needs frontier reasoning to succeed, the cheap chain breaks and the expensive chain gets the work done.

Quality cliffs. Smaller local models do well at most things and badly at a small set of things. The set is shrinking every quarter, but it is real. If your work is in that set today, the cost comparison is moot.

Battery. Local inference on a laptop costs battery. On the desk, that is irrelevant. On a flight, it is a 30 percent hit on hours of use. Cloud calls are nearly free in battery terms.

For the architecture that lets Maho route between the two without you rewriting your prompts, see the BYOK architecture deep dive.

Where local breaks and where cloud earns its keep

Recommendation

The honest recommendation is “both, routed”. Most prompts a heavy user sends are short, structured, and well within the capability of an 8 to 13 billion parameter local model. The remainder are long, hard, or precision-critical, and those want a frontier cloud model. A browser that can route between them per request is the configuration that minimizes cost without capping ceiling.

Concretely, for a heavy user with the workload above:

80 percent of prompts on local: roughly 5 cents in electricity per month.
20 percent of prompts on a frontier cloud model: roughly $1 to $3 per month, depending on provider.
Total: under $4 per month for behavior that, on cloud-only, would cost $5 to $10.

That is not a dramatic dollar saving in absolute terms. The real win is the latency floor on the 80 percent, and the freedom to keep most prompts on a machine you control.

Get notified

We are bringing the local-plus-cloud router to a small group of testers first. If you want to try the routing logic on your own workload, with your own keys and your own local models, get notified. We will reach out as slots open and the router stabilizes.