Choosing a Model

There is no single best AI model. The right choice depends on what you are doing, how much it matters, what you can spend, and who is allowed to see your data. A few simple distinctions make that choice much easier.

The main choices

Think of choosing a model like choosing transport. A bicycle, a car, a lorry and a plane are all "the best" for some trips and absurd for others. Models differ in the same way, along a handful of dimensions.

Hosted vs open-weight and local

• Hosted models run on a company's servers. You use them through a website, an app, or an API (a programming interface that lets software send requests and receive answers). The company keeps the model's internal numbers, called weights, private. As of November 2026, well-known hosted families include Anthropic's Claude, OpenAI's GPT and Google's Gemini.
• Open-weight models have their weights published so anyone can download them and run them on their own hardware, within the terms of the model's licence. As of November 2026, examples include Alibaba's Qwen, DeepSeek, Google's Gemma, OpenAI's gpt-oss and Meta's Llama families. "Open-weight" is not always the same as fully open source: the training data and code are often not released, and some licences add restrictions.
• Local means running a model on your own computer or server. This is only possible with open-weight models.

Hosted models generally offer the highest capability with no hardware to manage. Local models offer control, privacy and no per-use fees, but you need suitable hardware, and the largest open models need a lot of it.

Big vs small

A model's size is roughly measured in parameters, the adjustable numbers learned during training, usually counted in billions (written "B"). Bigger models tend to know more and handle harder tasks; smaller ones are faster and cheaper.

Worked example (local). Using the rough sizing behind this site's AI Index: an 8-billion-parameter model compressed to about 4.8 bits per parameter is a download of roughly 5 GB and runs on many recent laptops. A 70-billion-parameter model at the same 4.8 bits is a file of around 44 GB, which needs a high-end GPU setup or a computer with a large amount of shared memory. The Running AI Locally track explains these numbers.

Hosted providers also offer size tiers, often a large flagship plus cheaper, faster versions. As of November 2026, for example, Google lists Pro, Flash and Flash-Lite versions of Gemini, and OpenAI's GPT-6 family pairs its most capable model (Astra) with faster, lower-cost models named Sol, Terra and Luna.

Reasoning vs fast models

Some models are designed to "think" before answering: they generate a hidden or visible chain of intermediate working, then give a final answer. These are often called reasoning models. They tend to do better on maths, logic, planning and tricky coding, but are slower and use more tokens, which costs more. For a quick rewrite, a summary or a casual question, a fast model is usually just as good. Some apps let you switch between modes or choose a level of reasoning effort. (The lesson on reasoning models explains how this works.)

Multimodal

A multimodal model can handle more than text: images, audio, video or documents such as PDFs with charts. If you need to ask about a photo, read a scanned receipt, or talk by voice, check that the model accepts that kind of input. Some models can also produce images or speech; others only produce text.

Cost, speed, quality and privacy

These four pull against each other:

• Quality usually rises with model size and reasoning effort.
• Speed usually falls as size and reasoning effort rise.
• Cost for hosted APIs is charged per token, separately for input (what you send) and output (what the model writes). Consumer apps more often use a free tier plus monthly subscriptions with usage limits.
• Privacy depends on where the data goes and on the provider's terms (see the next lesson).

Worked example (hosted cost). Take an illustrative mid-range price of $2 per million input tokens and $10 per million output tokens (a made-up but realistic figure, not any specific model's price). Suppose you summarise 200 reports of about 3,000 tokens each, producing 500-token summaries. Input: 200 × 3,000 = 600,000 tokens, costing$1.20. Output: 200 × 500 = 100,000 tokens, costing $1.00. Total: about $2.20. A flagship model usually costs more per token for the same job, and a small model much less. For comparison, as of November 2026 Anthropic lists Claude Opus 5.5, one of its flagship models, at $4 per million input tokens and $20 per million output tokens, which would make the same job about $4.40. Prices change often, so always check the provider's current pricing page.

A simple decision table

Going deeper

Choose with your own evaluation

Public leaderboards are useful for shortlisting, but they measure general abilities on someone else's tasks. Two well-known approaches show the range: HELM (Liang et al., 2022) scores models on many scenarios and several metrics at once, including accuracy, calibration, robustness and efficiency, while Chatbot Arena (introduced by Zheng et al., 2023) ranks models from crowdsourced, head-to-head human votes. Neither knows your documents, your format rules or your users. Test questions can also leak into training data, inflating scores.

The professional approach is a small evaluation set: 20 to 50 real examples of your task, each with a pass/fail rule or a simple 1 to 5 rubric. Run each candidate model on the same set and record three numbers: pass rate, cost per task and latency (time to answer).

Count the cost of mistakes

A cheaper model is not cheaper if its errors are expensive to catch and fix. A useful estimate of the expected total cost per task is

$C = c_{\text{model}} + p_{\text{fail}} \times c_{\text{fix}}$

where $c_{\text{model}}$ is the price of running the model once, $p_{\text{fail}}$ the share of outputs that fail your checks, and $c_{\text{fix}}$ the cost of a person spotting and fixing a failure.

Worked example. Fixing a bad output takes about 6 minutes of an employee paid 30 dollars an hour, so $c_{\text{fix}} = \text{\textdollar}3$. A small model costs $\text{\textdollar}0.002$ per task and fails 12% of the time; a large model costs $\text{\textdollar}0.02$ and fails 4%.

• Small: $0.002 + 0.12 \times 3 = \text{\textdollar}0.362$ per task.
• Large: $0.02 + 0.04 \times 3 = \text{\textdollar}0.140$ per task.

The large model costs ten times more to run but is about 2.6 times cheaper overall. Over 10,000 tasks that is 3,620 dollars versus 1,400 dollars. When failures are cheap (a brainstorm you will edit anyway), the balance tips the other way.

Cascades: small first, escalate when needed

If you can automatically detect many failures (a format check, a missing field, a failed test), you can run the small model on everything and send only the failures to the large one. Assuming the check catches every small-model failure (optimistic, but a useful first estimate), the expected cost is

$C = c_s + (1-p_s)\,c_L + (1-p_s)(1-p_L)\,c_{\text{fix}}$

where $p_s$ and $p_L$ are the small and large models' pass rates. With the numbers above: $0.002 + 0.12 \times 0.02 + 0.12 \times 0.04 \times 3 = 0.002 + 0.0024 + 0.0144 = \text{\textdollar}0.0188$ per task, far cheaper than either model alone. Many production systems route requests this way.

Will an open-weight model run on my machine?

This site's AI Index uses simple rules you can apply yourself (they are estimates, not guarantees):

• Memory needed ≈ file size (parameters × bits per parameter ÷ 8, plus about 5%) + KV cache + about 1.5 GB of runtime overhead. It "fits" if that is no more than 85% of usable memory; on a Mac the GPU can use about 75% of unified memory by default.
• Generation speed (tokens per second) ≈ $1 \div \left(\dfrac{\text{bytes per token}}{0.55 \times \text{bandwidth}} + 1.5\text{ ms}\right)$, where bytes per token = active parameters × bits ÷ 8.

Worked example. An 8B model at 4.8 bits: file $= 8 \times 10^9 \times 4.8 / 8 = 4.8$ GB, plus 5% ≈ 5.04 GB. With a KV cache of 128 KiB per token (see the context-windows lesson), 8,192 tokens of context add about 1.07 GB. Total ≈ $5.04 + 1.07 + 1.5 = 7.6$ GB. On a 16 GB Mac, usable GPU memory is about 12 GB and 85% of that is 10.2 GB, so it fits. With 120 GB/s of memory bandwidth: $4.8 / (0.55 \times 120) = 4.8/66 \approx 0.0727$ s, plus 0.0015 s, gives about 0.0742 s per token, or roughly 13 tokens per second, comfortably faster than most people read.

A selection script

Once you have evaluation results, the decision is a few lines of arithmetic:

candidates = {
    # name: (pass rate on your eval set, $ per task, median seconds)
    "small":  (0.88, 0.002, 1.1),
    "medium": (0.94, 0.008, 2.0),
    "large":  (0.96, 0.020, 4.5),
}
FIX_COST = 3.00     # $ of staff time to catch and fix one failure
MAX_LATENCY = 3.0   # seconds users will tolerate

def total_cost(pass_rate, model_cost):
    return model_cost + (1 - pass_rate) * FIX_COST

ok = {name: total_cost(p, c)
      for name, (p, c, secs) in candidates.items() if secs <= MAX_LATENCY}
for name, cost in sorted(ok.items(), key=lambda kv: kv[1]):
    print(f"{name:7s} ${cost:.3f} per task")
print("choose:", min(ok, key=ok.get))

The large model is excluded by the latency limit; of the rest, the medium model ($0.008 + 0.06 \times 3 = \text{\textdollar}0.188$) beats the small one ($0.362$). Change FIX_COST to 0.05 (five cents) and the small model wins ($\text{\textdollar}0.008$ versus $\text{\textdollar}0.011$ per task), which is the whole point: the right model depends on what errors cost you.

Common pitfalls

• Choosing from a leaderboard alone. Shortlist with public rankings, decide with your own evaluation set.
• Testing on two or three examples. Small samples make lucky results look like real differences.
• Ignoring error costs. Price per token is only part of the cost; review and rework often dominate.
• Using a reasoning model for everything. Its extra tokens cost time and money on tasks that do not need them, and on many APIs hidden reasoning tokens are billed as output.
• Forgetting output pricing. Output tokens usually cost several times more than input tokens, so long answers add up.
• Not re-testing after updates. Providers retire and replace models; re-run your evaluation when the model behind an alias changes.

In practice

• Test on your own task. Run the same three or four real examples through two or three models and compare. This tells you more than any leaderboard.
• Start with a mid-range model and move up only if it falls short, or down if it is clearly good enough.
• Use reasoning modes selectively, for problems that genuinely need step-by-step work.
• Check the data terms before sending anything sensitive, whichever model you pick.
• Expect names to change. New versions arrive every few months. Learn the categories, not just the current names. The model section of the AI Index lists current models and the hardware they need.