# Training and inference

Explainer · Updated 2026-09-07
Canonical: https://aiforthedummies.com/articles/training-and-inference/

**Training** is the job of adjusting a model so that its outputs better match examples. It is slow, expensive, and usually done in batches by the lab or company that owns the run. **Inference** is using a finished (or temporarily frozen) model to answer a new prompt. That is what happens when you hit send.

**Your chat is usually inference.** Typing a question does not, by itself, retrain the public model. Some products later sample conversations for a future training run; that is a policy choice, not a law of physics. Read the product’s settings if that distinction matters to you.

## Why people mix the words up

Marketing says a tool “learns from you.” Sometimes that means the product stores context inside one thread. Sometimes it means a future training set. Sometimes it means nothing technical at all. The [product layer](https://aiforthedummies.com/articles/models-vs-products/) is allowed to remember; the base model often cannot update itself mid-sentence.

A related confusion: “It knows my files.” If you uploaded a document, the product may retrieve pieces of it at inference time. That is search-plus-generation, not the model having been retrained on your folder overnight.

## Fine-tuning and prompting are not the same

*Fine-tuning* is extra training on a narrower set of examples. It changes the model’s weights. *Prompting* is instructions and examples placed in the input at inference time. Prompting is cheaper and reversible. Fine-tuning is a real training job and can bake in both skill and bias. Neither one is a guarantee of truth. See [what AI is](https://aiforthedummies.com/articles/what-ai-is/).

## Cost, speed, and the environment — without invented totals

Training a large model uses far more compute than answering one prompt. Serving millions of prompts still adds up. That is why companies charge per token, cache answers, and offer smaller models. This desk will not invent a kilowatt-hour figure or a share of a national grid. When a lab publishes an energy note, cite that note; when they do not, say the number is unpublished.

## What this split does not settle

It does not tell you which vendor to buy. It does not tell you whether a system is “conscious.” It does tell you why a model can be outdated the day it ships, why a product can still look current (tools and retrieval), and why privacy settings are about logs and future training, not about the model rewriting itself every time you correct it. For harms that follow from that picture, read [risks without hype](https://aiforthedummies.com/articles/risks-without-hype/).

Next: [how to read an AI claim](https://aiforthedummies.com/articles/how-to-read-ai-claims/).