What generative AI is, stated in a single sentence: software that produces new text, images, audio, or code by predicting what should come next. Those predictions rest on patterns learned from an enormous quantity of existing material. It does not look anything up. That one fact explains both why such a system can write a plausible paragraph on almost any subject and why it can invent a citation that does not exist. By the end of this guide you should understand how the prediction works and where the invented answers come from. You should also be able to put these tools to work on a real task without being embarrassed by the result.
Throughout the guide, one small business serves as a running example: an independent insurance agency with twelve staff, handling roughly 220 policyholder emails each week. Its questions are repetitive, its answers have to be accurate, and a single wrong sentence can create a legal obligation. It is a useful test case precisely because it is unglamorous. Readers who would rather have the vocabulary first will find the terms defined in the Learn AI hub.
Table of Contents
- What is generative AI, and what does “generative” actually mean?
- How does a model learn to generate anything?
- How does it produce a whole answer one word at a time?
- What are the main families of generative models?
- Where does the knowledge come from, and what can’t it know?
- Why are fluent answers sometimes wrong?
- What does using one actually cost?
- How should a beginner actually use one?
What is generative AI, and what does “generative” actually mean?
Older artificial intelligence systems were, almost without exception, systems that sorted things. A spam filter decides whether a message belongs in the inbox or the junk folder, a fraud model assigns a transaction a score between zero and one, and a recommendation engine places an item into a category. Every one of those outputs is a label, a number, or a ranking. Generative systems, by contrast, produce the thing itself, whether that thing is a paragraph of prose, a photograph, a recording, or a block of working code. Stanford HAI’s definition remains a useful one-paragraph reference to keep beside this article.
The distinction sounds minor and turns out to be fundamental, because the two tasks make very different demands on a system. Sorting an email requires only the ability to recognise a pattern that distinguishes spam from legitimate mail. Writing the reply requires the ability to produce language that a human reader will accept as a reply. That in turn requires knowing what an ordinary reply looks like, which words tend to follow which other words, and what tone suits the occasion. None of that can be hand-coded; all of it has to be learned from examples.
A second distinction is worth holding onto, because most public confusion about these tools originates there. Generation is not retrieval. When the agency’s staff search their policy database for a clause, the system locates that clause and displays it, and the answer is traceable to a document. When the same staff ask a generative model to explain the clause, the model locates nothing. It produces an explanation that resembles the explanations it has encountered before, assembled from the words that tend to appear near words such as “deductible” and “exclusion.”
For the agency, the consequence is that the model can draft the tone of an excellent answer while remaining entirely unreliable about the particulars of any single policy. That split between form and fact is the thread running through every section that follows, and it is worth keeping in view even where the discussion turns technical.
How does a model learn to generate anything?
Training begins with a very large collection of text, images, or audio, together with one deceptively simple exercise: given everything so far, what comes next?
For text, the model is shown a passage and asked to predict the next small chunk of characters, which is called a token. It guesses, it is told the correct answer, and it adjusts its internal numbers slightly so that on the next attempt it is a little less wrong than before. That adjustment is repeated billions of times across billions of examples, and the internal numbers being adjusted are called parameters or weights, of which a modern system typically has many billions. The layered structure in which those numbers are arranged is what our guide to deep learning unpacks in detail.
Nothing in that process involves a person explaining facts to the model. It is never told that Paris is the capital of France, but it is told, repeatedly and in an enormous variety of contexts, which word tends to follow which other word. Facts about the world are absorbed as a side effect of becoming good at prediction, because text about France so often mentions Paris immediately afterwards.
This is the reason a model can know perfectly well that a deductible is paid before coverage begins while remaining shaky about the exact figure on a specific policy. The first is a pattern that appears across millions of documents. The second is a fact recorded in one table in one file that the model has never seen, and no amount of fluency substitutes for having seen it.
For the technically curious: what training actually optimises
Text is broken into tokens by a tokeniser, which splits words into commonly recurring fragments, so that a word such as “unbelievable” may become three separate tokens. As a rough guide for English, one token corresponds to about four characters or roughly three-quarters of a word. That is why pricing and context limits are quoted in tokens rather than in words.
The model is trained against an objective called cross-entropy loss, which measures how much probability the model assigned to the token that actually came next. A low loss means the model was rarely surprised by reality. Training pushes that number down across an enormous sample of text, and the resulting weights constitute a compressed statistical description of the patterns present in that sample.
Image models apply the same principle to different raw material. Rather than predicting the next token, they learn to remove noise from a degraded image in a sequence of small steps. Some instead encode an image into a compact representation and rebuild it from there. The machinery differs considerably, but the underlying move, which is to learn a distribution from examples and then sample from it, does not differ at all.
After pre-training, most commercial systems pass through additional stages. Instruction tuning teaches the model to follow requests rather than merely continue text. A further stage, commonly described as reinforcement learning from human feedback, nudges it toward answers that human raters judge to be helpful. This explains why a raw pre-trained model will cheerfully continue a question with further questions, whereas a tuned assistant answers the question that was asked.
How does it produce a whole answer one word at a time?
Once training is complete, generation proceeds by repeating a single operation. The model examines everything produced so far, computes a probability for every token in its vocabulary, selects one, appends it to the sequence, and begins again.
It follows that a 300-word answer is the product of several hundred separate decisions, each of which is influenced by everything that preceded it, including the model’s own earlier output. Once a sentence has begun, the model is committed to keeping that sentence coherent, which exerts a powerful pull toward completion even in the cases where the completion is wrong.
There is no planning step and no fact-checking step anywhere inside that loop. Nothing in the mechanism asks whether the sentence it is producing happens to be true; the only question the mechanism poses is which token is most probable given the context. Fluency is therefore a property of the process rather than evidence about the content, and a confident, well-structured, grammatically flawless paragraph is precisely what the machinery was built to emit.
Variation enters through sampling. If the model always selected the single most probable token, its output would be repetitive and deterministic, so instead it samples from the distribution, which introduces a controlled amount of randomness. The principal settings governing this behaviour are temperature, which flattens or sharpens the distribution, and top-p, which restricts sampling to the most probable portion of the vocabulary.
A low temperature makes the model cautious and repeatable, which suits extraction and summarisation, while a higher temperature makes it more varied and more inclined to wander, which suits brainstorming. Neither setting makes the output more truthful, because temperature governs how adventurous the predictions are rather than how well they are grounded in fact.
For the technically curious: why the same prompt gives two different answers
At each step the model emits a probability for every token in its vocabulary, and sampling draws from that distribution instead of taking the maximum. Two runs can therefore diverge at the very first word and remain divergent thereafter, because each run is conditioned on its own accumulated history.
Top-p sampling, also called nucleus sampling, retains the smallest set of tokens whose probabilities sum to a chosen threshold and then samples within that set. Lowering the threshold truncates the long tail of unlikely tokens, which removes many strange word choices along with a smaller number of genuinely good ones.
Determinism, wherever it is required, is a configuration choice rather than an intrinsic property of the model. Two organisations asking the same question of the same system can therefore receive different answers, and neither system is malfunctioning when they do.
What are the main families of generative models?
The word “generative” covers several distinct technologies, and the differences between them matter more than the marketing tends to suggest. The text-based families receive fuller treatment in our guide to natural language processing. What separates the families is both the kind of material they produce and the training objective used to produce it.
| Family | What it generates | How it is trained | Typical strengths | Typical weaknesses |
|---|---|---|---|---|
| Autoregressive language models | Text, code, and anything representable as a token sequence | Predict the next token | Long-form writing, instruction following, summarisation | Fluent errors, no built-in fact check, weak at arithmetic |
| Diffusion models | Images, audio, video, sometimes molecules | Learn to reverse a noising process | High visual quality, strong control over style | Slow to sample, hard to get exact detail such as text in images |
| Generative adversarial networks | Images, synthetic data | A generator and a discriminator trained against each other | Fast generation, sharp outputs | Unstable to train, prone to mode collapse |
| Variational autoencoders | Compressed representations, then reconstructions | Encode to a latent space and decode back | Useful internal representations, smooth interpolation | Outputs tend to be blurrier than diffusion |
| Multimodal models | Any mix of text, image, and audio | Joint training across modalities | Answering questions about images, describing documents | Errors compound across modalities |
The practical consequence is that a claim about artificial intelligence is usually a claim about one family in particular. A diffusion model’s reputation for producing convincing images says nothing whatever about a language model’s reliability with numbers, and the reverse holds equally. Anyone evaluating a tool is well advised to establish which family it belongs to before accepting any general statement about its competence.
A naming problem is worth flagging as well. The phrase “generative AI” now covers consumer chat assistants, code completion tools, image generators, voice synthesis, and the models embedded quietly inside other software. The underlying technology is shared across all of them, but the interfaces differ enormously, and so does the degree of freedom each tool gives the user to verify what it produced.
Where does the knowledge come from, and what can’t it know?
A trained model carries its knowledge inside its weights, and those weights are frozen once training concludes. This produces the single most practically important limitation of the technology. The model’s built-in knowledge stops at a point in time called the training cutoff, and nothing about the model’s fluency indicates where that cutoff falls.
When such a model is asked about an event that postdates its cutoff, it does not decline the question. It instead produces an answer of the kind that would have been plausible before the cutoff. That follows directly from the training objective, because the model was rewarded for producing probable continuations of text rather than for recognising the boundaries of its own data.
The standard workaround is to stop relying on the weights and to supply the necessary information at question time. The material so supplied occupies what is called the context window: the quantity of text, measured in tokens, that the model can see while composing an answer. Everything inside that window can be used, nothing outside it exists for the purposes of that request, and the model does not remember a previous conversation unless the surrounding software resends it.
Retrieval-augmented generation, usually shortened to RAG, is the conventional means of making that process systematic. Before answering, the system searches a document collection and retrieves the most relevant passages. Those passages are placed in the context window alongside the question, and the model then writes an answer that can use them.
For the insurance agency, this is the difference between a system that guesses at a policy exclusion and one that quotes the actual clause. It is equally the difference between a system that can state that something is absent from the documents it was given and one that invents a policy number. Retrieval does not make answers correct, but it changes the question being asked from “what is likely” to “what does this passage say,” which is a far easier question to get right.
The pace of change in this area is the reason releases are tracked in our AI news section rather than compiled into a frozen list. Longer context carries a second consequence that rarely appears in marketing material. Everything in the window must be processed on every request, so a model given a hundred-page document costs more and takes longer than the same model given a single paragraph. Larger windows represent a real capability rather than a free upgrade.
Why are fluent answers sometimes wrong?
The errors in question are called hallucinations, and they are not a bug in the sense of a broken component. They represent the mechanism working exactly as designed on a question it cannot answer: the model produces plausible text, and wherever the plausible text is untrue, the result is a hallucination.
These errors are not distributed evenly. Models are most reliable on patterns that appeared frequently in training data, and least reliable on specific facts, precise numbers, recent events, niche documents, and anything requiring a citation. The wider catalogue of failure modes is collected in our guide to the risks of AI. Names, dates, statute numbers, page references, and URLs are classic failure points, because each looks like an ordinary token and carries no internal check.
The scale is measurable, and the measurement is uncomfortable. In 2024 a Stanford RegLab team tested legal research tools built specifically on retrieval augmentation, among them Lexis+ AI, Westlaw AI-Assisted Research, and Ask Practical Law AI. They found that the tools hallucinated between 17% and 33% of the time (Stanford RegLab, 2024). They were professional products sold on a promise of accuracy, not general-purpose chatbots.
That result is the most useful finding in this article. If a vendor whose entire product is accuracy, built on retrieved professional documents, still fabricates roughly one answer in five, then confident tone in a chat assistant is not evidence of correctness.
The strongest objection to that reading is that the legal study was an adversarial test built to catch failures, whereas ordinary use, with the user’s own documents in the window and a human checking the output, is reliably adequate. The objection is fair, and it makes the point: the tool is safe in the ways described later in this article, and the danger lies in unverified output rather than in the technology itself. Because that danger is a property of the machinery, the checking has to be permanent.
Adoption is genuinely large, and the figures you meet depend on who was asked. Stanford HAI’s 2026 AI Index reports that 88% of surveyed organisations use AI in at least one business function. The same index puts regular generative AI use at 79% (Stanford HAI, 2026 AI Index, Figure 4.3.1, reporting McKinsey’s 2025 State of AI survey). The same report places generative AI’s population-level adoption at 53% within three years of release, which is faster than the personal computer or the internet achieved at a comparable stage.
Set that against an official statistical count and the picture changes. Eurostat’s 2025 enterprise survey found that 20.0% of EU enterprises used AI technologies, with 17% of small enterprises doing so against 55% of large ones (Eurostat, 2025).
Both figures are correct, and the gap between them is a lesson about scope. The HAI figure derives from a survey of organisations large enough to answer such surveys, whereas Eurostat is an official collection that counts micro-businesses alongside everyone else. Different populations and denominators produce different numbers, and none of them is lying.
The point generalises. Any statistic about adoption, satisfaction, or productivity remains meaningless until you know what was measured, on whom, and when, which is why our guide to the most important statistics on artificial intelligence collects those definitions alongside the numbers.
What does using one actually cost?
Commercial generative AI is billed by the token in both directions. Input tokens comprise the material sent to the model, including the prompt, the conversation history, and any retrieved documents. Output tokens comprise what the model produces, and output is usually priced higher than input because generation costs more than reading.
| Cost driver | Why it costs what it does | What a beginner can do about it |
|---|---|---|
| Long prompts and chat history | Every request reprocesses the whole conversation | Start a new conversation for a new task |
| Retrieved documents | More context means more input tokens per request | Retrieve fewer, better passages instead of whole files |
| Long outputs | Output tokens are priced above input and take longer to generate | Ask for the answer, not an essay |
| Reasoning-heavy requests | Some systems spend extra computation before answering | Reserve them for problems that need it |
| Repeated work | The same request run twice is billed twice | Cache or reuse results you already have |
Two further costs are less obvious than the ones in the table. The first is the attention cost inside the model. Relating every token to every other token grows with the square of the context length, which means doubling the material in the window does considerably more than double the computation. The second is human cost, because a cheap answer that must be checked line by line is often more expensive than a careful one. That is why verification habits matter far more than token prices.
How should a beginner actually use one?
The practical advice reduces to a question of division of labour. Ethan Mollick, a Wharton professor who studies how people work alongside these systems, describes the situation bluntly: several billion people now have access to a kind of intern.
They are weird, somewhat alien interns that work infinitely fast and sometimes lie to make you happy, but interns nonetheless. — Ethan Mollick, One Useful Thing, 2023
The intern framing is useful because it makes the right questions obvious, namely what would sensibly be delegated to a fast, tireless, occasionally unreliable assistant, what would never be delegated, and what would always be checked.
The sequence that follows works well for a first real task.
- Pick a task where a mistake is cheap and visible. Drafting a first version of a routine email, summarising a document whose contents you already know, or reformatting existing notes all qualify. Legal notices, medical advice, and anything a customer will act upon without review do not. Our tool reviews cover which products are worth paying for once the requirement is clear.
- Give it the material it needs. Pasting in the policy clause, the invoice, or the meeting notes turns the exercise into a reading question, whereas a model working without those sources is answering from memory and answering badly.
- Write the instruction as though briefing a new colleague. Stating the audience, the length, the tone, and anything that must not appear produces a usable draft, because vague instructions yield generic output and generic output is what makes AI writing recognisable.
- Ask for the claim, and then for the evidence. For factual work, require a quotation or a source for every assertion, and treat any claim whose source is absent from the supplied material as unverified.
- Check the specific details first. Names, numbers, dates, statute references, URLs, and anything that would prove embarrassing are simultaneously the highest-failure items and the cheapest to verify.
- Keep confidential material out. Assume that anything pasted into a third-party tool may be retained. For the insurance agency that means no claimant medical details, no policyholder financial data, and no documents covered by a client agreement.
- Record what worked. Keeping the prompts that produced usable drafts pays off, because improvement in results comes overwhelmingly from accumulated specifics rather than from discovering a magic phrase.
Running that loop for a week usually leads to an honest conclusion: the tool saves time on structure and first drafts, and saves almost none on verification. That is still a genuine gain, provided the verification is actually carried out.
Frequently asked questions
Is ChatGPT generative AI?
Yes. ChatGPT is an interface built on top of autoregressive language models, and those models constitute one family within the broader category of generative AI.
Is a chatbot the same thing as a large language model?
No. The language model is the trained system that produces text. The chatbot is software wrapped around it that manages the conversation, applies safety rules, and decides what to send to the model on each turn.
Can generative AI search the internet?
Only where the software around it performs a search and places the results in the context window. The model itself has no capacity to look anything up, which is why an identical question can produce a current answer in one product and an outdated one in another.
Why does it give different answers to the same question?
Because generation samples from a probability distribution rather than selecting a single fixed output. Settings such as temperature and top-p govern how much variation results, and neither of them makes an answer more accurate.
Do I need to know how to code to use this well?
No. Prompting is ordinary written instruction, and the skill transfers directly from the practice of managing people. Coding becomes useful only when the aim is to automate a repeated task or to connect a model to a private document collection.
Conclusion
Generative AI predicts what comes next from patterns in its training data, which accounts both for the fluency of its writing and for its inability to say when it is wrong. The appropriate response is not to trust it less in the abstract, but to verify it in the specific. Supply the source material, demand the evidence, and check names, numbers, and dates before anything leaves your desk.
The next step is to build that habit on a single low-risk task this week. Readers who want the next article in sequence can subscribe here, and it will be sent when it publishes.