Prompt Engineering

Designing and testing the input given to a language model. What a prompt is to a next-token predictor, system prompts and roles, zero-shot and few-shot prompting and in-context learning, how the choice and order of examples sway answers, chain-of-thought prompting and its limits, clear instructions and structured output, chaining, sampling temperature and nucleus sampling worked on one next-token distribution, prompt injection, and evaluating a prompt as carefully as a model, with the library's research notes on each.

A team wants a language model to label customer reviews as positive or negative. The first attempt is a single line:

Is this review positive or negative? "The battery died after two days."

The model answers with a paragraph discussing the review's tone, which the team's code cannot parse. The second attempt adds an instruction about the form of the answer, a sentence of context about what the labels are for, and three labeled examples:

We sort product reviews so that the support team sees complaints first.
Label each review Positive or Negative. Answer with the label only.

Review: "Arrived early and works perfectly." Label: Positive
Review: "Stopped charging within a week." Label: Negative
Review: "Does what it says, nothing more." Label: Positive

Review: "The battery died after two days." Label:

Nothing about the model changed between the two attempts. Only its input did. Prompt engineering is the practice of designing that input, and testing it, so that a model reliably does what is wanted. This article explains why prompts work the way they do, using the mechanism described in the language and generative AI articles, collects the techniques that published research has measured, and treats prompt design as what it is in a data science workflow: a choice among alternatives that has to be evaluated on held-out data like any other.

Model behavior changes from one version to the next, and advice that helps one model can be unnecessary or harmful for another. The techniques here are stated with the evidence for them and the models it came from; none is guaranteed to work on a given system, which is why the section on evaluation matters most.

What a prompt is

A language model computes a probability distribution for the next token given all the tokens before it (Generative AI). A prompt is the beginning of the sequence. Everything in it, the instructions, the examples, the document to summarize, the question, shifts the distribution of what follows. A pretrained model that has seen many labeled lists of the form "Review: ... Label: Positive" will tend to continue such a list with a label, which is why the examples in the second prompt above help: they make a one-word label the likely continuation.

Chat models are fine-tuned to treat their input as a conversation with roles. A system prompt, supplied by the developer of an application rather than its user, sets standing instructions: the task, the audience, the format, what to do with requests outside scope. User turns and the model's own previous replies follow. Anthropic's documentation describes setting a role in the system prompt as focusing the model's behavior and tone: "Even a single sentence makes a difference" (Anthropic, 2026a, "Give Claude a role"). The library's evidence map on written principles collects what research has measured about text that shapes model behavior, including system prompts and instructions given at inference time, and its first section asks where such text enters a model's training or deployment and who reads it.

A role in a prompt changes what the model draws on and how it writes; it does not add knowledge the model lacks. Asked to answer "as a tax attorney", a model writes in a tax attorney's register about what it learned in training, which may be out of date or wrong. Research on why personas work at all is summarized in the library's notes on The Persona Selection Model and The Assistant Axis.

Zero-shot and few-shot prompting

Brown et al. (2020, §2) defined the settings that the field now uses. In few-shot prompting the model is given "a few demonstrations of the task at inference time as conditioning", but "no weight updates are allowed"; one-shot allows a single demonstration; zero-shot gives only a description of the task. They typically used 10 to 100 examples, as many as fit in GPT-3's context window of 2,048 tokens. They called the model's ability to pick up a task from examples in its input in-context learning, and found that it improved with model size more steeply than zero-shot performance did (§1 and Figure 1.3). The library's note on In-context Learning and Induction Heads summarizes evidence about a mechanism that may underlie it.

Few-shot prompting is useful when the task is hard to describe but easy to show: a particular output format, a labeling scheme with conventions, a style. Its main costs are length, since examples use up the context and cost money to process on every call, and sensitivity to which examples are chosen.

The examples sway the answer

Zhao, Wallace, Feng, Klein and Singh (2021) measured that sensitivity. For GPT-3, "the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art" (abstract). They traced the variance to three biases (§4):

  • Majority label bias: the model favors answers that appear often among the examples. A sentiment prompt with more positive than negative examples pushes predictions toward positive, and on a fact-retrieval task with four examples, 50.2% of the answers of a 2.7-billion-parameter GPT-3 repeated one of the four example answers, where the correct repeat rate was 24.7%.
  • Recency bias: the model favors answers that appear near the end of the prompt, so a prompt ending with a negative example leans negative.
  • Common token bias: the model favors answers that are frequent in its pretraining data.

The practical lessons follow directly. Balance the labels among the examples, vary their order when testing, and evaluate more than one set of examples before trusting a result. The review prompt above ends with a positive example after a negative one, and two of its three examples are positive; whether that matters for a given model is an empirical question, which is the point.

The same mechanism can be turned against a model. The library's note on Many-shot Jailbreaking describes how hundreds of fabricated example dialogues, in which an assistant complies with harmful requests, can override safety training through in-context learning.

Chain of thought

Some tasks need intermediate steps: a word problem, a multi-step calculation, a question that combines several facts. Wei et al. (2022) showed that including worked examples whose answers spell out the reasoning, "a series of intermediate reasoning steps", improves performance on such tasks; they called this chain-of-thought prompting (abstract). On the GSM8K benchmark of grade-school math word problems, PaLM 540B solved 17.9% with standard few-shot prompting and 56.9% with chain-of-thought prompting (Appendix Table 2). The gains appeared only in large models, about 100 billion parameters or more; smaller models "produced fluent but illogical chains of thought, leading to lower performance than standard prompting" (§3.2). Gains were largest on the hardest problems and small or negative on one-step problems (§3.2).

Kojima et al. (2022) found that a single sentence could stand in for the examples: adding "Let's think step by step" before the answer raised the accuracy of an InstructGPT model (text-davinci-002) on the MultiArith problems from 17.7% to 78.7% and on GSM8K from 10.4% to 40.7%, with no examples at all (abstract). They called this zero-shot chain of thought.

Two cautions apply. First, these are results for particular models and benchmarks. Many current models are trained to reason before answering without being asked, and vendor guidance for such models changes accordingly; Anthropic's documentation, for example, describes models that decide for themselves how much to reason (Anthropic, 2026a, "Thinking and reasoning"). Second, written reasoning is not necessarily the reasoning that produced the answer. The library's thread on reasoning and faithfulness covers this in detail, from Measuring Faithfulness in Chain-of-Thought Reasoning, which found that how much a model's answers depend on its stated reasoning varies widely by task, to Reasoning Models Don't Always Say What They Think and Claude's Extended Thinking. A chain of thought is useful for accuracy and for spotting errors, but it should not be read as an explanation of the model's decision.

Clear instructions, context and structure

Much practical advice concerns writing the prompt clearly. Anthropic's guide for its models, as of this writing, is representative (Anthropic, 2026a, "General principles"):

  • Be clear and direct. State the output and constraints wanted, and give steps as a numbered list when order matters. Its test: show the prompt to a colleague with little context on the task; "If they'd be confused, Claude will be too."
  • Give the reason. Explaining why an instruction matters lets the model generalize from it. The guide's example replaces "NEVER use ellipses" with an explanation that the response will be read aloud by a text-to-speech engine that cannot pronounce them.
  • Use examples that are relevant, diverse and clearly marked, so that the model can tell examples from instructions and does not copy accidental features of the examples.
  • Separate the parts of a prompt (instructions, context, examples, input documents) with consistent markers, such as XML-style tags, so that none is mistaken for another.
  • Say what to do rather than what not to do, and match the style of the prompt to the style of output wanted (Anthropic, 2026a, "Control the format of responses").

These are sensible defaults for any model, but they are one vendor's guidance for its own current models, and the same documentation says of a technique that names a specific model: "treat it as measured on that model and re-check it against your own evals before applying it to another" (Anthropic, 2026a, "General principles").

Structured output

When code consumes a model's answer, the answer must be in a form code can parse, usually JSON with specified fields. Asking for the format in the prompt, with an example, usually works and sometimes fails: a missing field, a wrong type, text before the JSON. Some APIs offer constrained decoding, which restricts generation at each step to tokens that keep the output valid against a supplied schema; Anthropic's structured-outputs feature, for example, is documented as guaranteeing "schema-compliant responses through constrained decoding" (Anthropic, 2026b). A schema guarantees form, not content: the fields will parse, but whether their values are right still has to be checked.

Chaining

A complex task can be split into a sequence of calls, each with its own prompt, the output of one becoming input to the next: extract the facts from a document, then check them, then write the summary. Anthropic's guide describes the most common pattern as self-correction, generating a draft, reviewing it against criteria, and refining it, with each step a separate call so that it can be logged and evaluated (Anthropic, 2026a, "Chain complex prompts"). The library's note on Building effective agents describes prompt chaining and other workflow patterns, and argues for starting with the simplest arrangement that works.

Sampling: temperature and nucleus sampling

The prompt determines the next-token distribution; the decoding settings determine how a token is chosen from it. Greedy decoding always takes the most probable token. Sampling draws a token at random in proportion to its probability. Temperature reshapes the distribution before sampling: the model's scores (logits) $u_i$ are divided by a temperature $t$ before the softmax,

$$p_i = \frac{e^{u_i / t}}{\sum_j e^{u_j / t}},$$

so that $t < 1$ sharpens the distribution toward the most probable tokens, $t > 1$ flattens it, and $t \to 0$ approaches greedy decoding (Holtzman et al., 2020, §3.3, after Ackley, Hinton and Sejnowski, 1985).

A worked example makes the effect concrete. Suppose that after the prompt "The capital of France is" a model's logits for five candidate tokens are 4.0 for Paris, 3.0 for Lyon, 2.5 for the, 1.0 for France and 0.0 for banana (invented numbers, chosen for illustration). The resulting probabilities and their entropy, a measure of how spread out the distribution is (2.32 bits would be uniform over five tokens), are:

Temperature Paris Lyon the France banana Entropy (bits)
0.25 0.980 0.018 0.002 0.000 0.000 0.15
0.5 0.842 0.114 0.042 0.002 0.000 0.78
1 0.603 0.222 0.134 0.030 0.011 1.53
1.5 0.479 0.246 0.176 0.065 0.033 1.87
2 0.410 0.249 0.194 0.092 0.056 2.03
The same five next-token scores turned into probabilities at three temperatures. At t = 0.5 the most likely token takes 84% of the probability; at t = 2 it takes 41%, and the wrong answer Lyon and the irrelevant banana become much more likely. Temperature changes how often the model departs from its first choice, not what it knows.
The same five next-token scores turned into probabilities at three temperatures. At t = 0.5 the most likely token takes 84% of the probability; at t = 2 it takes 41%, and the wrong answer Lyon and the irrelevant banana become much more likely. Temperature changes how often the model departs from its first choice, not what it knows.

In 10,000 draws at each temperature, a token other than Paris came up 2.0% of the time at $t = 0.25$, 39.3% at $t = 1$ and 58.4% at $t = 2$.

Two corrections to common descriptions follow from the formula. Temperature does not change the range of data or knowledge a model draws on; it only rescales the probabilities of the tokens the model already scored. And a low temperature does not make output accurate. It makes the model choose its top-ranked continuation more consistently, which helps when that continuation is right and repeats the error when it is wrong. A model whose top-ranked next token after a request for a citation is a plausible but nonexistent title will produce that title reliably at low temperature. Holtzman et al. (2020) also found that the most probable continuations of long texts tend to be repetitive and bland, while pure sampling at $t = 1$ often drifts into incoherence (§1); lowering temperature "improves generation quality" but "at the cost of decreasing diversity" (§3.3).

Nucleus sampling (top-$p$ sampling) is their alternative: at each step, keep the smallest set of most probable tokens whose probabilities sum to at least $p$, renormalize, and sample from that set (Holtzman et al., 2020, §3.1). At $t = 1$ in the example, Paris, Lyon and the together hold 0.959 of the probability, so for $p = 0.9$ the nucleus is those three, with renormalized probabilities 0.629, 0.231 and 0.140, and France and banana can never be drawn. The tail of a large vocabulary, tens of thousands of individually unlikely tokens, is what nucleus sampling removes.

Prompt injection

An application that puts untrusted text into a prompt, such as a web page to summarize or an email to triage, gives whoever wrote that text a way to address the model. Prompt injection hides instructions in such content in the hope that the model will follow them as if they came from the user or the developer. Greshake et al. (2023, abstract) demonstrated "indirect prompt injection" against real applications, with outcomes including data theft and the spread of injected content between systems. Defenses combine training models to distinguish instructions from data, classifiers that screen untrusted content, and limits on what an application allows a model to do; the library's note on Mitigating the risk of prompt injections in browser use describes one developer's defenses for a browsing agent and reports that a residual attack success rate of about 1% still carries meaningful risk. For prompt design the lesson is to mark untrusted content clearly as data, never to rely on the prompt alone for security, and to restrict what actions a model's output can trigger.

Evaluating a prompt

A prompt is a choice among alternatives, and choosing among alternatives on the evidence of a handful of tries is the error that Training, Validation, and Test Sets warns against. The same discipline applies.

  1. Define success first. Anthropic's guide assumes three things before any prompt engineering: a clear definition of the success criteria for the use case, ways to test against them empirically, and a first draft to improve (Anthropic, 2026c). Its guidance on criteria asks that they be specific and measurable, for example an F1 score of at least 0.85 on a held-out set of posts, rather than "good performance" (Anthropic, 2026d).
  2. Build a test set that resembles the real inputs, including edge cases: empty or irrelevant input, very long input, ambiguous cases on which people would disagree (Anthropic, 2026d, "Eval design principles"). Keep part of it aside, untouched while the prompt is being revised, and use it once at the end, for the same reason a test set is kept aside when tuning a model.
  3. Grade automatically where possible. Exact or string matching and code checks are fastest and most reliable; another model with a detailed rubric can grade open-ended output but should itself be checked against human grades; human grading is the most flexible and the slowest (Anthropic, 2026d, "Grade your evaluations"). For classification tasks, the measures of Binary Classification Metrics apply directly.
  4. Report uncertainty and compare fairly. A difference of a few points between two prompts on 50 test cases may be chance. The library's note on Adding Error Bars to Evals works through standard errors and paired comparisons for exactly this situation, and the course wiki's Comparing Systems with Significance Tests covers the paired tests.
  5. Test the variations that should not matter. Given Zhao et al.'s findings, rerun the evaluation with the examples reordered and with small changes of wording; a prompt whose score swings widely under such changes is fragile.
  6. Re-evaluate when the model changes. A prompt tuned on one model version is an untested prompt on the next.

In Python

The structure of a prompt evaluation does not depend on the vendor. In the sketch below, ask stands for whatever function sends a prompt to a model and returns its text; everything else is ordinary Python:

import random

def evaluate(template, cases, ask, n_orders=5, seed=0):
    """Accuracy of a few-shot template over labeled cases, for several orders of its examples.

    template: dict with 'instructions' (str) and 'examples' (list of (text, label) pairs)
    cases:    list of (text, label) pairs held out from the examples
    ask:      function(prompt: str) -> str, supplied by the caller
    """
    rng = random.Random(seed)
    scores = []
    for _ in range(n_orders):
        examples = template["examples"][:]
        rng.shuffle(examples)
        shots = "\n".join(f'Review: "{t}" Label: {y}' for t, y in examples)
        correct = 0
        for text, label in cases:
            prompt = f'{template["instructions"]}\n\n{shots}\n\nReview: "{text}" Label:'
            correct += ask(prompt).strip() == label
        scores.append(correct / len(cases))
    return min(scores), sum(scores) / len(scores), max(scores)

Reporting the minimum and maximum over orderings, and not only the mean, shows how much of a prompt's score depends on an arbitrary choice. The temperature table and the nucleus example above are computed by tools/experiments/data-science/prompt-engineering.py in the site's repository.

References