Few-Shot Learning
- few-shot prompting
- in-context learning
What is Few-Shot Learning?
Few-shot prompting includes several input–output examples in the prompt to demonstrate the task. The model infers the pattern from those examples at inference time — no weights change, which is why the technique is also called in-context learning.
In practice
Examples communicate what instructions struggle to: exact output format, edge-case handling, the boundary between two similar categories, and the register the response should adopt. They are especially effective for classification with fuzzy labels and for structured output whose shape is easier to show than to specify. Zero-shot instructions are often sufficient for capable models on common tasks; examples earn their token cost when the task is idiosyncratic to your domain.
Composition matters more than count. Examples should span the range of inputs you expect, including the difficult cases, because a set drawn entirely from easy inputs teaches the model that the task is easy. Ordering has an effect, so a shuffled evaluation is worth running before you assume a gain is real. Every example is permanent context on every request, which makes this a direct trade against latency and cost — three well-chosen examples usually beat twelve redundant ones.
The misconception embedded in the name is that the model is learning. Nothing persists: the pattern is inferred fresh from the context on every call and is gone afterwards. If a behaviour must hold on every request forever, examples are a recurring tax that fine-tuning eventually replaces — but only once the behaviour is stable enough to be worth freezing into weights.
Related terms
Articles covering this
Where Few-Shot Learning shows up in practice rather than in definition.