System Prompt

A good system prompt is specific about behaviour and silent about everything else. Role framing, hard constraints, output conventions, tone and the handling of unknowns all belong there. Task-specific detail and retrieved material do not — they change per request, and mixing them into the persistent block makes the prompt impossible to cache and hard to reason about. Because the system prompt is prepended to every call in the conversation, it is also the part most worth keeping compact.

It is a real engineering artefact and deserves the same treatment as any other: stored in the repository, versioned, and covered by an evaluation set so a change to fix one complaint does not silently break three behaviours. Providers that support prompt caching make a stable system prompt cheaper as well, since an unchanged prefix can reuse prefill work across requests.

The misconception with security consequences is that the system prompt is a boundary. It is a strong prior, not an enforcement mechanism. Instructions arriving later in the context — including instructions hidden inside a retrieved document or a tool result — compete with it, and sometimes win. Anything that must hold regardless of what the model decides has to be enforced in code outside the model, not asserted inside the prompt.