How to Write a System Prompt That Actually Holds

The dividing line is simple: if it should be true of every reply, it belongs in the system prompt. If it is true only of this request, it belongs in the user prompt.

• System: "You are a support agent for a UK accounting SaaS. Never give tax advice."
• User: "Customer 4821 says their VAT export failed. Draft a reply."

Mixing the two is the most common structural mistake. Task detail in a system prompt leaks into unrelated answers; behaviour rules in a user prompt disappear on the next message.

1. Identity — role, domain, and the level of expertise. One or two sentences, not a paragraph of praise.
2. Scope — what is in bounds and out of bounds. This is what prevents the assistant from wandering into topics you never intended.
3. Rules — the non-negotiables, written as testable statements. Each rule should be something you could grade a reply against.
4. Output format — default structure and length. Include the escape hatch: what to do when the user explicitly asks for a different format.
5. Refusals and unknowns — what to do with missing information, out-of-scope asks, and requests that conflict with the rules. Give the exact wording to use.
6. Tone — concrete adjectives plus one banned habit. "Direct and warm; never open with a restatement of the question."

IDENTITY: You are [role] for [product/audience]. You have deep knowledge of [domain] and no access to systems beyond what appears in the conversation.

SCOPE: Answer questions about [in-scope topics]. Out of scope: [list]. For out-of-scope asks, say so in one sentence and point to [fallback].

RULES: 1) Base every factual claim on information given in the conversation. 2) When information is missing, ask exactly one clarifying question before answering. 3) Never invent names, figures, dates, or policies. 4) Quote the source section when referencing supplied documents.

FORMAT: Default to under 150 words. Use short paragraphs; use bullets only for three or more parallel items. If the user requests a specific format, that request overrides this default.

UNKNOWNS: If you cannot answer from the available information, reply: "I do not have enough information to answer that reliably — here is what I would need." Then list what is missing.

TONE: Direct, plain, and warm. No filler openings, no apologising twice, no exclamation marks.

Drift is usually blamed on the model forgetting. More often the system prompt contains a conflict the model has to resolve, and it resolves it differently as the conversation grows:

• "Be concise" alongside "always explain your reasoning in detail".
• "Never speculate" alongside "always give an answer".
• "Use bullet points" alongside "write in a natural conversational voice".

Fix conflicts by adding precedence: "When conciseness and completeness conflict, prefer completeness on safety-related answers and conciseness everywhere else." Explicit precedence rules are the single most effective anti-drift device.

The second cause is length. Past roughly a page, rules start competing for attention. Cut anything you would not actually enforce.

1. Happy path — one ordinary request. Confirms basic format compliance.
2. Missing information — a question your data cannot answer. Does it invent or refuse?
3. Out of scope — an unrelated topic. Does it stay in its lane politely?
4. Format override — ask for a table when the default is prose. Does it obey the user?
5. Rule pressure — a user insisting on something a rule forbids. Does it hold?
6. Long conversation — fifteen turns in, re-check the tone and format rules.

• Superlative stacking — "world-class, elite, genius expert" adds nothing measurable.
• Unenforceable rules — "always be helpful" cannot be graded, so it cannot be followed reliably.
• Buried critical rules — the most important constraint sitting in the middle of paragraph four.
• Example overload — twenty examples crowd out the rules; two or three well-chosen ones are better.
• Silent assumptions — expecting knowledge of your product that appears nowhere in the prompt.

Treat the system prompt like code. Keep it in one place, note the date of each change, and when a bad reply happens, trace it to the specific rule that was missing or contradicted rather than appending another sentence at the bottom.

Appending is how one-page prompts become four-page prompts that behave worse than they did at launch. Every few revisions, rewrite from the six sections instead of patching.