How do you write a good system prompt for a customer service chatbot?
What a system prompt is, what belongs in one, the mistakes that break customer service bots, and how prompt changes should be tested.
A good system prompt gives a chatbot one clear job, a bounded scope, a concrete tone with examples, and explicit instructions for what to do when it does not know the answer. That is the whole recipe. Most bad bots are not failing because the model is weak. They are failing because nobody ever decided what the bot should refuse to do.
The pattern behind good prompts is boring and repeatable, and that is good news, because it means it can be learned.
What is a system prompt?
A system prompt is the standing instruction a chatbot reads before every conversation. The visitor never sees it. It tells the model who it is, what it is allowed to talk about, how it should sound, and what to do at the edges. Think of it as the employee handbook for an assistant that reads the handbook fresh every single time, and follows it more literally than any human would.
That literalness is the key fact about prompts. The model does not know what you meant. It only knows what you wrote.
Why does the prompt matter more than the model?
Because the model is a generalist and your business is specific. The same model, given two different prompts, will produce a bot that politely redirects off-topic questions and a bot that happily gives legal advice under your company's name. Model quality sets the ceiling. The prompt decides how close to that ceiling you actually get.
There is a second reason: the prompt is the only part you fully control. You cannot retrain the model, but you can always sharpen the instructions.
What belongs in a customer service prompt?
Six things, roughly in this order:
- A role and an audience. Who the bot is, who it serves, and on whose behalf it speaks. One or two sentences.
- A scope with hard edges. What it answers, and just as importantly, what it does not. "You help customers with orders, deliveries and product questions. You do not give medical, legal or financial advice."
- A grounding rule. Tell the bot to answer from the provided knowledge and to say so plainly when the knowledge does not cover the question. This single rule removes most hallucinations.
- A tone with examples. Adjectives alone do not work. "Friendly but professional" means nothing to a model. A short example exchange, written the way you want the bot to sound, is worth a paragraph of adjectives.