Skip to content

Writing prompts that survive model updates

By Anwar Benhamada · August 6, 2026

You spent an afternoon tuning a prompt. It works. Three months later the provider ships a new model version, and your output quality drops — same prompt, different behaviour.

This is the maintenance cost nobody prices in when they build a workflow on these tools. It’s reducible, but only if you know which parts of a prompt are load-bearing and which are superstition.

What makes a prompt fragile

Magic phrasing. “Take a deep breath”, “you are a world-class expert”, “I will tip you $200”. These sometimes measurably helped specific model versions. They’re tuned to a particular model’s quirks, which is exactly what changes.

Exact output formatting by example only. If your parser depends on the model producing precisely Answer: X because your examples showed that, a new version that prefers markdown headers will break your pipeline while producing better answers.

Implicit reliance on length. Prompts tuned so the model stops at roughly the right point break when the new version is more or less verbose by default.

Chains of reasoning you specified manually. “First do A, then B, then C” was essential when models reasoned poorly by default. Newer models often do this better unprompted, and your rigid chain now constrains rather than helps.

What holds up

Explicit constraints, stated as rules. “Output must be valid JSON matching this schema” survives every model change, because it’s a requirement rather than a hint. Better still, enforce it structurally — most providers support a schema-constrained output mode, and a constraint the API enforces cannot drift.

Concrete examples of the task, not of the phrasing. Two or three input/output pairs teach the shape of the work. That transfers.

Negative examples. “Do not do X, here’s what X looks like” is unusually durable, because it defines a boundary rather than a style.

Context the model cannot infer. Your conventions, your domain rules, the thing that’s true about your data. No model update makes this unnecessary — it’s information, not technique.

Task decomposition. Splitting one large ambiguous job into three small well-specified ones survives everything, and it makes failures diagnosable.

The test that tells you which you have

Run your prompt against a different model — a smaller one, or a competitor’s.

This takes ten minutes and it’s the single most useful thing you can do to a prompt you depend on.

Build a regression set before you need one

The reason model updates hurt is that people discover the regression in production, from a user complaint, weeks later.

Keep ten to twenty input/output pairs that represent the work — including the edge cases you already know are hard. When a provider announces a new version, run them and diff. Twenty minutes, and it converts a vague worry into a yes-or-no answer.

Store the pairs as files in your repo alongside the prompt. If the prompt is load-bearing, it deserves the same treatment as code: versioned, reviewed, tested.

Pin your model version

Most providers let you pin to a dated snapshot rather than a moving alias. Do it for anything in production.

Then updates become a thing you schedule and test, rather than a thing that happens to you. The cost is that you have to actually do the upgrade occasionally — which is a much better problem than silent drift.

The uncomfortable part

Some prompt engineering is genuinely temporary. Techniques that mattered enormously two years ago are now unnecessary because the models improved.

That’s fine — as long as you can tell which of your prompt is task specification and which is model management. The first is an asset. The second is technical debt with an expiry date you don’t control, and it should be labelled as such in your own code.

A useful habit: comment the workarounds. // verbose because v3 truncated here tells future-you what to delete after the next upgrade.