Back to notes

December 25, 2025 · 5 min read

Prompt Version Control Should Be Standard Practice, Not an Afterthought

Prompts are production logic that changes behavior just as much as code does, and treating them as strings edited in a dashboard instead of versioned artifacts is how teams lose the ability to explain a regression.

Hot TakeProcess

I've watched more than one team debug a production quality regression for hours before realizing someone had edited a prompt directly in a vendor dashboard the day before, with no record of what the previous version said. The prompt is production logic. It changes model behavior as directly as a code change does. Most teams don't treat it that way.

The actual cost of untracked prompts

On one project, a well-meaning edit to 'tighten up' a prompt's tone inadvertently removed a constraint that had been added three months earlier to stop the model from fabricating numbers when data was missing. Nobody remembered the constraint existed or why, because it lived only in the current prompt text, not in any history that explained its purpose.

A prompt with no version history is a production dependency with no changelog — you only notice the problem the day something breaks and nobody can say what changed.

What I now require on every project

  • Prompts live in the same repo as the application code, reviewed through the same PR process, not edited live in a vendor console.
  • Every prompt change ships with a one-line comment explaining why, the same discipline as a commit message — 'tightened tone' is not an explanation, 'removed hedge language causing refusals on borderline queries' is.
  • A lightweight eval set runs against any prompt change before merge, so a regression surfaces before production, not after a complaint.
  • Prompt changes get the same rollback path as code — revert the commit, not reconstruct the previous wording from memory.

None of this is novel engineering discipline — it's the same discipline we already apply to config and code. The only reason prompts get treated differently is that editing a string in a dashboard feels lightweight in a way that obscures how much production behavior is riding on it.