All guides
Free report
For product and engineering leaders at U.S. software companies shipping or changing an AI feature
One prompt fix can quietly break answers you never re-checked.
Stop testing AI by vibes: the eval plan template
How to test an AI feature before it ships and keep it working after: the golden set, the graders, the release gate, and a spreadsheet to run it.
This answers
- How do we test an AI feature before it ships?
- What goes in a golden set, and what blocks a release?
- How do model upgrades stay safe?
Report · 15 pages, PDF · Deployed in client work
What is inside
- 1Every change gets a number, not a feelingExecutive summary
- 2Build the golden set from your own productThe golden set
- 3Grade every case, and compare every runGraders and runs
- 4Release gates, and keeping the set aliveRelease gates
- 5A filled eval plan: Harborline, and the spreadsheetWorked example
- 6Bring this page to the Product ReviewYour results
You also get
- The golden-set method: five kinds of case, and who writes the expected answer
- Four graders, with a per-rule check for the AI judge
- Side-by-side runs and a release gate with a stated tolerance
- The model upgrade steps, and keeping the set alive after launch
- A filled plan for a fictional product, and the spreadsheet that runs it
Preview



