Engineering 9 min read· 26 Aug 2026· By Engineering
Testing prompts with golden datasets
The lightweight eval process that catches regressions before your users do.
You wouldn't ship code without tests. Prompts deserve the same rigor — for less effort than you'd think.
The golden dataset#
20-50 representative inputs with expected outputs (or a rubric). Version this file. Run it on every prompt change.
Grading#
For factual answers, string match. For open-ended, use LLM-as-judge with a 1-5 rubric and a stronger model than the one under test.