"A curated set of inputs, each with the expected behaviour or reference answer that your team has agreed represents good. You run the system against it, score the results (with code, judges, humans, or a mix) and get numbers you can compare release over release."
So what? When you cannot define a single correct response, a golden dataset gives a stable baseline to tell whether a change (a new model, an edited prompt, altered retrieval) improved things or made them worse.
Examples: Building a few hundred cases from production traces, support tickets and expert-authored hard cases, tagged by category so results can be sliced; holding back a portion of cases so prompt tuning does not overfit to the whole set.
So what? When you cannot define a single correct response, a golden dataset gives a stable baseline to tell whether a change (a new model, an edited prompt, altered retrieval) improved things or made them worse.
Examples: Building a few hundred cases from production traces, support tickets and expert-authored hard cases, tagged by category so results can be sliced; holding back a portion of cases so prompt tuning does not overfit to the whole set.
