What matters for your decision
A benchmark of clean FAQ wording alone can hide failures with typos, missing details and user language. Record expected handling even when a direct answer is not appropriate.
Put this into a pilot
- Label each case with intent, channel, language and whether the agent should answer, clarify or transfer.
- Add partial information, outdated policies and interruption cases that cover specific failure modes.
- Version the dataset and compare the same held-out cases after changes to the model, sources or instructions.