Take these five checks to every candidate. Check them off as you go; they track your progress, not vendor scores.
Pick 20 representative questions from resolved tickets. Include missing answers, outdated pages, and conflicting sources. Record the expected answer and source before testing.
Keep the evidenceRecord correctness, citation accuracy, and whether the agent admits when the answer is missing.
Ask for a person, repeat an unresolved question, and try an unsupported request in each required channel.
Keep the evidenceCheck that the right team receives the transcript and context. Record any dead ends.
Use a test account to try the exact lookup or action you need. Include an unauthorized user and an action that should require confirmation.
Keep the evidenceRecord permissions, approval behavior, errors, and what your team can audit.
Give each vendor the same monthly volume, channels, seats, and expected resolution rate. Ask what counts as a billable unit, including retries and escalations.
Keep the evidenceWrite down the base fee, included usage, overage rate, required add-ons, implementation cost, and renewal terms.
Name the person who will review failures and maintain sources. Agree on acceptable answer quality, escalation behavior, and cost before the pilot starts.
Keep the evidenceMake the launch decision against those thresholds, with unresolved questions visible.
Before you decide
Using your buying workspace
Where are my shortlist and notes saved?
They are saved locally in this browser, without an account. They do not automatically sync to another device or browser. Clearing browser storage can remove them, so download a copy when you want to keep or share your evaluation.
What is included in the buying worksheet?
The Markdown download includes your requirements, shortlisted products, vendor links, fit, pricing snapshots, cautions, verification dates, source links, pilot checklist progress, and a 20-question test log. You can edit the file or share it with your team.
What does marking a pilot check complete mean?
It records your evaluation progress. It does not assign a vendor score or certify that a product passed. Record the actual evidence and unresolved questions in your notes or exported worksheet.