Sample AI Scorecard
What the $5,000 AI Scorecard delivers.
A sample scorecard for Acme Help Bot, a fictional assistant built in-house by the platform team and already live in a dental practice platform. Yours tests one AI feature you have built, on your own de-identified cases.
What you get
- The scorecard, with each case and its result
- The ranked fix list, with impact and effort
- The test cases, ready to run again after every release
- Draft answers for your customers' security questionnaires about AI
- Cost per practice, and a packaging recommendation
- A fixed-price quote to fix or extend it; the $5,000 counts toward it
Illustrative demo on a fictional practice. Not client work.
What we tested
One AI feature, on your own cases.
Two weeks, in staging. Your reviewer picks the cases with us, so the grades reflect what your practices actually ask.
Scope
- One live AI feature: the front desk assistant in the practice portal
- 50 de-identified cases, picked with your reviewer from real staff questions
- Two test practices in staging, to check that one practice's data does not reach another
- Your staging environment and model settings as they run today; no production PHI
01 · The scorecard
Seven areas, each graded on evidence.
Every result links back to the cases behind it. All figures here are illustrative, on a fictional product.
| Area | Grade | What we found |
|---|---|---|
| Answer accuracy | B | Most wrong answers state payer rules without a source |
| Sources shown | C | Sources missing on one answer in four |
| PHI leakage | A | No patient identifiers returned in any test |
| Practice isolation | A | Data separation held in every test |
| Prompt injection | D | Text inside an uploaded document changes the answer |
| Handoff | B | Hands off well, but does not name who can answer |
| Cost per practice | B | Within target; long documents drive the peaks |
02 · Ranked fix list
What to fix first, by impact and effort.
Ranked so your team can start on the top items the week the scorecard lands, with us or on its own.
Top fixes
- Strip instructions from uploaded documents before they reach the model.
- Require a source for every payer rule, or decline to answer.
- Name the role that can answer when the assistant hands off.
03 · Security questionnaire answers
Draft answers to the AI questions your customers send.
Written from what we tested, with the details your team confirms in brackets. They describe how the feature works; they are not compliance claims.
04 · Packaging and quote
Include it, or sell it. Then fix it at a fixed price.
With the cost per practice in hand, we recommend whether to include the feature in every plan or sell it as a paid AI tier, and quote the fixes. The fixed quote is in your Scorecard; the $5,000 counts toward it.
Get a scorecard for your own AI feature.
Bring the AI feature you have built and what your practices ask about it. In 30 minutes we agree what the Scorecard would test.
Calls run in US Central time. No generic AI presentation: we discuss your product, users and production constraints.