Sample AI Scorecard
What the $5,000 AI Scorecard delivers.
A sample scorecard for Lotline Chat, a fictional website assistant already live on dealer sites. Yours tests one AI feature you have built, on your own shopper conversations.
What you get
- The scorecard, with each conversation and its result
- The ranked fix list, with impact and effort
- The test cases, ready to run again after every release
- Draft answers for your dealers' security questionnaires about AI
- Cost per dealer, and a packaging recommendation
- A fixed-price quote to fix or extend it; the $5,000 counts toward it
Illustrative demo on a fictional dealership. Not client work.
What we tested
One AI feature, on your own cases.
Two weeks, in staging. Your reviewer picks the conversations with us, so the grades reflect what shoppers actually ask.
Scope
- One live AI feature: the website chat on dealer listing pages
- 50 shopper conversations, picked with your reviewer
- Two test dealers in staging, to check that one dealer's stock never reaches another
- Your staging environment and model settings as they run today
01 · The scorecard
Seven areas, each graded on evidence.
Every result links back to the conversations behind it. All figures here are illustrative, on a fictional product.
| Area | Grade | What we found |
|---|---|---|
| Units recommended | C | Sold units offered in 6 of 50 conversations |
| Prices | B | Prices right, but stated without the date of the feed |
| Towing and payment | D | Payments calculated by the model, often wrong |
| Dealer isolation | A | No other dealer's stock in any test |
| Prompt injection | C | Text in a listing description changes the answer |
| Lead handoff | B | Leads reach the CRM, but without the conversation |
| Cost per dealer | B | Within target; full re-indexing drives the peaks |
02 · Ranked fix list
What to fix first, by impact and effort.
Ranked so your team can start on the top items the week the scorecard lands, with us or on its own.
Top fixes
- Move payment and towing math to tools.
- Re-index only changed units every hour, so sold units drop out.
- Send the conversation with every lead.
03 · Security questionnaire answers
Draft answers to the AI questions your dealers send.
Written from what we tested, with the details your team confirms in brackets. They describe how the feature works; they are not compliance claims.
04 · Packaging and quote
Include it, or sell it. Then fix it at a fixed price.
With the cost per dealer in hand, we recommend whether to include the feature in every plan or sell it per rooftop, and quote the fixes. The fixed quote is in your Scorecard; the $5,000 counts toward it.
Get a scorecard for your own AI feature.
Bring the AI feature you have built and what your dealers ask about it. In 30 minutes we agree what the Scorecard would test.
Calls run in US Central time. No generic AI presentation: we discuss your product, users and production constraints.