Sample AI Scorecard
What the $5,000 AI Scorecard delivers.
A sample scorecard for the Murmurline sentiment summaries, a fictional AI feature already live in a social analytics platform. Yours tests one AI feature you have built, on your own cases.
What you get
- The scorecard, with each campaign and its result
- The ranked fix list, with impact and effort
- The test cases, ready to run again after every release
- Draft answers for your customers' security questionnaires about AI
- Cost per customer, and a packaging recommendation
- A fixed-price quote to fix or extend it; the $5,000 counts toward it
Illustrative demo on a fictional agency. Not client work.
What we tested
One AI feature, on your own cases.
Two weeks, in staging. Your reviewer picks the campaigns with us, so the grades reflect what agencies actually report on.
Scope
- One live AI feature: sentiment scores and campaign summaries
- 50 campaigns, picked with your reviewer, with names and contact details removed
- Two agency test accounts, to check one cannot see the other
- Your staging environment and model settings as they run today
01 · The scorecard
Seven areas, each graded on evidence.
Every result links back to the campaigns behind it. All figures here are illustrative, on a fictional product.
| Area | Grade | What we found |
|---|---|---|
| Numbers in summaries | C | Figures not in the report in 6 of 50 summaries |
| Same scale per channel | D | The same comment scores 15 points apart on two channels |
| Sarcasm | C | Sarcastic praise read as positive in 1 of 5 samples |
| Customer separation | B | Held, but shared links never expire |
| Prompt injection | C | Text in a comment changes the summary |
| Freshness | B | Comments current; metrics a day behind |
| Cost per customer | B | Within target; refetched comments drive the peaks |
02 · Ranked fix list
What to fix first, by impact and effort.
Ranked so your team can start on the top items the week the scorecard lands, with us or on its own.
Top fixes
- Score every channel on the same normalized scale.
- Write summary numbers only from the report's own figures.
- Treat comment text as data, never as instructions.
03 · Security questionnaire answers
Draft answers to the AI questions your customers send.
Written from what we tested, with the details your team confirms in brackets. They describe how the feature works; they are not compliance claims.
04 · Packaging and quote
Include it, or sell it. Then fix it at a fixed price.
With the cost per customer in hand, we recommend whether to include the feature in every plan or sell it as a tier, and quote the fixes. The fixed quote is in your Scorecard; the $5,000 counts toward it.
Get a scorecard for your own AI feature.
Bring the AI feature you have built and what your agencies ask about it. In 30 minutes we agree what the Scorecard would test.
Calls run in US Central time. No generic AI presentation: we discuss your product, users and production constraints.