Ask five people on a team whether a shared AI skill is working, and you'll get five honest answers. They won't agree.
We've spent the past year asking companies how they know a shared skill is actually landing, and the same gap turns up everywhere we look. Each person can describe what they've seen. Nobody can describe what the other four did, because nobody watched.
Your team's AI usage data can. Every prompt sent, every edit made before sending, every time somebody solved the task some other way, all of it shows up there already.
What acceptance mix measures
A skill automates a task. To judge whether it's working, look at every attempt at that task, not just every time the skill itself ran. That full set of conversations is your N.
One of five things happened in each of those N conversations.
| Outcome | Definition |
|---|---|
| as-written | Current skill text ran unchanged; output stood with no edits after |
| edited-before | Skill text changed before it ran; a variant ran instead of the canonical version |
| edited-after | Skill ran as offered; later turns revised the output before anyone used it |
| skipped | Skill offered for the task; nobody used it |
| no-skill | No version of the skill offered; not built yet, or not compelling enough to surface |
Those five sum to N: as-written + edited-before + edited-after + skipped + no-skill = N.
Acceptance mix is as-written divided by N. That's the one number: the share of attempts that ran the canonical skill exactly as published, nothing changed before or after.
The remainder splits four ways, and each looks different in the conversation itself:
- edited-before. The prompt sent doesn't match the current skill text: a near-match with specific lines changed, usually pasted in from a saved copy rather than typed fresh.
- edited-after. The prompt matches the skill exactly, and the turns right after the output rework it: a correction, a "make this more X," a manual rewrite pasted back in.
- skipped. The skill got recommended in the flow of work, and the conversation carries on without it.
- no-skill. The task shows up in the data, and nothing gets recommended: no skill exists for it yet, or one exists and the model didn't find it worth offering.
None of that shows up in a dashboard. Telling those four apart means parsing the actual conversation, and a raw usage count won't show you any of this important detail.
If you want to run that classification by hand before reaching for something that automates it, here's how.
How to pull acceptance mix for one skill by hand
| Step | How to run it | What you get |
|---|---|---|
| 1 · Define | Name the task, and pull the current text of the skill that automates it | What you're sorting against |
| 2 · Pull N | Every conversation that attempts that task over the window you're checking, whether or not the skill was used | Your denominator |
| 3 · Classify | Tag each conversation as-written, edited-before, edited-after, skipped, or no-skill | The five counts |
| 4 · Compute | as-written divided by N, and the other four as their own shares | Acceptance mix |
| 5 · Compare the edits | Line up the edited-before prompts and look for the change several people made on their own | Your next version |
| 6 · Publish it | Ship that change, with the name of whoever wrote it, as the new canonical version | A loop that closes |
A few gotchas:
- Steps two and three need admin access to your team's AI usage data, prompt text included, not just your own account. Invocation counts alone won't get you there.
- Classifying edited-before versus skipped takes a judgment call on close matches. Write down your rule before you start, so one conversation gets the same tag twice.
- edited-after only shows up if you scan past the skill's output into the turns that follow. Stop at the first response and you'll miss it every time.
Measuring acceptance mix at scale
At one skill, this is a spreadsheet and an afternoon. At twenty skills, it's impossible.
So: run it on one skill this week. Classify twenty conversations by hand, find the person whose edited-before prompt beats the canonical version, and publish theirs with their name on it.
Do that once and you'll learn a lot about how your team uses your shared skills. You'll also learn that it's the slow way to find out what's already sitting in your AI usage data, if you had the right tools on top of it.