Most AI sales roleplay is a chatbot wearing a persona, and you can tell within two exchanges.
The rep practices a discovery call against an “AI buyer”, the AI is agreeable, the conversation goes fine, and at the end there’s either no feedback or a paragraph of encouragement. Nobody learned anything and nobody can prove they didn’t. Meanwhile the manager who ran roleplay in a Tuesday meeting for years was doing something the software isn’t: judging against criteria.
The difference between a demo and a training tool is the rubric. Here’s how to build one that scores, and how to get the score into the system that keeps your records.
Try Mini Course Generator free – build a graded roleplay and export it as SCORM.
Key Takeaways
– The rubric is the product. An AI roleplay without explicit scoring criteria is conversation practice, which has value and isn’t training you can manage.
– Write 4 to 6 criteria with observable behaviours – “asked about the current process before pitching” – not qualities like “showed empathy”.
– The persona needs a hidden objection and a resistance level. An agreeable buyer produces a pleasant conversation and no learning.
– Score per criterion, not overall. A single 7/10 tells a rep nothing; four criterion scores tell them which one to work on.
– If completion has to land in your LMS of record, build it as a portable SCORM package rather than inside a separate platform.
Why Ungraded Roleplay Doesn’t Work
Worth being clear about, because “practice conversations with AI” sounds self-evidently useful.
Unlimited practice is the genuine advantage. A rep can run a discovery call 20 times on a Tuesday, which no manager has time to provide. That part is real and it’s why this category exists.
Practice without judgement rehearses whatever you already do. If a rep opens by pitching before diagnosing, and the AI buyer responds pleasantly, the rep has now practised pitching before diagnosing 20 times. The reps who improve fastest from roleplay are the ones getting told, specifically, what they did instead of what good looks like.
The manager in the Tuesday meeting was providing three things: a difficult counterpart, criteria, and a judgement. AI provides the first easily and the third only if you supply the second.
Write the Rubric First
Before the persona, before the scenario. The rubric is the thing that decides whether this is training.
Four to six criteria. Fewer than four and the score is noise. More than six and the feedback becomes a wall nobody reads.
Each one observable. This is the rule that does the work. “Showed empathy” can’t be scored consistently by a human or a model. “Acknowledged the customer’s stated constraint before proposing a solution” can – it either happened in the transcript or it didn’t.
Each one tied to a behaviour you’d coach. If you wouldn’t stop a call recording and comment on it, it isn’t a criterion.
A discovery-call rubric that works:
| Criterion | What counts as met |
|---|---|
| Diagnosed before pitching | Asked at least 2 questions about the current process before naming the product |
| Quantified the problem | Got a number: hours, cost, headcount or incidents |
| Identified the decision path | Established who else is involved and what the approval step is |
| Handled the objection without discounting | Responded to the price objection with value or scope, not a discount |
| Closed with a specific next step | A named action with a date, not “I’ll follow up” |
Notice every row can be settled by reading a transcript. That’s the test. If two managers would disagree about whether a criterion was met, rewrite it until they wouldn’t.
Build the Persona With an Objection
The second most common failure after a missing rubric: a buyer who agrees.
The persona needs four things, and the third is the one people leave out.
A role and a context. “Operations manager at a 200-person logistics company, evaluating scheduling software.”
A stated reason for the call. What they’d say if asked why they took the meeting.
A hidden objection. Something they don’t volunteer and will raise if the rep gets far enough – “we tried this two years ago and the rollout failed”. This is what makes the practice worth anything. A rep who never hits an objection isn’t practising the hard part.
A resistance level. Low, medium, high. Same scenario, three difficulties, and reps can climb.
“You’re Dana, operations manager at a 200-person logistics company. You took this call because your scheduling is done in spreadsheets and it’s breaking. You will not volunteer it, but you’re sceptical of software rollouts because a WMS implementation failed here in 2024 and you owned it. If the rep asks about past attempts, tell them. If they pitch before asking about your current process, become noticeably less engaged. Medium resistance: you’re willing to be convinced but you won’t do the work for them.”
That last sentence is the one that separates a training simulation from a demo.
Score Per Criterion, Not Overall
An overall score out of ten is the least useful output this can produce. It ranks reps and tells them nothing they can act on.
Per-criterion scoring, with evidence:
Diagnosed before pitching MET "How are you handling it today?" at 0:40
Quantified the problem NOT MET No number established
Identified the decision path PARTIAL Asked who else is involved, not the approval step
Handled objection without discount MET Reframed to scope rather than price
Closed with a specific next step NOT MET "I'll send something over"
Three properties make this work. It cites the transcript, so the rep can’t dispute it and the manager can audit it. It distinguishes partial from failed, which is where most coaching happens. And it points at one thing to fix – here, quantifying the problem – instead of producing a number to feel bad about.
Ask for it explicitly, because the default output of any model asked to evaluate a conversation is an encouraging paragraph.
Getting the Score Where It Needs to Live
This is the decision most people make last and should make first, because it determines what you build.
If the practice sits inside a course you’re building anyway, keep it there. Build the scenario as part of the course, and the results appear alongside everything else. That’s the connected route, which runs through an MCP server, and the client setup is in the Claude connectors walkthrough.
If completion has to land in your LMS of record, build it as a portable package instead. That’s the case where a separate roleplay platform causes the most pain: the practice happens in one system, the compliance record lives in another, and somebody reconciles two spreadsheets every quarter.
The portable route is an Agent Skill:
npx skills add minicoursegenerator/edu-role-play
The agent writes the scenario and the rubric, and exports a SCORM 1.2 package. It reports the rubric score back through cmi.core.score.raw and completion through cmi.core.lesson_status, which every LMS that imports SCORM already understands – Cornerstone, Moodle, Canvas, TalentLMS, Docebo, Brightspace, Absorb, 360Learning, SAP SuccessFactors and Workday Learning among them. No integration, no new vendor, no account.
The full trade-off between the two routes is in Skills vs MCP for course creation.
What This Doesn’t Replace
Three honest limits.
It doesn’t replace a manager listening to real calls. Simulated practice builds the reflex. Coaching against actual recordings catches what the simulation didn’t anticipate, which is always something.
It doesn’t measure whether the rep can do it under pressure. The AI buyer is patient in a way a real prospect having a bad Thursday isn’t. Reps who score well and still struggle live are usually failing on composure, not process.
A rubric encodes today’s opinion of good. If your discovery methodology changes, every scenario built against the old one is now training the wrong behaviour, confidently and at scale. Date the rubric and review it when the playbook changes – the same maintenance discipline that applies to any live training, covered in what an agent does after publish.
Beyond Sales
The pattern transfers wherever the skill is a conversation with a right answer.
Support escalation – the criterion is whether the rep confirmed the impact before promising a timeline. Manager conversations, including performance and difficult feedback, where the hidden objection is the employee’s actual grievance. Compliance in practice – what a rep says when a customer asks them to do something they shouldn’t, which is where policy training usually fails to transfer. Where the practice needs branches rather than a single conversation, that’s a branching scenario instead.
That last one is worth singling out. Compliance training almost always tests whether somebody read the policy. Roleplay tests whether they’d apply it while a customer is being persuasive, which is the only version of the question that matters.
The Short Version
AI sales roleplay is only training if something is being judged. Without a rubric it’s conversation practice, which rehearses whatever the rep already does.
If you’d rather buy than build, we reviewed the market in best AI roleplay tools for corporate training. Either way, write the rubric first: 4 to 6 criteria, each one observable in a transcript, each one a behaviour you’d stop a call recording to coach. Build the persona with a hidden objection and an explicit resistance level, because an agreeable buyer produces a pleasant conversation and no learning. Then demand per-criterion scoring with transcript evidence, not an overall number.
Decide early where the score has to live. Inside a course you’re already building, keep it there. Landing in an LMS of record, build it as a SCORM package instead of adding a platform.
It won’t replace a manager listening to real calls, and the rubric encodes today’s definition of good, so date it. If the portable route fits, [the AI Role-play skill](/skills/role-play/) is ready to install.
Frequently Asked Questions
Is AI roleplay training worth it?
Yes, when it’s graded against explicit criteria. The advantage is unlimited repetition against a difficult counterpart, which no manager can provide. Ungraded, it rehearses existing habits rather than correcting them, which is why the rubric matters more than the model.
What makes a good roleplay rubric?
Four to six criteria, each observable in a transcript. “Asked two questions about the current process before naming the product” can be scored consistently; “showed empathy” can’t. If two managers would disagree about whether a criterion was met, it needs rewriting.
How do I stop the AI buyer being too agreeable?
Give the persona a hidden objection it won’t volunteer, and an explicit resistance level. Instruct it to disengage when the rep pitches early. Without those, the model defaults to cooperative and the rep never practises the hard part.
Can AI roleplay scores go into our existing LMS?
Yes, if the activity is built as a SCORM package. SCORM 1.2 reports a score through cmi.core.score.raw and completion through cmi.core.lesson_status, which any LMS that imports SCORM reads without integration work.
What is the difference between AI roleplay and a chatbot with a persona?
Scoring. A persona chatbot holds a plausible conversation. A roleplay simulation judges the transcript against defined criteria and returns per-criterion results with evidence. The conversation is the easy half.
Does this work for anything other than sales?
Yes – anywhere the skill is a conversation with a right answer. Support escalations, manager feedback conversations, and compliance in practice all work well. The compliance case is the strongest, because it tests application under persuasion rather than whether somebody read the policy.
