Learning Program
Chatbot Analytics
done properly
Most teams deploy chatbots and then guess at what's working. This program teaches you to read the data — session drop-off rates, intent resolution, handoff triggers — and act on what you find. 8 weeks, 3 live group sessions per week, cohort size capped at 14.
Program Structure
Six modules, one clear thread
Each module builds on the previous one. You won't find isolated lectures — every session connects to a running project your group works on together, using real chatbot data from anonymised production environments.
Metric foundations
What chatbot metrics actually measure versus what they appear to measure. We cover containment rate, CSAT correlation, and why a 91% resolution rate can still indicate a broken bot.
Conversation flow analysis
Reading session transcripts at scale. Participants learn to identify fallback loops, intent misclassification clusters, and the 3 conversation patterns that reliably predict high abandonment.
KPI design for teams
Building dashboards that non-technical stakeholders can use without misreading. Covers threshold-setting, anomaly flagging, and which 6 metrics belong in a weekly ops review.
Handoff and escalation data
When a bot transfers to a human agent, that event carries data. This module focuses on extracting signal from handoff logs — timing, trigger phrases, agent resolution time — to close gaps in automation.
A/B testing in live bots
Running controlled experiments on deployed chatbots without disrupting production. Participants design and evaluate 2 live tests during this module using the group's shared test environment.
Reporting and decision cycles
Translating analytics into decisions that get implemented. The final module covers structuring monthly performance reviews, presenting findings to leadership, and building a continuous monitoring habit.
Format
Group sessions via video, recorded for async review
Weekly load
Sessions + independent practice exercises
Language
Materials available in written English throughout
Access
Participants join from any timezone, scheduling rotates
Where participants land after eight weeks
The program doesn't promise transformation. It gives you a repeatable process for reading chatbot performance data, finding the problems worth fixing, and communicating what you find to the people who control the roadmap.
Structured audit process
A documented 5-step method for auditing any chatbot deployment, applicable across platforms — not tied to one vendor's tooling.
Dashboard templates
Ready-to-adapt templates for weekly and monthly reporting, built around the 6 metrics that matter most in operational reviews.
Group practice network
Cohort members typically continue exchanging data interpretations and test results for months after the program ends — the group dynamic persists.
| Skill area | Before the program | After the program | Difficulty level |
|---|---|---|---|
| Reading session transcripts | Manual, time-consuming, inconsistent | Pattern-based review in under 40 minutes per audit | Foundational |
| KPI selection | Defaulting to platform-provided metrics | Custom metric sets aligned to business objectives | Intermediate |
| Stakeholder reporting | Raw data exports or verbal summaries | Structured reports with clear decision prompts | Intermediate |
| A/B test design | Ad hoc changes without control conditions | Controlled experiments with defined success criteria | Advanced |
| Escalation data analysis | Rarely examined, treated as support overhead | Systematic handoff log review integrated into weekly ops | Advanced |