Looking into the chatbot KPIs is the only way to know if installing a chatbot onto your site was a success. There are tons of KPIs to track in business & marketing, and chatbots are no different. Monitoring all of them is confusing, time-consuming, and may obscure real problems and opportunities to improve.
This is why we’ve picked 11 essential chatbots KPI examples every marketer should track!
In this article, we’ll briefly explain:
- The definition of each KPI mentioned;
- How to interpret their results in a fitting context;
- What may go wrong with your chatbot, why, and how to fix the issues.
Important Chatbot KPIs To Measure
1. Total Number of Users
Total Number of Chatbot Users counts each user that interacts with a chatbot: each new, returning, bounced, and engaged user.
2. Number of Engaged Users
An Engaged User is the site visitor who communicates with the chatbot — sends messages and responds to the chatbot.
Naturally, you should aim for high numbers: it means that the chatbot is serving its purpose.
3. Session Duration
This one is self-explanatory as well: it shows how long the exchange between the user and the chatbot lasted within a single interaction.
Chatbot Session Duration depends on the user’s and the chatbot’s intention. If the chatbot responds quickly with a satisfactory answer, a short session is good. A longer session may also be understandable if the chatbot needs to resolve a more complex customer support ticket.
This is why you need to learn the context of each interaction: it helps explain why a certain session duration is good or bad. Bad reviews, high Fallback Rate, and aborted sessions are clear indicators that something’s off, for example, no matter the Session Duration.
4. Fallback Rate
Fallback Rate shows the number of times a chatbot couldn’t understand the user’s request.
A high Fallback Rate is always a bad sign. It means that you need to update your chatbot’s AI component and make it smarter.
There are several pain points that make things hard for conversational chatbot developers:
- Ambiguity — Idioms such as “take notes” are often lost in translation and misinterpreted as a command. Prosodic features of speech can cause mix-ups as well.
- Sarcasm, irony, and humor — Chatbots are still unable to “read the room” every time: intonation and context aren’t available to them, so chatbots miss the subtle cues in users’ speech.
- “Angry” and rhetorical questions — A frustrated customer asking “Do you know how many days I’ve been waiting for the package?!” may receive a precise answer, and get even angrier. Answering rhetorical questions may not irritate them as much, but it makes the session unnecessarily lengthy.
5. Completion Rate
The higher the Completion Rate, the better!
The “completion” refers to a predefined action your chatbot needs to take. Each completed chatbot goal means success, and you can set multiple goals for your chatbot.
6. Activation Rate
Chatbot Activation Rate will show how many users continued to engage with the chatbot after they’ve completed their initial consultation.
Again, higher means better — a high Activation Rate means you’ve successfully programmed your chatbot.
To interpret the Activation Rate in the right context, you also need to consider some of the KPIs mentioned above:
- Total Number of Users;
- The Number of Engaged Users;
- A number of New Users — a metric that shows the number of new, unique users who interact with the chatbot for the first time. If you’re using the chatbot as a part of a promotion campaign, a high number of new users means you’re doing fine!
7. Response Time
Chatbot Response Time measures how long it takes for a chatbot to respond to the question/request.
Response time should be as short as possible, as not to annoy the users. Slow response times may indicate that there’s a technical flaw you need to fix, or that your chatbot needs a more sophisticated AI to become better at Natural Language Processing. A high Fallback Rate points to the same problem.
Natural Language Processing (NLP) is a branch of AI/ computer science that teaches computers how to comprehend human language, including speakers’ intentions and sentiments.
8. Bounce Rate
Chatbot Bounce Rate shows the number of people who have entered and/ or browsed your website and exited without using the chatbot.
Interpreting the bounce rate isn’t all that simple and one-sided; a high bounce rate isn’t automatically a bad thing, as many sources claim. If the chatbot’s purpose is to provide customer support, a high bounce rate might as well mean your website is well-structured and answers the questions before they’re asked.
However, if your intention was to use a chatbot to increase conversions, a high bounce rate is a bad sign. In that case, you need to make changes in the way it works, how it targets site visitors, or what it offers to them.
9. Most Frequently Asked Questions / Chatbot FAQ
The Chatbot FAQ metric reveals what are the most common questions users ask chatbots. In the same fashion, you can track the most common pathways users take when interacting with a chatbot.
This metric is useful because it points out possible flaws on your website; usually, it’s the lack of useful information. For example, if you notice that users increasingly ask the same questions about shipping, it can mean one of those things:
- You forgot to include the information important to your users, and shipping-related content needs enrichment;
- Shipping information is hard to find, so you need to make it more prominent on the website;
- Shipping details are imprecise — you should rewrite and make them easier to understand, possibly add new information or erase outdated info.
10. Customer Satisfaction Rate
At the end of each human-chatbot exchange, it pays off to ask them to rate their experience and improve your chatbot accordingly.
First, ask “How would you rate our exchange?”, “Was I helpful?”, “Please rate our service”, or something along these lines. Provide the star-rating option below so they can vote.
Next, ask them to elaborate on their answer if they’ve rated the chatbot experience poorly; offer a couple of options and leave space for open-ended answers. Be sure to thank them for their review!
Customer Satisfaction Rate is one of the easier KPIs to interpret. Star-rating and pre-made answer options make it easier to quantify and track customer feelings towards the chatbot. On the other hand, open-ended answers are more challenging, but their specificity reveals the weak spots like a charm.
11. Ticket Deflection Rate
Chatbots are often used to handle more simple customer support tickets — the ones that don’t require a human touch and intelligence and are fairly simple to resolve. Ticket Deflection Rate counts all the instances where the chatbot couldn’t help the customer with their issue and referred them to a customer support agent.
This is one of the numbers you should try and keep low. If it’s high, it means that your customer support works more than it should, and your chatbot is ineffective.
There are several ways to decrease the Ticket Deflection Rate by looking into other KPIs:
- Fallback Rate — Chatbots are usually set up to direct the users to customer support when they can’t respond to their requests;
- Response Time — If it takes too long for chatbots to respond, impatient users will opt for a “real person” instead;
- Customer Satisfaction Rate — Take a look at the explanations given by dissatisfied users to find the answers;
- Chatbot FAQ — If the answer to some of these questions isn’t included, users will try and get a hold of a customer support agent; the solution is to make it visible and easy to find on the website.
12. Referral traffic
Chatbots can be used to help users navigate around the site, finding the most helpful pages for their needs. In order to track the effectiveness of an initiative like this, you could try generating UTM links to measure usage. That way, you can customize the source/medium that appears in Google Analytics, confirming which users came from your chatbot workflow (and what they did next).
Conclusion
Chatbot KPIs are clear indicators of whether your chatbot is making you a profit, or merely spending it without a real ROI.
As there are tons of KPIs to choose from, we advise you to stick to a couple of these and interpret them within the context, so you can understand why something does or doesn’t work.
Get register to botsify chatbot
