Transparency and measurability are very important to determine the success of a chatbot. With the two measurement methods provided, suitable insights can be gained.
It is important to decide for each individual chatbot response whether it has a performance-oriented quality or whether the quality of a chatbot response should be measured by the user rating.
Here the rule of thumb helps:
Informative inquiry + information = user rating, which is to be measured. In other words, was the information helpful for the user?
Transactional request + information = user conversion to be measured. In other words, did the user click on the goal, thus the link?
Feedback question
There are informative intents that provide users with a piece of information. In this case, a feedback question is useful at the end of the chatbot answer, e.g. "Was this answer able to answer your concern?" + Quick Replies "Yes" and "No". Users can then easily select whether they are satisfied with the content of the answer.
In the Hub area Statistics, under Positive / Negative Feedback, you can see how much positive or negative feedback there is overall. Under Top Intents and in the associated focus report, the ratings given for the individual topics are shown in green or red in the statistics and the respective number is displayed when the mouse is moved over them.
Conversion goal
There are also intents that aim to get users to take a further action, e.g. to visit a landing page, i.e. to click on a URL button in the chatbot response. For such topics, it makes sense to track the success of the response content in the form of a goal.
A Goal is reflected in the goal that is to be achieved with the chatbot response.
Apart from a link within a chatbot answer, a follow-up question (redirection to a form or intent) can also be tracked as a conversion.
The outgoing links in the last dialogue step are automatically regarded as the goal of the response. This can be in the form of a follow-up message or a simple link that leads to a landing page.
- There is no separate evaluation of which URL button was clicked if an answer contains several URL buttons. If this is to be measured separately, ref parameters can be added to the links, which can be read out using an analysis tool such as Google Analytics.
- If a follow-up is stored at the end of an answer, only this element is measured as a conversion goal and not URL buttons, if the answer uses such.
Whether chatbot users have reached a conversion goal can be seen in the Statistics section under Conversions.
There, it is displayed how many users have successfully converted by means of a chatbot intent in the period under review. Under Top Intents and in the associated focus report, the statistics show the proportion of completed conversions for the individual intents and the respective number is displayed when the mouse is hovered over.
Change measurement method
Whether user ratings or conversions are to be measured can be set individually for each theme. To do this, the editing mode of a theme must first be called up.
In the settings, which appear by clicking on the settings button (
) on the right, you can now choose between "Rating question" and "Conversion target". By default, the "Rating question" is set.
If the conversion goal is set as the success measurement, the feedback question is automatically removed as an element in the answer editor and the happy path is set, which is played out after the conversion goal is reached.