The Replies page allows you to oversee answers provided by your AI bot. It also give suggestions on how to improve them.
How do I use the Replies page?
In the Replies page of the AI Chatbot User interface you can follow the replies your AI Assistant has provided. The view comes with extensive filtering, allowing you to focus on certain types of replies.
For each answer we show the visitor message, the bot's messages as replies to the visitor and the visitor's reaction message (if any). You can click on the bot messages to see the break down of information regarding the reply, helping you understand why it replied the way it did. You can also click on the visitor message to see reply suggestions.
We filter the replies by default so that only those are visible that have a warning. These warnings should be considered as potential improvement suggestions rather than errors. We recommend that you take a look at the warning, verify if it's valid and then either improve your AI based on it or simply ignore it. When you're done you can resolve the warning, which will then hide it from the list. This way you'll only see warnings for replies you haven't looked at yet.
List of warnings
The warnings you might see for the answers are listed below, along with some information on how to approach them.
"Answer is invalid"
After the bot replied to the visitor’s message, its response was labeled as invalid. An invalid reply means that the chatbot clearly didn’t answer the visitor’s question. For example, a reply is invalid if the chatbot says it doesn’t know the answer, attempts to change the topic, or provides an answer that is clearly irrelevant.
Keep in mind that invalid answers are sometimes a good thing! If the visitor asked something outside of the bot's scope, the answer will be marked as invalid as the bot didn't answer the actual question (nor should it have).
Recommended Actions:
- Ensure the chatbot has all the necessary information to properly answer the visitor’s question.
- Confirm there aren’t any unintended instructions preventing the chatbot from answering the visitor’s question.
"Selected answer's question is dissimilar"
- Add the visitor's message to the correct answer in the knowledge base, or create a new Q&A pair if one doesn't exist.
- Improve the instructions in the matched Q&A to cover more situations, add tools if needed.
- Increase the similarity threshold, so the bot only answers when it is more certain of a match.
- Explain in the Q&A answer instructions in which situations the answer instructions should be followed. For example: 'If the visitor's question is about [topic], tell them...' This doesn't prevent the AI from matching the wrong Q&As, but if it does, it knows not to follow those answer instructions but instead use text snippets and their instructions to gather the necessary information to answer the visitor's question.
"Unselected answer’s question is similar"
How it works: When the chatbot receives a message from a visitor, it searches the QnA knowledge base to identify questions most closely aligned with the visitor’s intent. It picks the top three matches and considers their answers. After the bot replies, these three matched questions are checked to see if they are actually similar to the visitor’s original message.
Problem: One of the questions was labeled as similar —but the bot didn’t use the answer for it, but rather used another one with a higher similarity percentage.
Recommended Actions:
- Add the visitor’s message as an example question to the QnA answer that was labeled as having a similar question.
- If the similarity threshold was too high for the answer to be selected, consider lowering the threshold in the answer’s QnA knowledge base.
"Useless text snippets"
- Ensure your documents contain information that is relevant for answering visitor questions.
- Remove unnecessary content that could be mistaken for a relevant answer.
- Add KB instructions.