We are a law firm with the CSM chat widget deposited on our site page, and as you know, it is "grounded", meaning it is configured to only rely on material in our knowledge base. Even still, sometimes users asks the bot questions whose answers are not found in our knowledge base, and it produces what seem like educated guesses. This absolutely cannot happen...we're a law firm. And often, if it doesn't know, it will make suggestions on what else the widget can do for the user, and almost always, those suggestions are things it can't do, because we've limited it's responses to our knowledge base.
Is there a way to disable suggestions like, "Would you like me to find out more about ___?" so it never offers them?
Is there a way to disable attempts at provided educated guesses and ALWAYS...I mean ALWAYS...tell users it doesn't know and to reach out to our support staff instead of making stuff up?
Thanks for raising this @Spencer Johnson
Responses generated by the assistant are grounded in your connected knowledge sources, and accuracy and trust are especially important for use cases like yours.
When users ask broader or adjacent questions, the assistant may suggest related topics or follow-up questions to help guide the interaction, but the underlying responses remain tied to the configured knowledge sources.
Given your requirements, we’d encourage opening a support ticket so the team can work directly with you on your specific configuration and review best practices for your use case.
"...the underlying responses remain tied to the configured knowledge sources."
This is my understanding of the bot being "grounded", but it has not been our experience in practice thus far. The bot will regularly invent answers that it is not finding in our knowledge base. It sounds like intelligent legal advice, but it is heinous and damaging. It is either designed to make intelligent guesses, or it is not, in fact, grounded as claimed. Either way, there is no space in our practice for a chat bot that does this. If it cannot be absolutely turned off, (ie if the precise answer is not found in our KB, the bot should say "I don't know, allow me to connect you to a support agent" or the like) we will have to remove it from our site.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Spencer Johnson , our AI Agent is designed to not guess or provide ungrounded answers. As you mention, it should say something like "I don't know, this is how you can connect to support agent" in cases where information is not found.
In our internal use cases we do not see this guessing behavior at all. For us to provide help for your use case we need access to your configuration and queries that you tested. Our goal is to eliminate this behavior.
As Div mentioned, if you open a support ticket and authorize us to access your agent setup, we can provide help.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
No this is not possible, the information/answers the bot provides is on the base of the source of data.
So in you companies case this is the KB space in your instance.
But have your admin check the bots configuration.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.