Virtual Service Agent not returning correct Confluence page

Question

I have Confluence. Added a space with 2 simple pages. Linked this Confluence space to a JSM project's Knowledge Base and making them available in cutomer portal.

Next I created a Virtual Service Agent for the portal, since Rovo agents require license and I want customers to be able to access too.

But VSA just cannot get it right, even if I give it the exact title of the page, it replied with wrong link:

I tried updating the Confluence page thinking it may be indexing issues, but has no effect.

Jaime Escribano _knowmad mood_ · Answer

Hi @KC Wong

I've also been playing around with Rovo Virtual Service Agents. (They ain't that bright just yet)

Have you attempted to limit the knowledge of your Rovo agent? Having the Confluence knowledge linked to the project shouldn't be enough for the Agent to consider that knowledge specially relevant.

What you wanna do is:

1. Navigate to your agent

2. Click on the Relevant Scenario for the situation you are in (note on scenarios later)

3. Inside the scenario, go to "knowledge" and include only those pages/scenarios that the agent HAS to know and no more.

-> I.E. my knowledge writer agent only has access to my personal space where I keep the templates it uses and the space where it has to write into.

It may also be usefull to set up an scenario specifically for whenever a user asks questions related to a topic. You can then point to the specific page URL in the prompt. (the scenario still needs access to that knowledge ;D )

See this documentation for some more detail into scenarios

Add scenarios to a Rovo agent | Rovo | Atlassian Support

Tomislav Tobijas · Answer

Hi @KC Wong ,

This definitelly looks frustrating 🫤

From my understanding, the VSA uses generative AI to search and summarize from all linked knowledge base articles, but it does not guarantee an exact title match. It may pick the article it "thinks" is most relevant, which can lead to the wrong article being shown.

So, it would appear to be a model problem. Not sure what would be the best way to report it to Atlassian - potentially via Atlassian Support 👀

Btw, if that article is relatively new (you've just added it), it might take some time to appear as the suggested answer/content by VA. Potentially, you could give it a couple of days and see how it will go 🤔

Cheers,
Tobi