Can Rovo Help with Atlassian Problems? [Common Misconception]

[#1]

… better than a regular Web search
… faster than my Atlassian Admin
… even if I don’t know the correct terminology

Since I’ve rolled out Rovo in multiple Organizations, I’ve seen repeating patterns like:

“I’ve asked Rovo about my Permission problem but it couldn’t help me.”
“Rovo said I should click here, but this button doesn’t exist.”
”Rovo always gives me answers that only apply to Data Center, not Cloud”.

The assumption: Rovo is Atlassian’s AI so it should know everything about Atlassian Apps and troubleshooting them.

While that starts to become truer for Admin tasks and queries, the average user will oftentimes get more confused by Rovo’s answers rather than getting the help they need.

The main problem

Rovo is connected to the Atlassian Support Documentation per default. It’s part of the company knowledge and is queried always, unless you switch it off in each Chat session. This connector follows the principle of “Shit in, Shit out”.

1. Terminology in the Docs is still outdated (Issues / Work Items; Projects / Spaces). Even within the Apps, terminology on is still outdated on a database level (e.g. projects still being called projects in JQL). So naturally Rovo also confuses terms and defaults to talking about Confluence when asked about spaces.

2. Not every article explicitly mentions if it applies to Data Center or Cloud. Sometimes this information is hidden in Screenshots or the URL - nothing that Rovo digests.

3. The Support Docs are not always kept in Sync with recent App updates.

4. Newer features are often only documented in the Community forums. They are not part of the Support Doc Connector and therefore not digested by Rovo.

Web search as a solution?

If not blocked in your organization, you could use Web search to get more up to date information or source the Community posts. Rovo’s web search is quite limited though. It doesn’t neccesarily query a lot of sources and in my test, other AI systems have performed much better at the task.

Better alternatives

Use a search engine of your choice to find relevant articles in the Web as well as in the Support Docs. Note that you need to know some of the terminology to be successful. If you use Ecosia, you are not only saving ressources by not triggering AI needlessly but also planting some trees like the Community does this month :)

Or you could use any other LLM with Web search function. Perplexity is known for that but other models are catching up quickly.

You could also build your own Atlassian Helper Agent that's based on internal Atlassian knowledgebases, may use Web search focussed on the Community and also leverage the Support Docs. That Agent could both be used in the JSM portal or simply as Chat Agent in the Sidebar. Problem with that: Users need to actively choose this Agent - that's where I see the gap.

So,

Rovo is not your ultimate Atlassian Troubleshooting buddy and your best to rely on other solutions. I sort of wish it was different and sort of wish it wasn't. All the more reason for us Atlassian Admins to keep our jobs ;)