AI tools for ticket categorization and summarization

Katie Nix May 9, 2024

Hi all! 

I have two use cases where I think an AI solution could help. We are open to paid 3rd party solutions!  

I've explored some of the built-in Atlassian Intelligence features and looked a bit on Marketplace, but not finding what I'm looking for yet.

1. Ticket categorization 

We have a huge backlog of feature requests and would like help categorizing them by "problem area". I can imagine using automation to pass ticket data one at a time out to an AI LLM (like the workflow described here) but I think a better solution would be trained on our own past tickets (i.e. we manually categorize 100 historical feature requests and the AI uses the history when categorizing new tickets). Does anyone know of a solution like this? 

2. Ticket summarization 

We have a project where developers and product managers answer questions about our product. There is often some back-and-forth on ticket comments until the question is fully answered. Then the ticket is handed off to our documentation team. It can take significant time for them to figure out what information from the ticket needs to be documented, and it would be helpful to have a quick summary to orient them as they pick up new tickets. The summary should consider not just the ticket fields, but also the comment history. Does anyone know of an app or integration that can smartly summarize, including ticket fields, comments, and questions?

 

 

1 answer

0 votes
Clark Everson
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
May 31, 2024

Hi @Katie Nix 

For number 1, it is possible to train AI on your own data but this actually wouldn't be an LLM for this type of AI. This would happen prior to even getting to an LLM, this is actually the foundation of AI. If your company has a data scientist they would be the best person to talk to, if not you may want to look at some consultants to help with this

For number 2 this is more what an LLM is about. If you were to use Jira Cloud's AI capabilities it should be able to do this from my understanding of how it works. However one thing to consider with AI it isn't just asking "summarize this", it needs to be prompted in a clear precise way for it to know what to look at.

So if I were to use Atlassians AI tool for instance and I was in a ticket with a lot of comments I would phrase it something like this :

1: 
You are a new team member onboarding into this work. Please review this ticket, including the fields and comments associated. Once reviewed please let me know when you are ready to provide a summary
AI responds: I am ready
You: Provide a summary of the most important data from a ticket that includes a brief overview of the problem statement, the acceptance and what the correspondence has been so far
Please format it as bullet points like
Problem statement
-Bullet 1
-Bullet 2
etc
Acceptance
-Bullet 1
Other information
-Bullet 1

etc

 

 

That is only one way, it depends on the model though

Once you make the templates for asking though they can generally be reused

Best,

Clark

Katie Nix May 31, 2024

Thanks Clark!

Understood about #1, I mentioned LLM because the linked blog post used ChatGPT as an example. Anyway, we don't have the resources for me to pull a data scientist off of something else and do something custom. There are some apps out there for sentiment analysis, but I haven't found anything on the marketplace for categorization that can actually use our own data. Just wondering if anyone else has found something.

For #2 - can you point me toward what feature you're talking about that would do this? There is a "Summarize" button that released recently for ticket Comments which fulfills most of my use case but has to be triggered by the person looking at the ticket, pressing the button each time they want a summary. I don't see a way to trigger anything like what you're describing as part of a workflow, via native Atlassian tools. But there's a lot of new stuff that I haven't looked at.

Clark Everson
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
May 31, 2024

Hi @Katie Nix 

It would be Atlassian Intelligence to my knowledge: https://www.atlassian.com/platform/artificial-intelligence

For the first, LLMs are where you would ask it what it thinks, but you still do the work(think asking google). But the model on the backend would be what determines that, hence the training data. If there is a tool out there that does it, honestly hiring a contractor maybe cheaper for the one off model build. I would recommend going forward though building some automation rules in your projects that do the categorization off of different fields

Best,
Clark

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
PRODUCT PLAN
PREMIUM
PERMISSIONS LEVEL
Site Admin
TAGS
AUG Leaders

Atlassian Community Events