Classify requests in Jira Service Management

[#1]

We often implement Jira Service Management (JSM) as a service desk solution to most of our customers and we clearly see from year to year the work of their dev and support teams is becoming more and more complex. They are now working on more diverse groups of issues and their number is growing rapidly.

For example, one of our customers — a US-based fintech company — manages their projects in JSM, and various users (e.g. colleagues from banks, law enforcement institutions, lawyers, etc) can create information requests in it. With hundreds of daily requests, the team is drowning. Moreover, due to the nature of our customer's business, they are to check each request's description to identify sensitive information and adjust issue security level accordingly.

We looked at a way to help the customer classify requests based on their title, description, or other parameters. We decided to implement natural language processing (NLP). This project is a proof of concept on how to build a machine learning (ML) model for JSM requests classification. The project helped us gain strong learning outcomes we would like to share with you.

Step 1. Tools selection

Before starting the implementation, choose ML tools that support NLP. We have analyzed ready ML solutions such as AWS Comprehend and Google AutoML. After further consideration, we decided to go with a custom solution. For that purpose, we recommend spaCy — a free production-ready Python library with NLP capabilities — based on positive reviews and a high rating. Besides, spaCy has the same functionality as the paid tools mentioned above.

Step 2. Model training

Fortunately, spaCy makes creating a text classification model relatively easy as it has a set of pre-trained models based on large amounts of information. Therefore, you do not need to collect a large dataset and a limited amount is enough to tweak the parameters to work. 

If your tickets are already labeled (based on the field value selected for the issue), all you need to do is to 

• backup Jira instance to get the whole dataset, 
• parse .xml file to get the list of issue descriptions and their classifications, 
• transform the file into .csv — a format that spaCy understands. 

Then, upload this file into spaCy and after a while, spaCy will send the final model file, which can be re-used elsewhere.

Step 3. Model usage

To start using spaCy for classifying JSM issues based on their descriptions, create an API endpoint that accepts text (in our case, it is the description of the ticket in JSM) and passes this text to spaCy. Based on examples the model has seen during training, it decides whether the text in the description is a named entity or not.

Then, set up an Automation rule in Jira itself. It is triggered when the issue is created and has the “Information Request” request type. The Automation rule sends an API request to our external Python-based application with ML model passing the issue description as a payload via smart value:

It is important to use the json_encode function while sending the issue’s description. Otherwise, in most cases, you will not get a valid JSON due to line breaks and other special characters.

Then, the Automation rule sends a web request. Once the Automation rule receives a response from API and sees that the classification value is equal to a named entity, it edits the Jira issue’s Assignment Group, Priority, and ML Classified Type. 

The request before the model intervention:

The request after the model intervention:

Of course, it is in no way a replacement for human review for various reasons, including the fact that the training dataset is limited, but it is still a nice addition to the triage process that we have in place.

If you also created a model for classifying information in JSM, what experience did you have (What instruments did you use? What pitfalls did you face)?

Share your ideas in the comments!