How is article effectiveness measured?

[Melissa Smith]

When looking at reporting our article effectiveness always says 0. 

In Atlassian documentation it states Article Effectiveness "Compares  knowledge base article  viewers with   request creators"

Can someone explain that measurement in more detail?

[Daniel Eads · Accepted solution]

Hey Melissa, welcome!

For the benefit of other people reading this question, just noting that it was tagged for JSD Cloud, but it looks like this question relates to Jira Server / Data Center. This feature functions differently in Cloud now than it does for Server (and if you're looking for the Cloud info, it's on this document).

So on Server, you're looking at the Article Effectiveness report. The two series on this graph are "problems solved by articles" and "requests created". In an ideal scenario, your graph would look something like this as your articles deflected more cases over time:

The specifics on how these series are generated:

1. Requests created is pretty straightforward - this is the number of issues created by people from the customer portal. The idea is that if your articles are effective in helping people solve their problems (self-help), they won't need to open as many requests.
2. Problems solved by articles is a bit murkier - as you noticed, it can be 0 for some customers. The numbers behind this are generated by people viewing KB articles in the customer portal, and then telling Service Desk that the article was helpful. In the screenshot below, you can see the "This helped me" link which would need to be clicked on to count in this series. 

What I've tended to notice in real usage is that people who find the article in the customer center may simply close the tab when they've read the solution. If this describes what your users are doing, then that would explain why the "problems solved" series isn't showing data.

This same data feeds into the Knowledge Base Usage report as well, where you can compare it against the raw views that the articles are getting.
Assuming the articles are getting views but nobody is clicking the "This helped me" button, there are a couple ways to use these reports:

1. Disregard the "people helped" series and try to make decisions based on the Views. This makes it a little more difficult to actually decide if the article content is helpful, but it does inform you if people are looking at them (and you can make some guesses based on how many requests they are creating anyway)
2. Re-train your users to actually click the "this helped me" link when giving them an overview of Jira or Service Desk. In the past, I've seen some success in getting better metrics if people know that clicking the link helps you make better decisions. Sometimes people simply don't see it, or they might not understand how clicking it helps you decide what articles to provide and how effective they are for deflecting requests.

Hope that helps clear things up!

Cheers,
Daniel

[Brett Cave]

Hi @Daniel Eads ,

Is the data for Article Effectiveness stored in Confluence data? Assuming there, and not on the service desk ticket.

I have a custom dashboard that I am looking to add a gadget to that shows article usage / effectiveness, much like the Service Project shows it, and trying to figure out how to query this / get it working - am trying to use JQL with a custom charts gadget, but no idea what sort of query I could use to get this info - any suggestions on this?

[Earl McCutcheon]

Hello @Brett Cave ,

Currently, there is not a supported option to call the CSAT data that is used by the article usage or article effectiveness data to use in a dashboard gadget.

We have the following feature requests to expand the functionality for both cloud and datacenter respectively noting the issue has a workaround listed using an experimental API endpoint that could be used as a stand-in that you could use to build out a custom dashboard gadget:

• Server/DataCenter:
  • Feature: https://jira.atlassian.com/browse/JSDSERVER-4799
  • Workaround: Add the possibility to get service desk reports using REST API
• Cloud:
  • Feature: https://jira.atlassian.com/browse/JSDCLOUD-4799
  • Workaround: Add the possibility to get service desk reports using REST API 
• Ecosystem (more API related features):
  • https://ecosystem.atlassian.net/browse/JSDECO-25

Next, On data Center versions only you could possibly use a SQL addon that has the gadget to call the desired data points directly from the DB to the gadget, a list of possible add-ons can be found here:

• https://marketplace.atlassian.com/search?category=Dashboard%20gadgets&hosting=dataCenter&product=jira&query=SQL

Then to get the desired content out the details of how the data is processed is that for the KB article usage report, we're specifically reporting on how many KB articles were viewed and how many were marked as solving a problem on the JSD Customer Portal (page views in regular Confluence are not tracked).

For the KB article effectiveness report, we're reporting on how many requests were created and how many articles were marked as solving a problem on the JSD customer portal.

The relevant statistics events are stored in the following database tables:

• AO_0201F0_STATS_EVENT - contains the occurrence of each KB page view, KB helpful and KB not helpful event
• AO_0201F0_STATS_EVENT_PARAM - contains the event parameters corresponding to each event stored in AO_0201F0_STATS_EVENT
• AO_0201F0_KB_VIEW_AGGR - contains the aggregated counts of KB page view events for each hour 
• AO_0201F0_KB_HELPFUL_AGGR - contains the aggregated counts of KB helpful events for each hour

Hope this helps.

Regards,
Earl

[Jack Brickey]

Hi Melissa, and welcome to the Community!

Don't take this to the bank but I believe that it is calculated based upon the number of times a customer comes to the portal to open a request but selects a KB article instead of creating an issue. Now I know that does not give you the algorithm and I too would like to understand this better so I will ping some Atlassian folks to see if they can get us all a better answer. :-)

[Melody KirkWagner]

This was my assumption as well - I'm not sure why Atlassian wouldn't assume that anyone accessing this sort of report on a ticketing portal would be looking for "ticket's avoided," but apparently they see it differently.