An updated workflow for Server feature suggestions

[#1]

As mentioned at the end of last year, we have been working towards improving our feature suggestion process. Our efforts are driven by your need for increased clarity and transparency on feature suggestions.

We're pleased to announce that we now have an updated workflow in place. Each of your suggestions goes through a more structured process that provides greater clarity into where it stands in the context of our product roadmap. We're also using new statuses in the workflow to convey more significant meaning to you.

The new workflow

When you create a feature request, your suggestion automatically enters the Gathering Interest status, and your suggestion stays here until it's generated enough interest. Interest is measured based on several factors, such as the number of votes a request receives, and the UIS (User Impact Score) of the feature. UIS is automatically calculated, taking into consideration factors such as vote velocity and active users impacted. If your request hasn't generated enough interest over time, it will eventually be closed. With the large volume of suggestions that we receive, especially in Confluence and Jira Server, the new criteria will ensure the most popular suggestions are highlighted to be addressed sooner. 

Once a request has generated enough interest, it will move into the Reviewing state. In this state, an Atlassian Product Manager will start investigating the viability and relative priority of your suggestion, and then triage it into one of these statuses: Under Consideration, Future Consideration, Not Being Considered.

• • Under Consideration — Requests that are deemed to bring significant customer value and are likely a strong fit for our roadmap. In this state, there is a high likelihood of delivering the feature in the near future. To keep you in the loop, we will provide an update in the following few months. 

  • Future Consideration — Requests that are deemed a potential addition to our longer-term roadmap. Once a year, we'll reconsider the request and alter the status if needed.

  • Not Being Considered— Unfortunately, we're unable to implement all of the excellent suggestions you make. When this happens, the requests will take on the not being considered status. While we appreciate the potential benefits of such requests, we won't work on these for the foreseeable future. We'll review such requests after a year, and consider whether we need to alter the status. We understand that it may be disappointing having a suggestion you feel passionately about either be closed or triaged into not being considered. However, we truly believe that we owe you, our customers, greater transparency into the feature requests that you can expect to be implemented, and we believe this new workflow will help us accomplish that for you.

The final resolution of your suggestion may be Resolved: Fixed or Resolved: Won't Do. We may close your feature request as Resolved: Won't Do for a number of different reasons, such as the suggestion has not gained enough customer interest or the suggestion doesn't fit with the direction of our product for the time being. Alternatively, more popular, duplicate suggestions may already exist (we will close the feature request as Resolved: Duplicate) or the suggestion is not expressed clearly enough to be considered thoroughly by a product manager (Resolved: Invalid). 

Here's a summary of the new workflow (you can also select View Workflow in the issue status to visualize):

Note: We've implemented the new workflow for Confluence Server, Jira Server, Jira Service Desk Server and Bitbucket Server. 

Huge thanks for making us part of your team

These changes couldn't have been implemented without all of your insights. Also, do keep the feedback wheel turning — this helps us determine how best to improve customer transparency in our processes, how to convey product decisions more clearly, and how to ensure real-time updates on the status of feature requests created.

As always, we look forward to hearing more of your feedback in the years to come!

Cheers,

Keshav Puttaswamy

Head of Product - Server & Data Center, Atlassian

[Thomas Deiler]

Dear @KP,

thanks for increasing the transparency of your selection process for user suggestions. I can imagine that having the right things on the backlog  is not the easiest job. There will be always somebody who will complain.

I have one annotation to this voting in status "Gathering Interest":

Taking the pure votes and also the traffic of one suggestion into account, can give a false result. Imagine, that on some social media platforms or this community, a link to specific suggestion is located. Suddenly you have a huge activity and probably votes on this issue. Yes for sure - in further steps this issue will be sorted out.

But vice versa, imagine, an admin from a 5000 user instance creates a new suggestion. As this person understands the need of all his users best , the suggestion can be really good. Another one of the same business branch votes for this, also. Probably, you will not even notice this suggestion. It gets auto-closed.

What I wanna say, a vote is not equal to another vote. If you can merge the "quality" factor inside the progress, you will get better results. Probably taking the users activities in your network into account, like checking how many votes he/she gave for "quality" suggestions ... 

So long

Thomas

[Christopher Klinge]

I agree with @Thomas Deiler. In my opinion, it is impossible to determine if a suggestion is worth looking at based on an automated algorithm like you described. I wager, that one quick scan of the first few hundred suggestions will most likely bring up plenty of worthwhile ideas which do not have many votes. Keep in mind, that your JIRA is not some resource every JIRA admin around reads through regularly. Most of your customers stumble upon existing suggestions, because they are missing a certain feature themselves. But this leaves out suggestions covering topics which are not yet on anyones radar, but are worth looking at regardless.

I'd suggest you implement a transparent screening process in addition to your automated algorithm to catch those suggestions too. Yes, this will require some manual labor and make room for lots of discussions, but it is a workflow bearing fruit in many large scale Open Source projects (i.e. Kubernetes).

[Matt Doar]

The status changes should make it clearer to everyone what is the level of engagement for each issue in j.a.c.

I don't think they are intended to change how Atlassian decides what work on.

[miikhy]

That's a great step in favor of transparency. At least we'll now be sure to catch Atlassian's attention even if the backlog can't be customer tailored 100%. Sounds interesting in my opinion, can't wait to see the the outcome!

[KP]

Thanks @Thomas Deiler and @Christopher Klinge for your feedback.

 A couple of points worth clarifying:

1. We look through many more suggestions, in addition to the ones which make it to ‘reviewing’, to ensure we get a balanced understanding of your requests. The new process is in place so we’re transparent about when/if customers can expect an update on their suggestion. 

2. UIS takes into account many factors, including, for admin votes, the relative instance size.

We will continue iterating and making improvements as we go along, so your questions and suggestions are super helpful. Thank you!

[BillyP]

@Thomas Deiler I like your mention of how not all votes are equal.  Reminds me of the US House and Senate that organizes on total constituents as well as equal representation for each part of the whole.  I don't have a solution as both of these variables could change throughout the community, only remark that this comes up in history a lot.  

For the Atlassian's that track this, it would be a cool mention to see the features that come from the community itself.  It would be a neat lead in for the key note a cool summit conference... :)

Looking forward to making the awesome sauce more awesome! :)

[barnesy]

HI @BillyP, if you are able to make it along to the Team Tour, which is happening at a number of cities around the globe over the next couple of months, then you may be pleased to see us talking about some of the great community features which we have implemented recently.

Adam - Confluence PM

[Mark Thompson]

How can we honestly determine whether this is a good or bad thing without all the necessary criteria?

1.   How many votes does an issue need to generate interest?  Personally if you say 100 upvotes I'll probably never vote on anything.  Not that it does any good anyway since we had over 700 upvotes on another issue you chose not to fix.  

2.  You haven't told us how long an issue will remain open before you mark an item as Resolved: Won't Do.   I've seen people upvote issues that have been in your system for years.  It takes us a while to find issues especially if there are duplicates. 

3.  You haven't mentioned how you will move non duplicate votes. 

Example 1:  An issue comes in and doesn't get interest, you close it, the issue is opened again and gets insufficient votes, and you close it again. 

Example 2:  An issue is opened and gets some votes, a duplicate is opened and get some votes, the duplicate is closed and the votes die with it.

See where I'm going here.  You need a way to gather up all the unique votes and map them to a single issue.  You haven't mentioned how to do that.

4.  How do we veto you?  It bothers me that 700+ likely administrators upvoted an issue and you decided not to fix.  People posted they were unhappy about the decision and the company basically ignored their concerns.  I encounter the problem every week and every week I become less of a Jira enthusiast.  I'd like to see some method of disputing your decision which right now there isn't.  All we can do is squawk in a ticket.  I've worked at fortune 500 companies.  10 users complaining about an issue was enough to get us to act.  At what point is Jira guaranteed to act on customer votes? 

[Luke Hanson]

Does this new workflow get applied to the hundred and thousands of Feature suggestions in your backlog that have yet to be addressed despite having a huge number of votes?

[Wireball MacCarter]

By the way, that summary of the new workflow image appears to be hosted on https://extranet.atlassian.com/download/attachments/3686665358/JAC_suggestions_v2.png?version=2&modificationDate=1517280213943&api=v2 , which appears as a broken image link and doesn't show up to people on outside Internet connections such as myself.  "Firefox can’t find the server at extranet.atlassian.com."

[Guido Wischrop]

@KP you said:

UIS takes into account many factors, including, for admin votes, the relative instance size.

Is there any way to check, what my vote is worth for your rating? e.g. I might be admin for several instances. Do you count all of them? How do you connect my account to a particular instance?

[Jenny]

Hi all,

Thanks for all your great questions and for providing us with feedback.

In terms of the number of votes/UIS score a ticket need before it gets reviewed - right now this is flexible. We’re constantly tweaking this to ensure we can respond to a large number of suggestions, but in a way that’s scalable for our product managers. In that respect, this new process is being gradually applied to all existing open suggestions. @Guido Wischrop - I’m sorry to say there’s no way to get visibility on how your personal vote translates to UIS.

We don’t have a limit for how long an item could remain open for before it’s closed as ‘resolved: won’t do’, although we will be putting additional effort into ensuring the older issues have an increased level of transparency. I’d be interested to know though whether you think a limit should be put on this, and if so what it would be.

For duplicate suggestions, we’re currently closing any duplicate issues we find and directing people to vote on the original issue. I agree with you that this isn’t an optimal solution, but implementing an alternative solution at the moment would detract effort away from our new workflow. It’s certainly something we’ll be bearing in mind for future improvements though.

And we always welcome your feedback on these suggestions, even if we’ve closed them as ‘resolved: won’t do’. We can’t promise you’ll always be happy with the decisions we make, but we can promise that we’re listening to your feedback and aiming to provide you with increased transparency of what we’re working on.

Finally @Wireball MacCarter - could you try clearing your cache for us? Let us know if that still doesn’t work! 

Thanks

Jenny | Product Manager, Confluence Server 

[Wireball MacCarter]

@JennyThanks - no need, I see that it's been updated to https://tnckb94959.i.lithium.com since I commented, and is visible now.

Thanks for your updates too.  Regarding redirecting voting on duplicates to one issue, I don't suppose the Jira software has a way of "merging" votes at this time, but that might be an idea for considering (e.g. duplicate the vote table entries and just change the ticket number they're assigned to - _assuming_ that's the only table it's tracked in : )

[Denise Wermager]

Kudos on the new method of managing some of these issues but I do not think this will work for all issues. How accurate is the UIS when issues like the fix being asked for in JRASERVER-27957 occurs. The product end-user impact is huge. That can't be measured in the individual reports of admins that have voted on this issue. If I included every user at our site that was impacted when this occurred it would increase by about 5000.

Additionally, is anyone reviewing the issues to make sure they are classified properly. This is a design bug and fundamental flaw and should not be labeled as a suggestion. The fact that there is no confirmation on deletion of a custom field context that can affect every issue in your database in one fell swoop is a huge oversight in the design and implementation.

Our company pays for premium support and I sure would like to see it in action and see this issue fixed!

My guess is that due to the new process this isn't even being looked at - not significant votes, and UIS is low. Please don't rely solely on the new process as I think critical things like this are going to be missed and the impact to your users is extremely significant.

[Kasia Derenda]

@Denise Wermager, it is good to hear you like our new approach to communicating status of the Server feature suggestions.

User Impact Score (UIS) calculation is actually taking into account how many users are impacted by a given issue so the suggestion you mention should have a higher impact score if admins of affected instances voted.

Not all admins though are looking at suggestions but this is mitigated by collecting and validating customer challenges across a variety of sources. Forums like jira.atlassian.com and the Atlassian Community are key inputs in our prioritization, but there are also 1:1 conversations at Atlassian Summit, the Team Tour, AUGs or Partner events. Atlassian Partners provide valuable insights on their customer deployments. In-product feedback and analytics data provide yet another perspective. Last but not least, our customer facing teams, the support team and technical account managers are bringing customer challenges to our attention on a daily basis, including the suggestion you mentioned.

While we would like to triage and respond to as many suggestions as possible, the amount of feedback makes us prioritize the response to suggestions with the highest number of votes and UIS. We started with a relatively high UIS baseline and plan to lower it in the future. 

Katarzyna - Jira Server PM

[Daniel Eads {unmonitored account}]

We've been very appreciative of the new workflow both at my organization and at the Atlassian User Group that I lead. The increased transparency is very helpful in giving people a true feel for if the feature they're interested in is being worked on.

Are there plans for the rest of the product teams to switch over to this workflow? There have been several Jira Service Desk features we've been looking at in the past weeks and have pined for the new workflow there.

[KP]

Thanks @Daniel Eads {unmonitored account}. Glad to hear that the increased transparency and new workflow is helping.

As you know, we started with Jira and Confluence Server since they have the largest volume of suggestions and, arguably, the biggest need for a refresh.

Your feedback helps us as we look at whether we should roll these changes to other products in the future. We'll keep you posted.

[Malcolm Cook]

Recommend that you continue to allow votes to be collected after closing with "resolved - won't fix" especially in light of closing comments such as " I'm closing this and will re-open if demand changes." as appears in, e.g. https://jira.atlassian.com/browse/CONFSERVER-33245 for how better to assess change in demand than to track votes?

[Mark Thompson]

Thanks to Malcolm Cook for providing an example of item #2 in my post of FEB 13.  I anticipated the problem you mentioned as soon as I saw the process.  The people at Jira do bug tracking for a living but they don't seem to get what their own users may need.  I never close a valid issue in my system.  I move them to a component that means it will likely never be fixed but anyone searching for the issue will still find it, can upvote it, or make a comment.  It prevents people writing duplicates of an issue that has already been decided and allows people to give their input when conditions change.   I don't always have the power to do this in the tracking system but it is always my preferred method. People don't typically search for closed issues so the practice of closing valid issues is just wrong.  Not that you'll listen.

[Jenny]

@Malcolm Cook, @Mark Thompson, thanks for the feedback.

In the past, we did leave these suggestions open but one of the goals we set out in the new workflow was to be more transparent even if it isn't the answer some of our customers may want to hear. We felt that closing the issues was the best way to increase transparency and clearly set expectations. These closed issues can still be searched for and commented on, although as you mention, they can't be voted on. We don't take closing an issue lightly. We only do this when there is a severe lack of votes over a period of time and where we don't foresee an influx of voting on these issues.

[Mark Thompson]

What period of time?  Nowhere have I heard a specific time requirement.  Mentioned that right away

[Mark Thompson]

I think this statement says it all "even if it isn't the answer some of our customers may want to hear"

[Daniel Eads {unmonitored account}]

Says what all? I’m pretty sure SpaceX customers want to hear “your launch is free” and don’t want to hear that there are weather delays, etc. But that’s the nature of their business. The nature of software is that people will request features that can’t be built, either because of resources or because it doesn’t fit the mission of the project.

I don’t want to hear that my phone battery only lasts 12 hours, but that’s a trade-off with the phone size, current battery technology, and the cost associated with developing a better battery. If my neighbor wants their phone to run on cobalt batteries instead if lithium, he’s going to be disappointed. It’s better to know what to expect though, than to have suggestions go into the abyss where other people can’t see them.

[Malcolm Cook]

@Jenny - thanks for sharing Atlassian's reasoning in this regard.

If the count and rate of votes is the true driving decisions on which issues to close, in the interest of (a) more transparency, and (b) better customer service through continuous improvement, I recommend you consider to

• provide more details to you consumers/clients about how you count and rate actually drive your decisions
• explicitly encode in your resolution or close status when the reason for closing was lack of sufficient demand
• introduce the possibility of demand-based re-opening (based on count and rate of votes)
• allow for votes to be accepted on closed items
• adopt and announce a practice of periodic review of closed issues for re-open consideration (based on updated demand due to new votes).
• consider allocating a fixed number of votes to customers (feedly does this to good effect I think) - or maybe a number based on "cred" or "reputation"

In the meantime, I recommend editting https://jira.atlassian.com/browse/CONFSERVER-33245 to remove the remake "I'm closing this and will re-open if demand changes." since that does not appear to be how you are handling this issue.

Cheers,

Malcolm

[Mark Thompson]

@Daniel Eads {unmonitored account} - I've watched people complain on these forums for items that were reasonable.  700+ upvotes from likely JIRA administrators and the issue is not fixed, no response to genuine questions about criteria on the process so obviously 700 upvotes doesn't require JIRA to do anything so why waste the time to upvote, no answers to questions about failures or holes in the outlined process and the big one of the last UI redesign that made lots of people upset that can't be reversed unless you are on server.  Malcolm is not the first person to ask the questions he outlines. 

We are paying customers as well even if we aren't running server.  In this process I see little chance of getting a change so why even participate.  Issues need to be resolved with the closing/reopening/voting on issues.  I used to believe that an upvote meant something.  I don't anymore.