How to make Agile board planning loading to a decent performance level?

[Sorin Sbarnea (Citrix)]

Several JIRA Agile users reported serious performance problems when they have a big number of issues on their boards, some having like 10000 or even more.

In cases like this, even with topped hardware configuration on both JIRA server and the clients, it seems that loading the planning board or the current board can take 10-20 seconds and in some cases even timing out.

Clearly this is related to the number of issues, and we tried to lower their number by adding a filter like 

resolution IS EMPTY

Still, while this worked well for the planning, it did screw up all the sprint reports, and the current running sprint, because the DONE column became empty.

Update, over 95% of these issues are already resolved, long time before we started agile or during the last year(s) of Scrum planning.

Eliminating the resolved ones makes totally sense for Planning, but doesn't for for Work tab because they want to see what was done during current sprint and from Reporting they need to be included, otherwise the reports are broken (burn-down especially).

The problem is not the amount of issues being listed on the planning or work tabs, their number is quite managable, like under 200. The problem is that when the filter is broad the load is slow, even if it does show only 200 issues. If I make the filter to remove the resolved ones and return only the ~200 ones, the load time is good but the Reports are broken.

How to solve this problem?

Al this moment I am inclined to believe that this is a design bug with Agile planning mode, which, I am assuming, trying to send all the issues to the browser, even if they are resolved.

Also this problem became visible in the last week, right after we updated JIRA to 6.3 and Agile to 6.5, so this may be related to this.

Here are two issues that seem to address this problem:

https://jira.atlassian.com/browse/GHS-10976

https://jira.atlassian.com/browse/GHS-9920

Don't forget to vote/comment them.

[Nic Brough -Adaptavist-]

Mmm, I suspect the stock answer here is going to be "why have you got 10,000 issues on a board?".  It's completely beyond any human to handle that amount of data usefully, and your users are, in reality, only needing a subset of that 10,000, so that's what they should be working with.

[Chag]

I'm assuming they have work that was completed and never closed the tickets or someone created too many.

[Julia Wester [Wittified]]

I think Nic's point it valid though. When you have that much information on one board, you're not going to really be able to use the board effectively. 200 objects on the board will still take a while to load. I would think about breaking down the query farther.

[Sorin Sbarnea (Citrix)]

Guys, please read the entire question. If we made the filter to return only the current backlog (200) it will be damn fast but it will break all the historical reports. This question is not about dealing with 10.000 issues in planning mode, we have 10.000 issues because these is the number of issues that were resolved over time and that have to be present inside the reporting screens, so we can see the burn-down and velocity charts.

[Nic Brough -Adaptavist-]

We have read the question. What we're trying to tell you is that 10,000 issues is not a useful number to try to work with on a board, it's nonsense.

[Sorin Sbarnea (Citrix)]

I agree with you and in fact the 10.000 are never displayed. Still, they are needed because the REPORT tab would give incorrect reports if you remove them from the board filter. The problem is that the same board filter is used in all places planning, work and report. Report one needs the historical data.

[Julia Wester [Wittified]]

Sorin, I get what you are saying. It would be nice if you could have historical reporting captured somewhere that didn't require a load of all the of the data that goes into it.

[Alan Parkinson [Hindsight Software]]

You can filter out all the issues that were RESOLVED before switching to scrum by filtering on resolution date. 

Adjust the boards "Filter Query" and use the date of when you switched to scrum

project = "your project" AND (resolution is EMPTY OR (resolution is not EMPTY and resolutiondate > '2014/03/01' )) ORDER BY Rank ASC

In General Board use +1 for @Nic Brough [Adaptavist] answer.

I particularity wonder how people manage to rank 10,000 issues and most agile related books suggest keeping no more 90 days worth of items on the backlog. Don't be scared of removing items, if they are truly important they will be suggested at a later date.

We limit the number of issues on our boards by the following "work sub-filter"

(fixVersion in unreleasedVersions() OR fixVersion is EMPTY) AND (resolution is EMPTY OR resolution in (Done, Fixed))

On large projects we use "labels" to filter what goes onto the board. Usually label any issue with enough information from the business with the a "ready" label. The "work sub-filter" would look like this

(fixVersion in unreleasedVersions() OR fixVersion is EMPTY) AND (resolution is EMPTY OR resolution in (Done, Fixed)) AND labels in (ready)

[Alan Parkinson [Hindsight Software]]

@Sorin Sbarnea (Citrix) I've read your updated question "Update, over 95% of these issues are already resolved, long time before we started agile" Is there a reason why my amended answer using "Filter Query" wouldn't work considering 95% of your issues wouldn't be required for your report?

[Michael Mühlebach]

We have the same issue with a Scrum Board and would like to keep using the reporting functionality. The idea with the filter has, at least for us, no impact. Curiously recreating everything has a huge positive effect for some time.

[MattS]

For Kanban boards I choose to not show issues that have been closed for more than a few weeks, e.g. subfilter:

status not in (closed,) or resolved > -2w

and I recommend our customers don't have more than 500 issues in a board, max 1K issues

[Timothy]

Clearly this is related to the number of issues, and we tried to lower their number by adding a filter like 
resolution IS EMPTY

Then find a different filter that would work for you. I mean, that is not the only thing you can use.

Another thing that you need to take note. JIRA Agile is also very front end heavy in the sense that the browser does alot of the heavy lifting to render the page. A better machine for them will also help.

[Shannon Davis]

We had a Kanban board with 800 issues that was performing very slowly.  Reading this thread was helpful because it helped me remember that the purpose of the Kanban board isn't to see every issue we've ever had, it's the plan the next release.  So, we added a sub-filter to the board to only show Priority Major and above.  After all, if it were not a high priority, why would we spend time on it?  that cut the number down to 400 issues and it is performing much faster.

[John Bayne]

We too are aware of this issue. Our new projects are starting afresh and using a wide open project filter and so will eventually see the degradation in performance if they maintain this.

Import an old project from bugzilla with several thousand issues and we see it immediately.

Thinking on filters I'm considering that I should make 'Affect version/s' mandatory for every issue created on the system. Then I can use a subset of versions (current/head and those active in the field) as a sliding window on the backlog.

Comments?

How to Atlassian manage this internally I'm tempted to ask?

John

[Sergey Svishchev]

GHS-9611 is probably the reason.

[Alan Parkinson [Hindsight Software]]

Even if it is a regression within JIRA Agile, improving your projects JQL filters should help general performance. I've updated my answer to show how to filter out issues before a certain date

[Nic Brough -Adaptavist-]

They threw more memory at their problem, but we don't know the size of the their boards. I'd guess if memory fixed it, they were still boards with only a few issues on it. Memory may help a bit with 10,000 issues, but I doubt it's going to fix it completely.

[Sorin Sbarnea (Citrix)]

Similar to https://answers.atlassian.com/questions/75556/agile-planning-board-is-slow-in-certain-project

[Alan Parkinson [Hindsight Software]]

@Sorin Sbarnea (Citrix) How many issues in the resolved state do you need to be available for the work and reports features to work? Do just need the current sprint or the last 2 weeks?

[Nic Brough -Adaptavist-]

But the point remains - 10,000 (or even 9,800) issues is quite simply a silly amount to try to handle on a single board. Your humans can't do anything useful with them. Break them down, find the ones that matter, get them filtered to the top of the list and ignore the rest. If your team can handle say 100 issues at a guess, then pick out the top 300, to allow for some flexibility and see how they're doing at the end of the first sprint. Break the issues into many teams if you have several teams, use labels, fields, anything, just break up the list into something manageable. 10,000 is simply not useful on boards.

[Sorin Sbarnea (Citrix)]

To those questioning the number of issues, from those 10.000, only ~200 are Unresolved, so visible in the planning mode. The problem is that by making the filter narrow, it will solve the planning load time, but it will break the Work and/or Reporting ones.