Browser slow to show issue when large number of comments

[justindowning]

I have a project that has issues that take an inordinate amount of time to render themselves in a browser. For example, one issue has 80+ comments with varrying role visibility. When I go to browse the issue, it always takes 50+ seconds to render the issue. When I say render, I mean the browser is 'thinking' while showing a white page. When it's done 'thinking,' the entire issue is presented. I can reproduce this behavior (long load times) on other issues with similar traits--namely issues in the same project with a high number of comments.

This project uses Project Role security for comment visbiilty as well as an Issue Security Scheme limiting overal project visibility. I mention this because I have other projects in JIRA that do not exhibit this behavior. In fact, I'd say the majority of issues load in less than 2s usually.

The JIRA server is beefy (4 procs, 32GB RAM, SAS disks) and physically on the same LAN. I am suspicious of either (a) the project role/issue security setup or (b) the database connectivity. However, I don't want to unfairly point anyone in the wrong direction toward the best answer here. :)

Any help/guidance is appreciated! TIA!

[JamieA · Accepted solution]

The "security scheme" part of your question is the significant bit. If other issues are not slow you can rule out database and lucene as the issue.

My guess is that is that it's doing hundreds of user and group lookups to decide whether to show each comment... jira is not at all efficient when you have large numbers of groups or large numbers of users in a group. And rather than checking group membership once, it will be doing it hundreds or thousands of times for that page. Check the cpu while that page is rendering, and/or turn on the profiling to see if that shows anything.

[justindowning]

I'm glad you chimed in here Jamie! I read your post on improving JIRA performece (written for 3.x I believe) and am considering some of your recommended changes. The one suggestion in particular that jumps out is the elimination of the use of the jira-users group. I ran a profile thread dump on an issue with over 100 comments and noticed a high number of DB quereies for each comment. Have you utilized this suggestion in JIRA 4 and do you think it would apply to improving this situation (namely, the time used to lookup role permissions on every comment in a given issue)?

[AlexH]

There's an even easier way to tell if it's a slow index or a slow database:

1) JIRA's lucene indexes are used when performing filters.
2) Database queries are used when loading the issue page.

If your filters are fast, but your issue loads are slow that's almost a dead give away for database issues. If both are slow -- well then all bets are off the table.

[J. Caldwell]

There should be a atlassian-slow-queries.log file on your system somewhere. If you hit that specific page and then grab the log you should be able to look and see if it is a slow index or slow database.

[AlexH]

Sounds like you have a slow database...

You would also benefit from turning on log profiling and then loading one of those pages and see where JIRA is spending all of its time.

[justindowning]

Yeah, I did that. Here is the relevent information:

2011-07-22 11:29:38,737 TP-Processor964 DEBUG jdowning 688x7757088x3 fzkvm9 192.168.10.13 /browse/ABC-1234 [atlassian.util.profiling.UtilTimerStack] [44656ms] - /browse/ABC-1234

[44617ms] - QuickLinkServlet.service()

[5450ms] - Getting actions for com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel

And hundreds of lines like this (presumably for every comment):

[87ms] - Getting action HTML

[133ms] - DefaultProjectRoleManager.isUserInProjectRole

[0ms] - Iterator.hasNext()

[0ms] - Iterator.next()