Slow loading of Issue Navigator / Query Component

[Paweł Kowalik]

Hello,

I'm trying to find out what might be causing long response times for Issue Navigator (/secure/IssueNavAction!default.jspa) and Query Component (/secure/QueryComponent!Jql.jspa). Even when there is no traffic on Jira, these requests can take about 10 seconds and if more of them happen at the same time, they can slow each other up (and other requests as well) to even 30 seconds or more.

If I understand correctly, these components are responsible for rendering panel with all the options you can use when performing a JQL search (projects, statuses, issue types, custom fields etc.). I've tried to find some information with profiling tools, but this is all I got - nothing interesting after or before these lines:

[12512.8ms] - QueryAction.execute()
[9435.7ms] - IssueNavAction.execute()

A little bit more information can be found with JavaMelody plugin after hovering over request:

org.apache.velocity.runtime.VelocimacroFactory.getVelocimacro(VelocimacroFactory.java:571)
org.apache.velocity.runtime.RuntimeInstance.getVelocimacro(RuntimeInstance.java:1562)
org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:199)
org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)
org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
org.apache.velocity.runtime.directive.Foreach.performIteration(Foreach.java:393)
org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:316)
org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)
org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
org.apache.velocity.Template.merge(Template.java:328)
org.apache.velocity.Template.merge(Template.java:235)
org.apache.velocity.app.VelocityEngine.mergeTemplate(VelocityEngine.java:381)
com.atlassian.velocity.DefaultVelocityManager.writeEncodedBodyImpl(DefaultVelocityManager.java:117)
com.atlassian.velocity.DefaultVelocityManager.writeEncodedBody(DefaultVelocityManager.java:97)
com.atlassian.jira.template.velocity.DefaultVelocityTemplatingEngine$DefaultRenderRequest.toWriterImpl(DefaultVelocityTemplatingEngine.java:122)
com.atlassian.jira.template.velocity.DefaultVelocityTemplatingEngine$DefaultRenderRequest.asHtml(DefaultVelocityTemplatingEngine.java:113)
com.atlassian.jira.template.velocity.DefaultVelocityTemplatingEngine$DefaultRenderRequest$2.with(DefaultVelocityTemplatingEngine.java:101)
com.atlassian.jira.template.velocity.DefaultVelocityTemplatingEngine$DefaultRenderRequest$StringRepresentation.toString(DefaultVelocityTemplatingEngine.java:77)
com.atlassian.jira.template.velocity.DefaultVelocityTemplatingEngine$DefaultRenderRequest.asHtml(DefaultVelocityTemplatingEngine.java:103)
com.atlassian.jira.issue.search.searchers.renderer.AbstractSearchRenderer.renderEditTemplate(AbstractSearchRenderer.java:98)
com.atlassian.jira.issue.search.searchers.renderer.IssueTypeSearchRenderer.getEditHtml(IssueTypeSearchRenderer.java:160)
com.atlassian.jira.components.query.DefaultSearcherService.getEditHtml(DefaultSearcherService.java:440)
com.atlassian.jira.components.query.DefaultSearcherService.getValueResults(DefaultSearcherService.java:261)
com.atlassian.jira.components.query.DefaultSearcherService.getSearchResults(DefaultSearcherService.java:196)
com.atlassian.jira.components.query.DefaultSearcherService.searchWithJql(DefaultSearcherService.java:169)
...
Suppressed some not interesting lines...
...
com.sun.proxy.$Proxy3789.searchWithJql(Unknown Source)
com.atlassian.jira.plugin.issuenav.IssueNavAction.performSearch(IssueNavAction.java:1104)
com.atlassian.jira.plugin.issuenav.IssueNavAction.doNav(IssueNavAction.java:423)
com.atlassian.jira.plugin.issuenav.IssueNavAction.doDefault(IssueNavAction.java:282
...
Suppressed some not interesting lines...
...

This is where it always hangs - VelocimacroFactory.getVelocimacro - and the only thing that changes here is the com.atlassian.jira.issue.search.searchers.renderer line depending on what is being rendered at the moment (issue types, projects, issue constants).

There are no any spikes on CPU, memory, disk and the JQLs themselves are fast (once the page where you can type them is loaded), so what can be causing this problem?

Our instance - Jira Software 8.14.0:
Attachments 621086
Comments 3258773
Components 3249
Custom Fields 533
Groups 885
Issue Security Levels 166
Issue Types 105
Issues 644407
Priorities 18
Projects 484
Resolutions 15
Screen Schemes 583
Screens 1320
Statuses 329
Users 9756
Versions 27246
Workflows 466

[Paweł Kowalik · Accepted solution]

Ok, I think I've found a solution:

In <jira-install>/atlassian-jira/WEB-INF/classes/velocity.properties the following lines were uncommented:

class.resource.loader.cache=false
velocimacro.library.autoreload=true

It was probably in order to edit notification templates without having to restart Jira. After commenting them out again and restarting Jira, the problem seems to be gone.