Severe performance issues with Jira 6.4.. AjaxIssueAction takes forever. anyone else had issues?

[Jeff Louwerse]

I know this is a shot in the dark but..

long story short, after upgrading to JIRA 6.4 ( a second time) with Scriptrunner 3.1.4 I have severe performance issues with AjaxIssueAction often taking 30-50 seconds.  There are no errors or anything being logged so I have zero idea where to start.  we rolled back the last attempt in October for this very reason. 

In many cases if I perform a transition not requiring a transition screen with a right click and say "open in new tab" it comes back in 2 seconds.  If I do it the normal way and simply hit the refresh button, it comes back updated in 2 seconds.  If I click the edit button or a transition the normal way.. it might come back in 30 seconds or might just tell me "an unknown error occurred"

Atlassian pointed to Scriptrunner as the issue but I can't find any issues.  all the post functions and listeners complete in about 1 second.. then there is a long pause..

I have has anyone else seen any issues where AjaxIssueAction?  any idea where to start?

[Jeff Louwerse]

The issue has been resolved and it turns out I was sent on a snipe hunt as the problem had noting to do with scriptrunner.    The support from Jame at Adaptavist  was above and beyond what I have come to expect with many plugin vendors... offering ideas and help to fix my problem when the issue wasn't even with their plugin.

[Doug Swartz]

Memory? Garbage collection? ...?

[Jeff Louwerse]

old SQL query referencing pkey (which is now null) on a custom field plugin.    This generated a full table scan every time the field was loaded (as jamie pointed out this is an oracle issue on null fields)... add that we have 5 fields like this.. and at any point in time 200 users on the system it created a log jam on the connection pool. 

It didn't produce any output/debugging info until I actually change the field in the UI.  and it wasn't noticeable in all the testing as there were only a handful of people testing it. again had I used the plugin you recommended I probably would have seen this sooner!

I also did other tweaks to the memory sizing as recommended by Atlassian .. basically bringing the max and the min closer to each other.

[JamieA]

Thanks for your comments Jeff, much appreciated.

[Doug Swartz]

If your problem is in Scriptrunner, that means it's most likely in your script code, not in Scriptrunner itself. .

Do you have the Java Melody monitoring plugin installed? That plugin has a feature that will let you see the java call stack of a running transaction. I've used it several times to find slowness problems in scripts and our own plugins. Note: Java debugging experience likely required.

[Jeff Louwerse]

I had assumed it was my scripts as well so I redid them all.  I added logging to them all and they all seem to complete withing a second.   

I do not have that plugin installed and don't have a log of Java debugging experience myself but I will give it a shot and muddle my way though it.

[Jeff Louwerse]

Thanks for the Java Melody idea.. With all that happened yesterday I did not have a chance to investigate it but Jamie also mentioned that it probably would have helped pinpoint my problem a lot faster.    On my bucket list of things to do once all the dust settles.

[Dan Deftos]

Would you mind sharing a little of what the problem was? I see that at times when clicking on an issue (either from a JQL generated list or from the "breadcrumbs" at the top of an issue, it can take up to 10 seconds for the issue to appear in the browser. In looking at what is going on I see a lot of scriptrunner calls. As this is not a transition I would not expect scriptrunner to be invoked.

Thanks

[Jeff Louwerse]

You will probably see SR calls if you have any workflow conditions or any calculated/scripted fields.  Also if you have behaviours enabled there will be at least one call to disable inline edit.

my problem was 100% my own fault and absolutely nothing to do with ScriptRunner.

[Jeff Louwerse]

One thing you might want to look at is your database and the state of its indexing.  I routinely force mine to be re-analyzed and on occasion force them to completely rebuild.