transactions using scripts result in timeouts

In JIRA 6.4 we have some transactions, that use scripted postfunctions or scripted values of customfields. From time to time (around once per day) the workflows stop working and the transactions result in a timeout.

For example: If a subtask fetches some information from the parent during creation, it will not finish.

To make it work again, we have to restart the complete JIRA isntallation.

Nothing is shown in the log. It seems, that the request does not reach the system at all. Network is not an issue as performing other steps at the same time from the same computer work.

Can anybody point me to somewhere to start to inverstigate. JIRA is not usable with this issue.

4 answers

This widget could not be displayed.

I think we need to establish the point of failure.  Are the scripts actually starting or not?  Could you insert a simple log.debug("Script XYZ has started") right at the top of the suspect scripts to prove that it is actually stopping within the scripts?  Also, I'd add one just before the exit point too.

The problem here is that "Nothing is shown in the log" means we don't know if it's failing as the scripts are running, before or after them.

No, they are not getting triggered. I have debug-output on different places in the scripts.

I get a timeout, when trying to execute a transition, that uses any scriptet function.

If trying to open such a particular transition in workflow designer, it will also run into a timeout and not open.

If adding a scripted function to an existing transition and trying to save it, it will also run into a timeout.

We have to restart the whole JIRA instance to make it work again. It will work for some time, but after some hours or more, it will stop working.

 

When the system is in such a state, no errors are shown in the logs concerning the scripts. It seems, the transition or the adding of the scripted function is never triggered.

 

We also experienced issues with background reindex. It seems, that triggering a reindex in the background makes JIRA unavailable for users. At least some functions. 

 

Good debugging info there, especially the look at the workflow editor and re-indexing. You've established that it's not the scripts themselves, but the attempt to actually open them - that seems to be stopping dead, and it's consistent across different methods of trying to access them. (The indexing looks at scripted fields, and the workflow designer obviously needs to check them when they are referenced in the workflow) So, it sounds like there's something killing the script-runner on any attempt at access. Problem is I don't know what it could be. Could you do a couple more things for us? * List the versions of JIRA, Java and the Script Runner in use - obviously, we should check the compatibility of all three, but as I suspect we're heading for raising a proper support call with Adaptavist, my colleagues are going to ask no matter what (You already said JIRA 6.4, but they'll probably ask for the point release as well - 6.4.??) * Check the resource usage the next time it freezes. Is the CPU or Memory on the server being thrashed? Ideally, the next time it happens, get a thread trace of what the process is doing.

This widget could not be displayed.

Jira is 6.4.9 build 64024
Java is 1.8.0_40
ScriptRunner Plugin is 4.1.1 (same behavior on 3.1.4)
I have a stack trace from one of the last crashes (happens around once per day)
The system is not having a very high load, when it comes to a crash.

This widget could not be displayed.

Sounds like a possible deadlock. But I don't think it's anything intrinsic in the plugin, there are plenty of large sites running many script functions all day.

You need to get a thread dump when the system becomes unresponsive or when the bad behaviour starts... how you do that varies depending on the server OS, but Atlassian have plenty of documentation on how.

> I have a stack trace from one of the last crashes (happens around once per day)

A crash seems different from what you're describing, but please give us the stack trace or create a request here: https://productsupport.adaptavist.com/servicedesk/customer/portal/3

 

This widget could not be displayed.

Firt of all, Sorry for not writing back earlier, as I had many things to finish in the last year. Now I can concentrate on the issue again.

Currently it seems, that it is not the script but other things, as it also occurs in other projects when creating subtasks.

We are fetching fields from parent task to the created subtasks in the subtask-workflow-create-function. It seems, that this fetching causes the problems. If some of the fields are set in the "new"-screen, it works. 

I changed my script to use another logfile, as we are having tons of entries in the logfile. As soon as it occurs again, I will give more information on that.

I also monitor the JIRA installation with jconsole. the memory rises up to 8GB (which is the limit) and when the garbage-collection starts, it goes down to 2,5GB. 

 

Suggest an answer

Log in or Sign up to answer
Community showcase
Posted Tuesday in Jira

What modern development practices are at the heart of how your team delivers software?

Hey Community mates! Claire here from the Software Product Marketing team. We all know software development changes rapidly, and it's often tough to keep up. But from our research, we've found the h...

178 views 1 3
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you