How to auto-restart Jira when JVM gives OnOutOfMemoryError ?

[Sorin Sbarnea (Citrix)]

Some told me that you can run commands when JVM runs out of memory but I am not sure if the below command will work or not as I do expect that this will only kill jira service, and not have time to start it again.

This has to go inside bin/setenv.sh file:

JVM_SUPPORT_RECOMMENDED_ARGS="-XX:OnOutOfMemoryError=\"service jira stop;service jira start\""

Please state if this will work and if not please suggest and alternative that works.

This command works just fine

java '-XX:OnOutOfMemoryError=nohup bash -c "service jira stop;service jira stop" &' test

Still, the trick of putting it inside the is more complex than it may seem, check http://stackoverflow.com/questions/11687292/how-to-do-a-proper-double-or-triple-bash-quote-escape

[Dave C]

Rather than addressing the symptom of the problem, I'd suggest addressing the root cause of why your JIRA instance is running into OutOfMemoryError's (OOME) in the first place.

Have you tried enabling GC logging or heap dumps on OOME to investigate further? You could do this by adding the following JVM arguments:

-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath= <JIRA_HOME>/log/
-verbose:gc
-Xloggc:<JIRA_HOME>/log/gc.log
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails

Every time JIRA is restarted, the GC log will be overwritten. Please ensure to manually rotate it (copy it to another name) before restarting JIRA.

You could then look at tools like GCViewer and Samurai to review your GC logs and heap dumps, respectively, to see if you can isolate the root cause. Likely problems may be third-party plugins or the box simply doesn't have enough resources for the JVM's requirements.

[Sorin Sbarnea (Citrix)]

The questions was not about how to debug jira and was only about keeping it running. This is required by an 3.x jira instance which is not supported by atlassian so debugging it doesn't help at all. We already tried this approach but what I want is a watchdog/keepalive tool.

[Martin Aparicio]

you might consider using a monitoring app like Nagios which can also restart JIRA or any service upon failure. I don't know if I would want my JVM to restart JIRA without me knowing. That said, I haven't run into out of memory issue since version 3. Good luck

[Radu Dumitriu]

Man, I had the impression that -XX:OnOutOfMemoryError appears only from7.x. Good to know.

java -XX:OnOutOfMemoryError=/opt/atlassian/script.sh, and place all your stuff inside.

service jira stop may not kill if for good. kill it manually with kill -9 if still alive

[Sorin Sbarnea (Citrix)]

Oracle docs reports it added to Java 1.4.2 Update 12 - http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

Still, there is the trick of killing jira, probably we need to add a nohup command too.

[Radu Dumitriu]

I think you should launch a process with & (i.e in background) that sleeps a bit, then kills -9 <pid>.