Upgrade of JIRA to 7.8 - Tempo Timesheets are disabled on JIRA Server reboot

[Jiri Pik]

Is there any new feature in JIRA 7.8 which disables all addons not certified to work with 7.8? Pre-7.8, all such addons had a warning.

As of 7.8, Tempo Timesheets keep disabling themselves on reboot. 

[Vickey Palzor Lepcha · Accepted solution]

@Jiri Pik My timeout value is around 500 for timeout value and I've allocation almost 8gb for 200000 issues right now.

[Jiri Pik]

500 is the magic number. 300 is too low and 500 is just right.

THANK YOU.

[Vickey Palzor Lepcha]

@Jiri Pik I'd suggest you add the addin timeout param in your jvm argument. give plugins sufficient time to start . That should work.

[Jiri Pik]

Yeah, but what timeout do you have in mind?

JVM_SUPPORT_RECOMMENDED_ARGS="-Datlassian.plugins.enable.wait=300"

does not help

[Vickey Palzor Lepcha]

How foolish of me that I had forgotten to check my JVM arguments after upgrade.

Restored my argument back and it works fine now - plugin is up.

My jvm had plugin timeout params before - it was removed after upgrade. restored them back

[Jiri Pik]

@Andy Heinzer: I still do not understand why the sentenv.sh and server.xml are always removed on upgrade of JIRA / Confluence, unlike the config file with database connection strings. Yes, they are in different directories but the use cases are identical.

If the server.xml has a https URL in previous version, then it would most likely remain the same after upgrade!

[Vickey Palzor Lepcha]

Interestingly , Jiri and Andrew - I noticed this on my 7.8 jira server version as well.

[Vickey Palzor Lepcha]

Did you check that Tempo needs an update before taking it to 7.8 ? I had faced it - maybe that's why it was disabled.

Did you check Tempo had an update ?

[Jiri Pik]

Hi: yes, I have done and no help.

[Andy Heinzer]

There is not a feature to disable plugins based on any kind of certification.    However Jira will disable a plugin if that plugin fails to load repeatedly in Jira. 

From looking at the version history of this plugin, it appears that the vendor just released a new version of this plugin, 8.9.0, which appears to be the first version of this plugin that is noted as compatible with Jira 7.8.x.   Considering this version appears to have been released 1 day after you created this post, it feels like a safe bet you were running an older version of this plugin that wasn't really built for this version of Jira.

[Jiri Pik]

Hi:

The problem is that as of the latest JIRA Server Version this addin, unlike any other addin, has startup problems - see the logs below - and the Log analyzer points to 

https://confluence.atlassian.com/jirakb/jira-startup-fails-from-application-context-error-278073550.html?utm_medium=logScan&utm_source=STP

and

https://confluence.atlassian.com/jirakb/jira-applications-system-plugin-timeout-while-waiting-for-add-ons-to-enable-212173447.html?utm_medium=logScan&utm_source=STP

There is certainly NO CPU bottleneck. 

Could it be some Postgres connection string setting? 

What to do?

[Andy Heinzer]

When we usually see this kind of timeout, there are a pair of things I recommend:

1. You can increase the timeout Jira will wait for a plugin to start:

   It is possible to increase the plugin timeout period by adding the -Datlassian.plugins.enable.wait=300 argument to the JIRA application startup arguments, as in Setting Properties and Options on Startup.

   This will increase the time taken to start  the JIRA application up, and it does address the symptom rather than the root cause. If the machine is not powerful enough to run the application, we would suggest looking at upgrading to a more powerful host.

2. The other helpful action can be to increase the amount of memory that Jira is using.  Steps on how to do this are in Increasing Jira application memory.  As for how much to increase by, well that depends on your hardware, and the size of your Jira instance.  I recommend trying to keep in line with the Jira sizing guide.   If your install is under resourced, it can cause problems like this.  

Also sometimes when you upgrade Jira, I have seen the setenv.sh file, (or the windows service starting Jira) can become reset or overwritten.  In cases like this Jira tends to default back to the default memory settings.  Which works fine for a new install, but tends to be insufficient in environments with lots of issues, custom fields, plugins, etc.

[Jiri Pik]

Andrew:

I have added 

JVM_SUPPORT_RECOMMENDED_ARGS="-Datlassian.plugins.enable.wait=300"

but there is apparently another timeout 

[Andy Heinzer]

If you're still seeing a timeout with a wait time of 300 (seconds), then it feels like we need to allocate more resources here.   Which would be part #2 of my previous reply:

1. How much memory does this server have?
2. How much memory is allocated in the '-Xmx' parameter in the setenv.sh?
3. How many issues does your Jira instance have? 

[Jiri Pik]

500 is the right number. I have no idea why the default worked for the Tempo up to the previous version of JIRA.

The memory is 8 GB and all is under control.

Thanks, Andrew, for all your assistance.

Would it be possible to rethink the need to manually fix the following files after each update of JIRA (robots.txt, setenv.sh, server.conf)?

[Andy Heinzer]

Funny you should mention it, seems like there are existing feature requests for this such as in https://jira.atlassian.com/browse/JRASERVER-37621

Which appears to have been updated today.  At least from reading Gosia's comments on this, it does not appear that this feature will be getting attention any time soon though unfortunately.

I am glad that you got this working.  I don't want to beat a dead horse here, so if you are content with the solution you have now, stop reading here.

I still feel like waiting 500 seconds to enable this plugin is probably too long for most environments (maybe I'm just impatient).   I still believe the preferred way to address this problem is instead to focus on the performance of the server, and the allocation of memory to Jira in the Xmx parameter of your setenv.sh.

We still don't know what the Xmx value is for this environment, or how large this Jira installation is in terms of the Jira sizing guide.  I would also be interested to see your results of a Disk speed test for Java applications.

[Vickey Palzor Lepcha]

True Andrew - I am myself using HDD which isn't recommended anyway. So maybe if I am careless with recommendations - I should prepare myself to wait for 500seconds haha

[Vickey Palzor Lepcha]

In terms of retaining customization changes along with upgrade - I dont like that idea. The reason being making my changes again helps me understand the intricacies of my system.

I think my idea of my system stays intact - and it also helps me understand what can happen with every new updates and releases.

[Jiri Pik]

Well, some customization are permanent, like the URL of the reverse proxy server. 

For such customization I see no difference between this and the URL to the database server. 

So either both are customizations, or none. 

[Jiri Pik]

@Andy Heinzer: it was never 5 seconds until the latest release of JIRA. I have boosted Xmx params to these

JVM_MINIMUM_MEMORY="1024m"
JVM_MAXIMUM_MEMORY="5120m"

[Jiri Pik]

@Andy Heinzer: the server is compliant with the JIRA sizing guide, the disk is General SSD (AWS).