JIRA Datacenter can't install/enable/disable plugins

Matt Johnson November 11, 2014

I've got a JIRA Datacenter cluster up and running, and everything seems to be fine except for trying to install, enable, or disable plugins. The setup is currently using Pound as the load balancer, with two JIRA application servers running JIRA Datacenter, pointing to a single backend DB/Storage server. This is just a lab setup so I can test our production environment to see if we want to move to JIRA/Confluence Datacenter edition. 

The issue I'm seeing is that when I access through the load balancer everything seems to operate normally, except for dealing with plugins. If I try to install, enable, or disable a plugin I get the following error:

The JIRA server was contacted but has returned an error response. We are unsure of the result of this operation.

I've checked the JIRA logs and there are no entries that point to where to even start looking. 

Logging directly in to either JIRA instance I can install/enable/disable without issue. And after disabling on node1, I can log on to node2 and see that it is disabled, and then enable. 

This seems to point to an issue with the load balancer, but I'm not sure what to check, as all other functions seem to work without issue. Does anybody have experience with something like this?

2 answers

1 accepted

1 vote
Answer accepted
crf
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 11, 2014

I would suspect that you haven't properly configured session affinity.  If the plugin is uploaded to node1 but then it hits node2 before node1 is done activating the plugin, then node2 won't know anything about it yet and the request would fail.

Long term, we hope to eliminate this session affinity requirement, but it is necessary for the time being.

Matt Johnson November 12, 2014

If session affinity were the issue would it be reasonable to assume that taking down node2, having just node1 up, and then trying to enable/disable would be a valid test? I've done that, and see the same issue with just node1 being active. I'm admittedly new to this type of load balancing, so it's entirely possible I've done something wrong. I'll keep testing, but if you or anybody else has some ideas I'd appreciate any input. MJ

crf
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 12, 2014

You are correct. Is JIRA's base URL setting pointing at the load balancer? That's another thing I could see causing problems. Otherwise I would use a browser like Chrome that has good network traffic logging in its web development tools and use that to see where the error is coming from. For reference, I'm on JIRA's Enterprise development team (the one that created Data Center) and this is not a problem I've heard reported elsewhere.

Matt Johnson November 17, 2014

I've done some troubleshooting using Chrome's developer tools, and I can see that it's getting a 501 (Not Implemented) when trying to modify a plugin through the load balancer. I've compared the session from the load balancer to a session directly to node1, and the only differences I can see are that the load balancer seems to be passing additional cookies that the session to node1 one is not, and the load balancer gets a 501 where node1 gets a 200. The one that works passes a JSESSIONID and atlassian.xsrf.token The one that does not work passes a JSESSIONID, atlassian.xsrf.token, seraph.rememberme.cookie, and ump.atl.id The base URL is pointing to the load balancer, however I've tried pointing it directly at node1 and it shows the same behavior. I know you've never seen this before, but do you have any other ideas of what to look for? Or perhaps a recommendation for a load balancer other than Pound? I'm open to trying something else. Thank you, MJ

crf
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 17, 2014

We did all of our internal testing using apache as the front end and give a sample configuration here: https://confluence.atlassian.com/display/JIRA/Installing+JIRA+Data+Center

0 votes
Matt Johnson November 17, 2014

Just an update. I've switched over to HAProxy for load balancing and that seems to have fixed the issue I was seeing. I'm still not sure why Pound was having trouble with only the UPM and modifying add-ons.

Thanks for the help.

MJ

Suggest an answer

Log in or Sign up to answer