Recently we migrated from Server Version to Data Center Version.
We use VMWare Servers. So, we are using inbuilt Load balancer.
Configured 4 Nodes.
But access is very slow sometimes. Not able to reproduce always. But almost all users are facing issue.
Does someone can share the sample Load Balancer Configurations?
Hi Sivarama,
Atlassian doesn't publish a VMware sample - the official examples are only Apache mod_proxy_balancer and HAProxy, and the docs say they "do not officially support any specific Load Balancers, but most commonly see customers using Apache or F5". So there isn't really a file to copy. But whatever VMware gives you has to do four things:
1. Cookie-based session affinity on JSESSIONID. Not IP-based, not plain round robin. The docs are explicit about this one: "it is required to bind a session to the same node by enabling cookie-based sticky sessions (or session affinity)".
2. Health check against /status on each node, and check the response body, not just the status code. A healthy node returns {"state":"RUNNING"} with a 200. A plain TCP port check will cheerfully send users to a node that's still starting up or reindexing - we got caught by that one.
3. Graceful draining, so you can pull a node for maintenance without dropping everyone's session with it.
4. If you terminate SSL at the LB, pass X-Forwarded-For and X-Forwarded-Proto through, and make sure each node's Tomcat connector has the matching scheme / proxyName / proxyPort.
On the slowness though - "slow sometimes, not able to reproduce always, almost all users affected" on a cluster you've just built is nearly always point 1. When affinity isn't working, a user's requests land on a different node mid-session and they keep paying for session re-establishment. It looks random because it is.
You can rule it in or out in a couple of minutes without touching the LB config. Each node writes its own $JIRA_INSTALL/logs/access_log.yyyy-MM-dd, and the third field on every line is the username. Pick one user who's complaining and grep for them on all four nodes:
grep ' theusername ' access_log.$(date +%F) | tail -20
If their requests inside a single minute turn up on more than one node, affinity isn't working and that's your slowness. If they stay put on one node, affinity is fine and I'd go look at the shared home next - NFS latency there is the other usual suspect, and annoyingly it produces the same "random, everyone, can't reproduce it" pattern.
Hope that narrows it down.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.