Jira datacenter- Can't login after a database failover or backup xml restore

Divya TV June 11, 2021

Hello Jira users, 

Wanted to share my experience with one particular behavior of Jira that I have encountered multiple times and was able to fix with the same steps.

Problem: 

After a database failover(due to production database crash or not an unplanned failover), users even Jira-administrators, can't log into Jira. Error: Sorry username or password incorrect.

This error also occurs when you restore from backup xml from one Jira instance to another(where the cluster nodes are different)

Solution:

Check clusternode database table. When you restore from backup XML, it includes the entries of  clusternode table which has hostnames and status(OFFLINE/ACTIVE) of the source Jira instance. They do not match the cluster or node hostnames on the destination Jira instance.

select * from clusternode

When you can't log in: 

NODEID, STATUS, TIMESTAMP, IP, CACHE_LISTENER_PORT, NODE_BUILD_NUMBER, NODE_VERSION

node1 OFFLINE 1621274609792 destinationnode01  40001 805004 8.5.4
node2 ACTIVE 1622135262959   sourcenodejiradc01 40001 805004 8.5.4

What to rectify?

  •  Make sure the status of the node match the actual status in Jira cluster. If a node says active, it needs to be up and running in the cluster. If its not up and running change the status to OFFLINE.
  • Make sure the IP matches to that of the destination Jira instance(or where you are hosted). Remove the ones which show source node hostnames or different hosts not belonging to the cluster.
  • During an unplanned failover, the correct status of clusternodes don't copy over correctly, in that case the hostnames might be correct but check their status.

You can use simple update SQLs to do that.

Restart your Jira instance or in some cases no need to restart as well, you will be able to log into Jira.

When you CAN log in, the entries must look this way:

NODEID, STATUS, TIMESTAMP, IP, CACHE_LISTENER_PORT, NODE_BUILD_NUMBER, NODE_VERSION

node1 OFFLINE 1621274609792 destinationnode01  40001 805004 8.5.4
node2 ACTIVE 1622135262959  destinationnode02  40001 805004 8.5.4

node2 or destinationnode02  - is up and running .

node1 or destinationnode01 - is not yet started.

 

Please let me know if you have tried other ways to solve this mystery.

Thank you for reading.

Regards,

Divya TV

 

 

 

0 comments

Comment

Log in or Sign up to comment
TAGS
AUG Leaders

Atlassian Community Events