Jira and Confluence does not startup after linux system reboot.

[Deleted user]

I have Jira, Confluence and BitBucket  running each on own standalone Centos7 linux VM servers (latest version) . Every-time i reboot VM servers (patching linux or rebooting KVM server)  the Jira and Confluence will not startup.  To fix it,  i have to manually restart Jira and Confluence.  This happening past last year. Before i did not have this problem.   

This is what i have to execute every single time after reboot VMserver to start jira

/opt/atlassian/jira/bin/shutdown.sh
/opt/atlassian/jira/bin/startup.sh

It look like i have problem with some old PID!!!!! 

/var/log/messages 

ay 16 12:27:35 s-jira jira: To run JIRA in the foreground, start the server with start-jira.sh -fg
May 16 12:27:35 s-jira xrdp[3928]: (3928)(140593122466240)[INFO ] listening to port 3389 on 0.0.0.0
May 16 12:27:35 s-jira systemd: Started Command Scheduler.
May 16 12:27:35 s-jira systemd: Starting Wait for Plymouth Boot Screen to Quit...
May 16 12:27:35 s-jira jira: executing using dedicated user: jira
May 16 12:27:35 s-jira postmaster: < 2019-05-16 12:27:35.861 MDT > LOG: redirecting log output to logging collector process
May 16 12:27:35 s-jira postmaster: < 2019-05-16 12:27:35.861 MDT > HINT: Future log output will appear in directory "pg_log".
May 16 12:27:35 s-jira systemd: Started GNOME Display Manager.
May 16 12:27:35 s-jira jira: `sMMMMMMMMMMMMMM+
May 16 12:27:35 s-jira jira: MMMMMMMMMMMMMM
May 16 12:27:35 s-jira jira: :sdMMMMMMMMMMM
May 16 12:27:35 s-jira jira: MMMMMM
May 16 12:27:35 s-jira jira: `sMMMMMMMMMMMMMM+ MMMMMM
May 16 12:27:35 s-jira jira: MMMMMMMMMMMMMM +MMMMM
May 16 12:27:35 s-jira jira: :sMMMMMMMMMMM MMMMM
May 16 12:27:35 s-jira jira: MMMMMM `UOJ
May 16 12:27:35 s-jira jira: `sMMMMMMMMMMMMM+ MMMMMM
May 16 12:27:35 s-jira jira: MMMMMMMMMMMMMM +MMMMM
May 16 12:27:35 s-jira jira: :sdMMMMMMMMMM MMMMM
May 16 12:27:35 s-jira jira: MMMMMM `UOJ
May 16 12:27:35 s-jira jira: MMMMMM
May 16 12:27:35 s-jira jira: +MMMMM
May 16 12:27:35 s-jira jira: MMMMM
May 16 12:27:35 s-jira jira: `UOJ
May 16 12:27:35 s-jira jira: Atlassian Jira
May 16 12:27:35 s-jira jira: Version : 8.1.0
May 16 12:27:35 s-jira systemd: Started PostgreSQL 9.6 database server.
May 16 12:27:35 s-jira jira: If you encounter issues starting or stopping JIRA, please see the Troubleshooting guide at https://docs.atlassian.com/jira/jadm-docs-081/Troubleshooting+installation
May 16 12:27:35 s-jira jira: Server startup logs are located in /opt/atlassian/jira/logs/catalina.out
May 16 12:27:35 s-jira systemd: Started Virtualization daemon.
May 16 12:27:36 s-jira jira: Existing PID file found during start.
May 16 12:27:36 s-jira jira: Unable to read PID file. Start aborted.
May 16 12:27:36 s-jira systemd: jira.service: control process exited, code=exited status=1
May 16 12:27:36 s-jira systemd: Failed to start (null).
May 16 12:27:36 s-jira systemd: Unit jira.service entered failed state.
May 16 12:27:36 s-jira systemd: jira.service failed.

[Deleted user · Accepted solution]

OK, I fixed it. I found solution. User permissions in Linux were not set correctly on the files.

https://community.atlassian.com/t5/Confluence-questions/Confluence-not-start-up/qaq-p/760312

The fix in general is to run

for Confluence

chown -R confluence.confluence /opt/atlassian/confluence/
chown -R confluence.confluence /var/atlassian/application-data/confluence/
chmod -R u=rwX,go-rwx /opt/atlassian/confluence/
chmod -R u=rwX,go-rwx /var/atlassian/application-data/confluence/

or

for Jira 

chown -R jira.jira /opt/atlassian/jira/
chown -R jira.jira /var/atlassian/application-data/jira/
chmod -R u=rwX,go-rwx /opt/atlassian/jira/
chmod -R u=rwX,go-rwx /var/atlassian/application-data/jira/

and its working

[Mike Rathwell]

hi @[deleted] 

A bit of clarification, please... sounds like you are rebooting your VM host in this case and not the actual VMs that Jira and Confluence are running in. (let me know if I misunderstood that)

In this case, it's not necessarily a problem with Jira or Confluence. Basically what is happening in this case is they're simply getting their feet kicked out from under them and go down hard rather than gracefully. Aside from the pid file still lurking about that can be uber bad juju for the indexes, database, etc.

You might need to look at the restart of the VM host so that it waits for the graceful shutdown of both of those before it kills itself. Likely then it'll go smooth.

[Deleted user]

This is what i will execute when I'm  updating  Jira or Conflucen VM (Centos7) 

sudo yum -y update
sudo reboot

or 

sudo yum -y update
sudo systemctl reboot

after reboot

sudo rpm -q kernel
sudo package-cleanup --oldkernels --count=1 -y

after that jira or confluence will not start. The only way i can restart jira or confluence is to use 

/opt/atlassian/jira/bin/shutdown.sh
/opt/atlassian/jira/bin/startup.sh

deleting PID file and starting jira with "systemctl start jira" will not startup jira

rm /opt/atlassian/jira/work/catalina.pid
systemctl start jira

[Deleted user]

Actually after any reboot 

sudo reboot

or 

sudo systemctl reboot

Jira or Confluence will not startup!!!

[Deleted user]

These are jira errors after VM reboot.

May 17 00:00:31 s-jira jira: To run JIRA in the foreground, start the server with start-jira.sh -fg
May 17 00:00:31 s-jira jira: executing using dedicated user: jira
May 17 00:00:31 s-jira systemd: Started GNOME Display Manager.
May 17 00:00:31 s-jira postmaster: < 2019-05-17 00:00:31.809 MDT > LOG: redirecting log output to logging collector process
May 17 00:00:31 s-jira postmaster: < 2019-05-17 00:00:31.809 MDT > HINT: Future log output will appear in directory "pg_log".
May 17 00:00:31 s-jira systemd: Started PostgreSQL 9.6 database server.
May 17 00:00:31 s-jira jira: `sMMMMMMMMMMMMMM+
May 17 00:00:31 s-jira jira: MMMMMMMMMMMMMM
May 17 00:00:31 s-jira jira: :sdMMMMMMMMMMM
May 17 00:00:31 s-jira jira: MMMMMM
May 17 00:00:31 s-jira jira: `sMMMMMMMMMMMMMM+ MMMMMM
May 17 00:00:31 s-jira jira: MMMMMMMMMMMMMM +MMMMM
May 17 00:00:31 s-jira jira: :sMMMMMMMMMMM MMMMM
May 17 00:00:31 s-jira jira: MMMMMM `UOJ
May 17 00:00:31 s-jira jira: `sMMMMMMMMMMMMM+ MMMMMM
May 17 00:00:31 s-jira jira: MMMMMMMMMMMMMM +MMMMM
May 17 00:00:31 s-jira jira: :sdMMMMMMMMMM MMMMM
May 17 00:00:31 s-jira jira: MMMMMM `UOJ
May 17 00:00:31 s-jira jira: MMMMMM
May 17 00:00:31 s-jira jira: +MMMMM
May 17 00:00:31 s-jira jira: MMMMM
May 17 00:00:31 s-jira jira: `UOJ
May 17 00:00:31 s-jira jira: Atlassian Jira
May 17 00:00:31 s-jira jira: Version : 8.1.0
May 17 00:00:31 s-jira jira: If you encounter issues starting or stopping JIRA, please see the Troubleshooting guide at https://docs.atlassian.com/jira/jadm-docs-081/Troubleshooting+installation
May 17 00:00:31 s-jira jira: Server startup logs are located in /opt/atlassian/jira/logs/catalina.out
May 17 00:00:31 s-jira systemd: Started Virtualization daemon.
May 17 00:00:31 s-jira jira: Existing PID file found during start.
May 17 00:00:31 s-jira jira: Unable to read PID file. Start aborted.
May 17 00:00:31 s-jira systemd: jira.service: control process exited, code=exited status=1
May 17 00:00:31 s-jira systemd: Failed to start (null).
May 17 00:00:31 s-jira systemd: Unit jira.service entered failed state.
May 17 00:00:31 s-jira systemd: jira.service failed.

All the time it will find

May 17 00:00:31 s-jira jira: Existing PID file found during start.
May 17 00:00:31 s-jira jira: Unable to read PID file. Start aborted.

deleting PID file and starting services will not work. 

only this sequence of commands will cleanup left over from previous jira

/opt/atlassian/jira/bin/shutdown.sh
/opt/atlassian/jira/bin/startup.sh

[Alex Gallien]

Hi Miroslav, seems like you have an issue with your start script.  I would recommend stopping the process, finding that .pid file, deleting it, then restarting Jira. The exact location depends on your settings, but you should be able to find it using

sudo find / -name "*.pid"

[Deleted user]

That is still manual intervention every-time i reboot server. My question is why its happening now. Before it worked without problem. 

I have script  restart-jira.sh  that i have to run every-time i reboot server. What is wrong with Jira and Confluence. BitBucket is working OK. 

restart-jira.sh

#!/bin/sh

/opt/atlassian/jira/bin/shutdown.sh
/opt/atlassian/jira/bin/startup.sh

[Deleted user]

Did not work.

systemctl status jira show  taht process is running but it does not.