Confluence not working after restart of the server DEBIAN 9

[Julian Castello]

Hello,

Confluence worked fine before the restarting of the server (with safe shutdown in the shell) and i don't understand what's happen.. When i try to open the main page, it's say "this page isn't working". The apache server work fine because i use other application.

I don't what can i search to help me to troubleshot this problem.

I tried to open /opt/atlassian/confluence/logs/catalina.out but the file won't open "reading file " stay on the screen and never open it.

I already try to restart confluence service and apache2.

Thank you for you help.

[Nic Brough -Adaptavist-]

You need to check if the Tomcat that supports Confluence is running first.  "ps -ef | grep java" is a simple way to find that out.

If it is running, you need to read <confluence home>/log/atlassian-confluence.log to see what is wrong, although if that says nothing, you might need to read the catalina.out that you've already tried to read. 

With that, I'm afraid "file won't open" does not tell us what is wrong - you need to read the file, it is simple text, you'll need to debug that for yourself.  Although I'd take a guess that you've disabled log rotation, the file has become massive and your editor is struggling to read it all.  If that's the case, try "tail -200 catalina.out" to see the end of it, or "tail -f catalina.out" to read it as it's being written to

[Julian Castello]

Hello,

the problem is every time the same and i don't understand the origin of that. For each restart of the server, if a check the status of confluence, it's Running (exited).

I need to stop the service and type 

ps -ef | grep confluence
root 1295 988 0 11:53 pts/0 00:00:00 grep confluence
kill 988
service confluence start

And it's works. Every time the same. What could explain that? Permission issue somewhere? 

Thank you! 

[Julian Castello]

I speak to myself... 

I solved it (again lol)

I kill the process using root :ps -ef | grep confluence

KILL PIDNUMBER

restart service without error. 

I see confluence page but error to read confluence confluence.cfg.xml . I assume it's because i change the password of confluence user (during my tests).

Re-apply right on the directory

chown -R confluence.confluence /var/atlassian/application-data/confluence

All works. WHY when i restart the server, all is broken ;((((

[Nic Brough -Adaptavist-]

You need to read the application log to see what the problem is.  What you've looked at before is the Tomcat server log, which is actually telling us that things are working as we'd expect.

My guess is that your startup process is trying to start the service as the correct user, but because you've been restarting it as root, the ownership of the files has changed and that user can no longer open them.

[Julian Castello]

The server start as service with confluence user (i use service start confluence).

It write it starting with confluence user. Not good?

[Nic Brough -Adaptavist-]

I don't know, you need to read the logs to find out why it's not starting.

[Julian Castello]

I restarted the server just for the test and not working anymore lol... 

I used ps -ef | grep java as you ask :

conflue+ 783 1 3 18:16 ? 00:00:21 /opt/atlassian/confluence/jre//bin/java -Djava.util.logging.config.file=/opt/atlassian/confluence/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -XX:ReservedCodeCacheSize=256m -XX:+UseCodeCacheFlushing -Dconfluence.context.path= -Datlassian.plugins.startup.options= -Dorg.apache.tomcat.websocket.DEFAULT_BUFFER_SIZE=32768 -Dsynchrony.enable.xhr.fallback=true -Xms1024m -Xmx1024m -XX:+UseG1GC -Datlassian.plugins.enable.wait=300 -Djava.awt.headless=true -XX:G1ReservePercent=20 -Xloggc:/opt/atlassian/confluence/logs/gc-2019-01-25_18-16-55.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M -XX:-PrintGCDetails -XX:+PrintGCDateStamps -XX:-PrintTenuringDistribution -Dignore.endorsed.dirs= -classpath /opt/atlassian/confluence/bin/bootstrap.jar:/opt/atlassian/confluence/bin/tomcat-juli.jar -Dcatalina.base=/opt/atlassian/confluence -Dcatalina.home=/opt/atlassian/confluence -Djava.io.tmpdir=/opt/atlassian/confluence/temp org.apache.catalina.startup.Bootstrap start
root 2621 901 0 18:28 pts/0 00:00:00 grep java

Now the error is :

root@debian:/opt/atlassian/confluence/bin# service confluence status
● confluence.service - LSB: Atlassian Confluence
 Loaded: loaded (/etc/init.d/confluence; generated; vendor preset: enabled)
 Active: active (exited) since Fri 2019-01-25 18:32:07 CET; 2min 25s ago
 Docs: man:systemd-sysv-generator(8)
 Process: 2549 ExecStop=/etc/init.d/confluence stop (code=exited, status=1/FAILURE)
 Process: 2752 ExecStart=/etc/init.d/confluence start (code=exited, status=0/SUCCESS)

Jan 25 18:32:07 debian confluence[2752]: If you encounter issues starting up Confluence, please see the Installation guide at http://confluence.atlassian.com/display/DOC/Confluence+Installation+Guide
Jan 25 18:32:07 debian confluence[2752]: Server startup logs are located in /opt/atlassian/confluence/logs/catalina.out
Jan 25 18:32:07 debian confluence[2752]: ---------------------------------------------------------------------------
Jan 25 18:32:07 debian confluence[2752]: Using Java: /opt/atlassian/confluence/jre//bin/java
Jan 25 18:32:07 debian confluence[2752]: 2019-01-25 18:32:07,314 INFO [main] [atlassian.confluence.bootstrap.SynchronyProxyWatchdog] A Context element for ${confluence.context.path}/synchrony-proxy is found in /opt/atlassian/confluence/c
Jan 25 18:32:07 debian confluence[2752]: ---------------------------------------------------------------------------
Jan 25 18:32:07 debian confluence[2752]: Existing PID file found during start.
Jan 25 18:32:07 debian confluence[2752]: Removing/clearing stale PID file.
Jan 25 18:32:07 debian confluence[2752]: Tomcat started.
Jan 25 18:32:07 debian systemd[1]: Started LSB: Atlassian Confluence.

When i try to stop it:

root@debian:/opt/atlassian/confluence/work# service confluence stop
root@debian:/opt/atlassian/confluence/work# service confluence status
● confluence.service - LSB: Atlassian Confluence
 Loaded: loaded (/etc/init.d/confluence; generated; vendor preset: enabled)
 Active: failed (Result: exit-code) since Fri 2019-01-25 18:39:49 CET; 2s ago
 Docs: man:systemd-sysv-generator(8)
 Process: 3617 ExecStop=/etc/init.d/confluence stop (code=exited, status=1/FAILURE)
 Process: 3465 ExecStart=/etc/init.d/confluence start (code=exited, status=0/SUCCESS)

Jan 25 18:39:49 debian confluence[3617]: Server startup logs are located in /opt/atlassian/confluence/logs/catalina.out
Jan 25 18:39:49 debian confluence[3617]: ---------------------------------------------------------------------------
Jan 25 18:39:49 debian confluence[3617]: Using Java: /opt/atlassian/confluence/jre//bin/java
Jan 25 18:39:49 debian confluence[3617]: 2019-01-25 18:39:49,739 INFO [main] [atlassian.confluence.bootstrap.SynchronyProxyWatchdog] A Context element for ${confluence.context.path}/synchrony-proxy is found in /opt/atlassian/confluence/c
Jan 25 18:39:49 debian confluence[3617]: ---------------------------------------------------------------------------
Jan 25 18:39:49 debian confluence[3617]: PID file found but either no matching process was found or the current user does not have permission to stop the process. Stop aborted.
Jan 25 18:39:49 debian systemd[1]: confluence.service: Control process exited, code=exited status=1
Jan 25 18:39:49 debian systemd[1]: Stopped LSB: Atlassian Confluence.
Jan 25 18:39:49 debian systemd[1]: confluence.service: Unit entered failed state.
Jan 25 18:39:49 debian systemd[1]: confluence.service: Failed with result 'exit-code'.

[Julian Castello]

Is there another way to stop the server? I used shutdown. Have i to stop all services of confluence before?

[Julian Castello]

I disable the auto start of the service confluence and i tried to start it manually, now i can see the error :

Jan 25 17:32:39 debian confluence[1263]: ---------------------------------------------------------------------------
Jan 25 17:32:39 debian confluence[1263]: Existing PID file found during start.
Jan 25 17:32:39 debian confluence[1263]: Tomcat appears to still be running with PID 660. Start aborted.
Jan 25 17:32:39 debian confluence[1263]: If the following process is not a Tomcat process, remove the PID file and try again:
Jan 25 17:32:39 debian confluence[1263]: UID PID PPID C STIME TTY TIME CMD
Jan 25 17:32:39 debian confluence[1263]: conflue+ 660 1 74 17:28 ? 00:03:23 /opt/atlassian/confluence/jre//bin/java -Djava.util.logging.config.file=/opt/atlassian/confluence/conf/logging.properties -Djava.util.logging.manage
Jan 25 17:32:39 debian systemd[1]: confluence.service: Control process exited, code=exited status=1
Jan 25 17:32:39 debian systemd[1]: Failed to start LSB: Atlassian Confluence.
Jan 25 17:32:39 debian systemd[1]: confluence.service: Unit entered failed state.
Jan 25 17:32:39 debian systemd[1]: confluence.service: Failed with result 'exit-code'.

What can i check? I just restarted my linux server and now it's broken :( 

Thank you for your help!

[Julian Castello]

FIxed.

I do that :

root@debian:/opt/atlassian/confluence/bin# mv /opt/atlassian/confluence/work/catalina.pid catalina.pid.bkp
root@debian:/opt/atlassian/confluence/bin# service confluence restart

root@debian:/opt/atlassian/confluence/bin# service confluence status
● confluence.service - LSB: Atlassian Confluence
 Loaded: loaded (/etc/init.d/confluence; generated; vendor preset: enabled)
 Active: active (exited) since Fri 2019-01-25 18:06:52 CET; 5s ago
 Docs: man:systemd-sysv-generator(8)
 Process: 3160 ExecStart=/etc/init.d/confluence start (code=exited, status=0/SUCCESS)

Jan 25 18:06:51 debian confluence[3160]: executing using dedicated user: confluence
Jan 25 18:06:51 debian runuser[3165]: pam_unix(runuser:session): session opened for user confluence by (uid=0)
Jan 25 18:06:51 debian confluence[3160]: If you encounter issues starting up Confluence, please see the Installation guide at http://confluence.atlassian.com/display/DOC/Confluence+Installation+Guide
Jan 25 18:06:51 debian confluence[3160]: Server startup logs are located in /opt/atlassian/confluence/logs/catalina.out
Jan 25 18:06:51 debian confluence[3160]: ---------------------------------------------------------------------------
Jan 25 18:06:51 debian confluence[3160]: Using Java: /opt/atlassian/confluence/jre//bin/java
Jan 25 18:06:52 debian confluence[3160]: 2019-01-25 18:06:52,214 INFO [main] [atlassian.confluence.bootstrap.SynchronyProxyWatchdog] A Context element for ${confluence.context.path}/synchrony-proxy is found in /opt/atlassian/confluence/c
Jan 25 18:06:52 debian confluence[3160]: ---------------------------------------------------------------------------
Jan 25 18:06:52 debian confluence[3160]: Tomcat started.
Jan 25 18:06:52 debian systemd[1]: Started LSB: Atlassian Confluence.

All works fine now ! I don't understand what's happen. 

Found solution here : https://community.atlassian.com/t5/Jira-questions/Jira-service-not-starting-correctly-after-basic-config/qaq-p/652888