Confluence not start up

[Haipeng Lin]

Hi,

I am new for using Confluence. Just 8 hours ago, the confluence stop working. After we reboot the PC and start Confluence by run "./startup.sh" in the Confluence/bin folder. But the issue still exists.  The console output after running the startup.sh is shown as below:

If you encounter issues starting up Confluence, please see the Installation guide at http://confluence.atlassian.com/display/DOC/Confluence+Installation+Guide

Server startup logs are located in /home/jira/atlassian/confluence/logs/catalina.out
---------------------------------------------------------------------------
Using Java: /home/jira/atlassian/confluence/jre//bin/java
2018-03-28 16:01:16,577 INFO [main] [atlassian.confluence.bootstrap.SynchronyProxyWatchdog] A Context element for ${confluence.context.path}/synchrony-proxy is found in /home/jira/atlassian/confluence/conf/server.xml. No further action is required
---------------------------------------------------------------------------
Using CATALINA_BASE: /home/jira/atlassian/confluence
Using CATALINA_HOME: /home/jira/atlassian/confluence
Using CATALINA_TMPDIR: /home/jira/atlassian/confluence/temp
Using JRE_HOME: /home/jira/atlassian/confluence/jre/
Using CLASSPATH: /home/jira/atlassian/confluence/bin/bootstrap.jar:/home/jira/atlassian/confluence/bin/tomcat-juli.jar
Using CATALINA_PID: /home/jira/atlassian/confluence/work/catalina.pid
Existing PID file found during start.
Tomcat appears to still be running with PID 2965. Start aborted.
If the following process is not a Tomcat process, remove the PID file and try again:
UID PID PPID C STIME TTY TIME CMD
jira 2965 2098 5 15:43 pts/18 00:01:04 /home/jira/atlassian/confluence/jre//bin/java -Djava.util.logging.config.file=/home/jira/atlassian/confluence/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dconfluence.context.path= -Dorg.apache.tomcat.websocket.DEFAULT_BUFFER_SIZE=32768 -Dsynchrony.enable.xhr.fallback=true -Xms1024m -Xmx1024m -XX:+UseG1GC -Datlassian.plugins.enable.wait=300 -Djava.awt.headless=true -XX:G1ReservePercent=20 -Xloggc:/home/jira/atlassian/confluence/logs/gc-2018-03-28_15-43-16.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M -XX:-PrintGCDetails -XX:+PrintGCDateStamps -XX:-PrintTenuringDistribution -Dignore.endorsed.dirs= -classpath /home/jira/atlassian/confluence/bin/bootstrap.jar:/home/jira/atlassian/confluence/bin/tomcat-juli.jar -Dcatalina.base=/home/jira/atlassian/confluence -Dcatalina.home=/home/jira/atlassian/confluence -Djava.io.tmpdir=/home/jira/atlassian/confluence/temp org.apache.catalina.startup.Bootstrap start

I have checked the log files. In catlina.2018-3-28.log file, there are exception related to HTTP request. See in information appended to the end of this message. 

If possible, could you please help me to provide some hints to make the confluence start up and come back to life. Thanks for your help.

28-Mar-2018 15:23:04.731 INFO [http-nio-8090-exec-1] org.apache.coyote.http11.AbstractHttp11Processor.process Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
at org.apache.coyote.http11.AbstractNioInputBuffer.parseRequestLine(AbstractNioInputBuffer.java:235)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1045)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1539)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1495)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)

28-Mar-2018 15:23:04.736 INFO [http-nio-8090-exec-5] org.apache.coyote.http11.AbstractHttp11Processor.process Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
at org.apache.coyote.http11.AbstractNioInputBuffer.parseRequestLine(AbstractNioInputBuffer.java:235)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1045)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1539)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1495)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)

[James Richards · Accepted solution]

Hi,

I worked with Haipeng on this, and the main issue was that the user permissions in Linux were not set correctly on the files.

The fix in general is to run

chown -R confluence.confluence /opt/atlassian/confluence/
chown -R confluence.confluence /var/atlassian/application-data/confluence/
chmod -R u=rwX,go-rwx /opt/atlassian/confluence/
chmod -R u=rwX,go-rwx /var/atlassian/application-data/confluence/

And then to make sure you always run 

/opt/atlassian/confluence/bin/start-confluence.sh

 and

/opt/atlassian/confluence/bin/stop-confluence.sh

to make sure the correct user is used when starting/stopping Confluence. If you use root user or startup.sh / shutdown.sh then the home directory files all get owned by the wrong user.

Regards,
James.

[Deleted user]

Thank you. It worked for me. Fixed both Jira and Confluence. 

[Michael Oancea]

It worked, thank you guys ! 

[Nic Brough -Adaptavist-]

Looks to me like it's already running.  Check what is really running with

ps -ef

[Shannon S]

Haipeng,

Can you try ps aux | grep java and see if the Confluence PID is running? Then you can kill it and try to start Confluence again.

The error you sent looks like it's just complaining that Confluence was already running, so it might not have stopped properly last time.

As for your original issue, what was the problem? Do you have any error logs from that? We will need to see further logs as the ones you've included don't include enough information for us to know what the problem was.

Regards,

Shannon

[Haipeng Lin]

Hi Shannon,

My original problem is I cannot access to the confluence web page and it looks that confluence not running even after I call the start up program "startup.sh" in the bin folder. 

I also typed ps aux | grep java, but cannot see any confluence pid running. Please see the screen shot below

But I have found there are 2 java running, one in jira server and one in confluence. Will this cause the problem. If it is, how can I fix it?

Thanks for your help.

Regards

Haipeng Lin

[Haipeng Lin]

Hi Shannon,

I have also tried to kill the java pid related to confluence and restart the confluence by calling "startup.sh". But it still has the same problem,

Regards

Haipeng

[Haipeng Lin]

For the logging file, I checked the folder and cannot find the "atlassian-confluence.log" file. see in picture below

[Shannon S]

Haipeng,

It doesn't look like you're looking in the right location. Have a look here:

• Working with Confluence Logs

When you start Confluence, log entries will be sent to the application server logs until Confluence has completed its initial bootstrap. Any log entries written to the console will be repeated into the log in the Confluence home directory as described below.

Once the initial startup sequence is complete, all logging will be to <confluence-home>/logs/atlassian-confluence.log. For example: c:/confluence/data/logs/atlassian-confluence.log.

Note that the default location is the Confluence home directory, not the application server's log file. The home directory is specified in <confluence-installation>/confluence/WEB-INF/classes/confluence-init.properties.

From your example, it doesn't look like you're in your home directory but in the Confluence installation folder /logs.

Please make sure to go to your home directory and review the logs there.

Did you also try Nic's suggestion and try to review the processes that way? If you're getting the error from earlier then it definitely means Confluence is currently running. Something in your logs will tell us what the issue is.

Regards,
Shannon

[Haipeng Lin]

Hi Shannon,

I have followed your instructions and find the confluence log file. In the log file, there are a lot of errors during the start up. One of the error is related to the serverlet broken pipe. The log detail is shown as below. 

If possible, could you please help me to fix this problem.

And also if possible, could you please send me your email address to haipeng@mrd.com.au, so that I can send you the full logging details.Thanks.

2018-03-27 09:42:39,202 ERROR [http-nio-8090-exec-1] [[Standalone].[localhost].[/].[file-server]] log Servlet.service() for servlet file-server threw exception
-- referer: http://10.0.0.65:8090/pages/viewrecentblogposts.action?key=TRAC | url: /s/cc3fe40f5a0553b7cbda70280bb655fe-CDN/en_GB/7601/d9a7d86ddd6172b1b5a38b9e600c8b90c362def3/fe1f5fbbe96232562b250920a0dd816b/_/download/contextbatch/js/editor-v3,editor,macro-browser,-_super,-atl.general,-viewcontent,-main,-blogpost,-space/batch.js | traceId: 9948e967d3d3e30d | userName: Yvie
java.lang.RuntimeException: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
at com.atlassian.plugin.cache.filecache.impl.StreamsCache.streamFromFile(StreamsCache.java:140)
at com.atlassian.plugin.cache.filecache.impl.OneStreamCache.stream(OneStreamCache.java:65)
at com.atlassian.plugin.cache.filecache.impl.FileCacheImpl.cache(FileCacheImpl.java:100)
at com.atlassian.plugin.webresource.impl.http.Controller.sendCachedInProduction(Controller.java:355)
at com.atlassian.plugin.webresource.impl.http.Controller.sendCached(Controller.java:305)
at com.atlassian.plugin.webresource.impl.http.Controller.serveResources(Controller.java:234)
at com.atlassian.plugin.webresource.impl.http.Controller.serveBatch(Controller.java:111)
at com.atlassian.plugin.webresource.impl.http.Router$5.apply(Router.java:92)
at com.atlassian.plugin.webresource.impl.http.Router$5.apply(Router.java:83)
at com.atlassian.plugin.webresource.impl.support.http.BaseRouter.callHandler(BaseRouter.java:197)

[Shannon S]

Haipeng,

Unfortunately, the logs you're showing haven't shown any relevant information that could help me to understand the full issue you are experiencing. I'm not comfortable sharing my contact information on a public forum.

I will go ahead and create a support ticket for you. Once you receive an email, please upload your full logs under your support ticket number to:

• transfer.atlassian.io

Thank you!

Shannon