After reboot, Not running confluence and Not ERROR Log

Question

I use jira and confluence for 7 month. (untill one month, I reboot OS, jira and confluence is running well) I upgrade confluence before one month - So sorry, I can not remember exact term and version number. Before and after. - If you want to know about confluence version, please talk to me how I can check that. I will answer I don't understand why I can not running confluence after reboot. I can not find any error log (catalina.out) like below. I try to find some method on the internet for several time, but I can’t find answer. please answer to me. P.S) OS is Ubuntu and I got your license for one year. I use your product for 7 month. (After reboot, Jira and mysql is running well) I check firewall. (firewall is not running.) I check “confluence.cfg.xml” in the “confluence-init.properties.”. mysql login id/pwd is right. (jira is running well) I think OS have a enough free memory below. -------------------------------------------------------------------------------------------------- 13-Apr-2019 03:45:37.715 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server} Setting property 'debug' to '0' did not find a matching property. 13-Apr-2019 03:45:37.807 WARNING [main] org.apache.catalina.startup.SetAllPropertiesRule.begin [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'debug' to '0' did not find a matching property. 13-Apr-2019 03:45:37.934 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine} Setting property 'debug' to '0' did not find a matching property. 13-Apr-2019 03:45:37.943 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'debug' to '0' did not find a matching property. 13-Apr-2019 03:45:37.992 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property. 13-Apr-2019 03:45:38.018 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property. 13-Apr-2019 03:45:38.340 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8090"] 13-Apr-2019 03:45:38.358 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read 13-Apr-2019 03:45:38.370 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 741 ms 13-Apr-2019 03:45:38.384 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Tomcat-Standalone] 13-Apr-2019 03:45:38.384 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/9.0.8 # ps -ef | grep tomcat jira 2567 1 22 03:37 ? 00:06:06 /opt/atlassian/jira/jre//bin/java -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms384m -Xmx768m -Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory -XX:-OmitStackTraceInFastThrow -Datlassian.plugins.startup.options= -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Xloggc:/opt/atlassian/jira/logs/atlassian-jira-gc-%t.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=20M -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCCause -Dignore.endorsed.dirs= -classpath /opt/atlassian/jira/bin/bootstrap.jar:/opt/atlassian/jira/bin/tomcat-juli.jar -Dcatalina.base=/opt/atlassian/jira -Dcatalina.home=/opt/atlassian/jira -Djava.io.tmpdir=/opt/atlassian/jira/temp org.apache.catalina.startup.Bootstrap start root 7204 1838 0 04:04 pts/0 00:00:00 grep tomcat # ps -ef | grep conflu avahi 879 1 0 03:31 ? 00:00:00 avahi-daemon: running [wedance-confluence.local] conflue+ 1892 1 99 03:35 ? 01:30:32 /boot/vmlinuz conflue+ 1921 1 0 03:35 ? 00:00:00 ./Oo5mIH6 ./I7f979N conflue+ 1922 1 0 03:35 ? 00:00:00 ./Oo5mIH6 ./PCAaKB6 conflue+ 7277 1921 0 04:05 ? 00:00:00 sleep 10 conflue+ 7282 1922 0 04:05 ? 00:00:00 sleep 5 root 7284 1838 0 04:05 pts/0 00:00:00 grep conflu top - 04:34:02 up 1:02, 2 users, load average: 3.17, 3.20, 3.17 Tasks: 258 total, 1 running, 190 sleeping, 0 stopped, 1 zombie %Cpu(s): 37.7 us, 0.1 sy, 0.0 ni, 59.8 id, 2.3 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 8092724 total, 4004964 free, 2384344 used, 1703416 buff/cache KiB Swap: 2097148 total, 2097148 free, 0 used. 5253924 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1892 conflue+ 20 0 328908 8236 0 S 299.6 0.1 176:42.63 1279 wedance 20 0 3858600 137492 70500 S 0.9 1.7 0:05.05 gnome-shell 2567 jira 20 0 6791712 1.402g 34152 S 0.4 18.2 7:23.24 java 10846 wedance 20 0 52756 4480 3708 R 0.4 0.1 0:00.02 top 1 root 20 0 225532 9484 6820 S 0.0 0.1 0:01.83 systemd 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd 4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/0:0H 6 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 mm_percpu_wq 7 root 20 0 0 0 0 S 0.0 0.0 0:00.01 ksoftirqd/0 8 root 20 0 0 0 0 I 0.0 0.0 0:00.53 rcu_sched 9 root 20 0 0 0 0 I 0.0 0.0 0:00.00 rcu_bh 10 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0 11 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/0 13 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/1 14 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 15 root rt 0 0 0 0 S 0.0 0.0 0:00.01 migration/1 16 root 20 0 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/1 18 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/1:0H 19 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/2 20 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/2

Daniel Eads · Answer

Based on this, it seems like you might have been affected by a malware attack from the CVE-2019-3396 Widget Connector vulnerability from March 20th (see Confluence Security Advisory - 2019-03-20). Do you remember exactly when you upgraded Confluence? We started seeing attacks in the wild for this vulnerability on April 10.

Confluence 6.15.2 is a version that is patched against the vulnerability. It was released on April 5 so I am wondering if you maybe upgraded late last week (and possibly got infected with malware before upgrading).

If it was a malware attack, you'll still need to clean up the malware from the server. The LSD malware cleanup tool will be useful for removing the Kerberods malware, which we saw most commonly on April 10 specifically. Other malware payloads might need different cleanup tools depending on which attack and payload were used. A good starting place for detecting other types of infections are the scanners linked here. Once a particular infection is identified, googling for "____ removal tool" is a good place to start if the scanner was unable to remove the malware automatically.

jyj0629 · Answer

Do you know why that process is using CPU fully?

Confluence is downed and that is appeared.

what is that and how can i do?

I killed that every time but this situation is repeatly over and over...

jyj0629 · Answer

Thanks for your advise.

I found confluence version

"Confluence 6.15.2"

How can i do?