please it's not working

Question

17-Apr-2019 15:46:38.133 INFO [localhost-startStop-2] org.springframework.context.support.DefaultLifecycleProcessor.start Starting beans in phase 2147483647 17-Apr-2019 15:46:38.150 INFO [localhost-startStop-2] org.springframework.web.servlet.DispatcherServlet.initServletBean FrameworkServlet 'dispatcher': initialization completed in 1199 ms 17-Apr-2019 15:47:02.462 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8090"] 17-Apr-2019 15:47:02.476 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 64705 ms 17-Apr-2019 15:47:03.365 INFO [http-nio-8090-exec-2] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:47:03.605 INFO [http-nio-8090-exec-5] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:47:06.077 INFO [http-nio-8090-exec-10] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:47:06.662 INFO [http-nio-8090-exec-1] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:49:02.974 INFO [http-nio-8090-exec-5] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:49:02.974 INFO [http-nio-8090-exec-2] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:49:03.000 INFO [http-nio-8090-exec-2] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc 17-Apr-2019 15:49:03.002 INFO [http-nio-8090-exec-2] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorGrammarsSupport 17-Apr-2019 15:49:03.003 INFO [http-nio-8090-exec-2] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport 17-Apr-2019 15:49:03.000 INFO [http-nio-8090-exec-5] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc 17-Apr-2019 15:49:03.009 INFO [http-nio-8090-exec-5] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorGrammarsSupport 17-Apr-2019 15:49:03.009 INFO [http-nio-8090-exec-5] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport 17-Apr-2019 15:49:03.193 INFO [http-nio-8090-exec-1] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:49:03.208 INFO [http-nio-8090-exec-4] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:49:03.526 INFO [http-nio-8090-exec-9] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:49:03.571 INFO [http-nio-8090-exec-8] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:49:03.833 INFO [http-nio-8090-exec-2] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:49:05.944 INFO [http-nio-8090-exec-10] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:55:41.020 INFO [http-nio-8090-exec-1] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.8-atlassian-16 03/23/2015 10:20 PM' 17-Apr-2019 15:56:23.935 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server} Setting property 'debug' to '0' did not find a matching property. 17-Apr-2019 15:56:24.328 WARNING [main] org.apache.catalina.startup.SetAllPropertiesRule.begin [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'debug' to '0' did not find a matching property. 17-Apr-2019 15:56:24.356 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine} Setting property 'debug' to '0' did not find a matching property. 17-Apr-2019 15:56:24.449 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'debug' to '0' did not find a matching property. 17-Apr-2019 15:56:24.500 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property. 17-Apr-2019 15:56:24.559 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property. 17-Apr-2019 15:56:25.093 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8090"] 17-Apr-2019 15:56:25.122 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read 17-Apr-2019 15:56:25.145 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 1337 ms 17-Apr-2019 15:56:25.152 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service Tomcat-Standalone 17-Apr-2019 15:56:25.152 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.0.48

Daniel Eads · Answer

Based on the Tomcat version listed in your log output, I can infer you are on a version of Confluence we are concerned about as part of a security issue. From what's in your logs, it sounds like your instance was affected by an opportunistic attack against the CVE-2019-3396 Widget Connector vulnerability from March 20th (see Confluence Security Advisory - 2019-03-20). We've seen an infection going around that injects malware and the bitcoin miner it tries to run uses all the CPU available on the box. Initially the kerberods malware was being deployed as the payload, but other attacks might be trying to inject different payloads.

I'd recommend tackling things in this order:

* Kill malicious processes
* Clean up your crontab
* Upgrade Confluence
* Use a malware scanner to find remaining malware traces

Malicious processes

The top command will help you find processes (probably running under the confluence user account) that are consuming a large amount of CPU. If Confluence is currently stopped, you can probably plan on killing any processes running as the confluence user. note the process ID (pid) from the top output and then kill the process using kill -9 followed by the pid. Example:

sudo kill -9 12395

Clean up your crontab

Since most malware adds a cronjob that relaunches the malware every few minutes, you'll also need to check the crontab file and remove any suspicious-looking entries. For Ubuntu, this is stored in the /var/spool/cron/crontabs/ directory. Normally you should use the crontab command to edit the crontab, but for cleanup purposes we'll be inspecting the file for any pre-existing entries.

Using vim (or whichever text editor you're comfortable with), you'll open the file and remove suspicious-looking jobs.

sudo vim /var/spool/cron/crontabs/confluence

Confluence comes up on system startup through the SysV/systemd daemons, so we would expect the confluence user's crontab to not exist under normal circumstances. It's most likely the case that any entries in this file are malicious, but make sure you check them before deleting them entirely.

Upgrade Confluence

Once your CPU is under control and new malicious process aren't spawning, you need to upgrade Confluence to a version that isn't affected by the vulnerability. I'd recommend looking at one of these versions (latest releases as of this post):

* 6.13.4 (6.13 is an Enterprise release)
* 6.14.3
* 6.15.2

Use a malware scanner

Finally, you need to clean up any remaining traces of malware on your system. The LSD malware cleanup tool will be useful for removing the Kerberods malware. Other malware payloads might need different cleanup tools depending on which attack and payload were used. A good starting place for detecting other types of infections are the scanners linked here. Once a particular infection is identified, googling for "____ removal tool" is a good place to start if the scanner was unable to remove the malware automatically.

Please let me know if you have more questions!
Daniel | Atlassian Support