nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

No access to Conlfuence Webpage, 100% CPU usage from "dbused"

TUHH-DKA

Hello together,

I have the problem that I can't connect to my Confluence instance, which I host on a virtual machine. If I look into htop I see that the command "dbused" from the user "confluence" is using 100% CPU and a lot of RAM. I'm not able to find anything useful searching the web.

If I restart confluence or even the server this behaviour will start again after a few minutes. If I kill this process it will start again after a short time.

Does anybody know what to do about that or under what keyword I can search for a solution?

I'm sorry if I'm missing something basic. We are a small research institute and we don't have "real" IT admins who are hosting our servers, so I have to do it with only intermediate linux/server knowledge...

Best regards

Dennis

Accepted answers

Alex Rangeo

Sounds like a miner of some type. Here is some of the steps I've taken, along with upgrading Confluence to the latest LTS.

Delete and Block access to a flagged file from Symantec -

rm -rf /tmp/dbused

chmod -R 000 /tmp/.pwn

Remove the cronjob that reloads the miner. It is running under the user that runs confluence.

crontab -e

You will see a job running every minute that is clearly attempting to connect and run rouge bash commands

Then looked up the host that the cron job communicated with, and added firewall rules to block the traffic.

reboot

Upgrade to the latest LTS

All comments

Robert Wen_Cprime_

This sounds a lot like your Confluence server has a virus.

Scot Weir

Ran into this issue as well today for the first time.

On Confluence restart attempt, logs reveal:

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000f6500000, 32505856, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 32505856 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmp/hs_err_pid23285.log

free -m command shows memory dwindling slowly until approx 40MB available, then "resets" and begins the drain again.

We are running Confluence and Jira on AWS instances and truly do not want to move up to next size of instance type just to accommodate. I have tried making the recommended changes in the /tmp/hs_err_pid23285.log file with no success.

I have opened a support ticket with Atlassian in the meantime

Alex Rangeo

Sounds like a miner of some type. Here is some of the steps I've taken, along with upgrading Confluence to the latest LTS.

Delete and Block access to a flagged file from Symantec -

rm -rf /tmp/dbused

chmod -R 000 /tmp/.pwn

Remove the cronjob that reloads the miner. It is running under the user that runs confluence.

crontab -e

You will see a job running every minute that is clearly attempting to connect and run rouge bash commands

Then looked up the host that the cron job communicated with, and added firewall rules to block the traffic.

reboot

Upgrade to the latest LTS

Scot Weir

Thank you @Alex Rangeo , followed the above steps and Confluence has returned! Will be updating to LTS at the earliest opportunity.

Michael O_Brien

I have this issue as well starting at midnight on 1 Sept 2021 - bumping up my instance from 4g - but I was running fine on 70% ram utilization in the past

actually it is kworker (crypto) - I left 0.0.0.0/0 open - locking down the security group firewall

pavel shvetsov

i was infected as well; you can read the "/tmp/xms" source script here

http://bash.givemexyz.in/xms

and find out that its better to check all these files and folders

/etc/cron.d/root /etc/cron.d/apache /var/spool/cron/root /var/spool/cron/crontabs/root /etc/cron.hourly/oanacroner1 /etc/init.d/down

in my case i found crontab job inside `/var/spool/cron/crontabs/confluence2` file like Alex said

Gustavo Jimenez_1ykm0p

i have the same problem on differents machine with confluence on AWS, all these machine has a CPU consume 100% o more

wirat_leenavonganan

I also found `/var/spool/cron/crontabs/confluence`

Kamil Beer_ust9kw

Same problem here. I delete the /var/spool/cron/confluence file, but it regenerates shortly and starts the dbused process again.

Trung Vu_66x5p4

try reboot the server, then quickly remove it

/var/spool/cron/confluence

/tmp/dbused

/tmp/.pwn

check the location as above comment:

etc/cron.d/root /etc/cron.d/apache /var/spool/cron/root /var/spool/cron/crontabs/root /etc/cron.hourly/oanacroner1 /etc/init.d/down

upgrade the server to LTS version will help resolve it.

Kamil Beer_ust9kw

Hey everyone,

after some time struggling, it seems that, at least for some time, I have managed to shut the miner, or whatever it was, down, and get Confluence running. What I did I did in a very short time window.

1) I removed the cron entry for confluence, which was located in /var/spool/cron

2) I killed all the suspicious processes, like the dbused

3) I applied Atlassian's patch.

From then, dbuserd didn't start, nor did the cronfile generate again.

Finally, I removed all the suspicious files from /tmp.

I'm now going to apply the LTE version.

Daniel Stiven Ballen Garcia

https://confluence.atlassian.com/doc/confluence-security-advisory-2021-08-25-1077906215.html

m_lang

It seems that we had to start Confluence after applying the patch, provided by atlassian.

Our steps:

- Shut down confluence
- Apply Bugfix patch
- removed all cronjobs, located in /var/spool/cron/crontabs and /var/spool/cron/atjobs
- removed suspicious files in /tmp
- killed dbused process, via kill -9 [pid]
- Start confluence

After confluence started the cronjob didn't regenerate under /var/spool/cron/crontabs. The dbused process was gone and no other suspicious files in /tmp.

Daniel Stiven Ballen Garcia

De los archivos del /tmp cuales reconocen como maliciosos?

Daniel Stiven Ballen Garcia

@Kamil Beer de los archivos que encontro en el /tmp cuales fueron sospechosos o que prefijo tenian?

Gracias

m_lang

I compared to another confluence instance, and the only folder there was: hsperfdata_confluence/

All other files owned by the confluence user were suspicious.

Without warranty, of course.

Kamil Beer Vodafone

@Daniel Stiven Ballen Garcia, check how old they are, perhaps using "ls -lah". Most of the suspicious files were made on September 1st or 2nd (or whenever did this problem occur to you).

Kamil Beer Vodafone

Yeah, the confluence ownership is a giveaway, as the confluence user wouldn't have much of a reason to create the files there.

Kamil Beer Vodafone

@Trung Vu, yeah. The cronfile, for me, generated in /var/spool/cron, with the confoluence user. Anyway, it seems I resolved it using the workaround, but will update to LTS soon anyway.

Alex Miu

https://tolisec.com/multi-vector-minertsunami-botnet-with-ssh-lateral-movement/
this is also a good read about this malware. Will also apply the patch later. Seems like removing the crons and files don't help for long. It was fine for 1-2 hours, then i got another xmm monero crypto miner thing running with 100%. good thing they all get created in /tmp