As of around July 4th'ish Hipchat server would go down as the log directory was at 100%. I purged it, then 2 days later it happened again. Then today I was notified from a user on my mail server that mail wasn't working. I investigated and hipchat had tens of thousands of emails about the UK lottery winner blah blah. I shut down hipchat and it's still offline. Anyone else have this issue? How can I resolve it? I don't know of any anti-virus or ways of cleaning hipchat such as clamav. I'm running within Virtual Box on Windows Server which has symantec anti-virus scanning all servers.
I ended up installing a new instance of HC Server and exported from the bad and imported to the new instance. All is fine now, no activity like before with the emails or log files. Had this been a compromised email account the issue would have remained in the new instance. Somehow malware got into HC 2.4.0.
Nope, not what I meant to say, I said what I said. HC 2.4.0 is vulnerable to malware. We don't know what people say to Atlassian. I think those that saw what they did in mine were a bit surprised. Typical unix mentality that it's not vulnerable, why there isn't even Clam AV installed in the instance is odd. All moot, HC is dead, hello Slack.
Nope, still wrong.
> We don't know what people say to Atlassian.
I asked. Not only has no-one else reported this, if it had got malware in it, they would have pulled it completely and announced it.
So, the only possibility is that you or an attacker (via an insecurity in your systems) managed to install the malware on your system.
Ah, I see. I apologise, I misunderstood.
The way Hipchat is usually installed means that it's usually almost impossible to inject code into it, such as a virus or hack.
I suspect what has happened is one or more of your users has been compromised, and there are now 'bots logging into your Hipchat as that user and dumping spam into a room.
It might be the case. But that is extremely unlikely. There are no reports that I can find with any version of Hipchat that are not down to an account being compromised. If a virus or trojan somehow got into an Atlassian distribution, they would have removed if from download and told everyone not to use it. They have not.
By far the most likely explanation is that one or more of your accounts have been compromised.
Go have a look at the account(s) that are causing this spam. You are going to find that they have been hacked.
It does not sound like hipchat is the problem, it sounds like your email addresses have been hit with thousands of spam email which hipchat tries to process and fills up the disk logging it.
I think you need some anti-spam software on your email server, something to drop the incoming spam before hipchat looks at it.
What may be the case is Atlassian shipped 2.4 with a virus/trojan. I unfortunately had the IP address whitelisted which may be how I learned of this. I have since removed the whitelist so SMTP AUTH is required. It doesn't mean everyone doesn't have the virus/trojan in HC Server 2.4. I've seen this once in the past a software vendor unintentionally shipped a software update with a virus.
Today I did a hipchat upgrade --force-upgrade to hopefully replace the install with a fresh copy. With the IP address no longer whitelisted at least one of these two actions have mitigated the situation. Hopefully Atlassian didn't have a virus/trojan in the 2.4 install set.
I don't believe it's compromised account as the IP address of the sender is the Hipchat Server IP address. While the issue hasn't reoccurred, this may be due to the removing of the open gateway for this IP address. I'm still having issues with the log file folder reaching 100% and shutting me down. Working with Atlassian to see if there is anything they see. I still believe it's a virus/trojan causing it.
The emails started again yesterday so it's not an open gateway on the mail server which was removed. I again believe it's malware as I shut down the hipchat instance and the mails completely stop, restart the instance and the mails send again. I have since restored a backup to HC server 2.2.9 then reapplied the 2.4 update. We lost all conversations since then but so be it, we need to move on.
Nothing but routine updates of HipChat. HipChat 2.2.9 to 2.4 would not install properly, the Chef package was corrupt so I followed Atlassian's guidance to download/install that and one KB article about passwords mismatching again per Atlassian. I think somewhere in there was malware OR it was a security issue known by Atlassian which drove the 2.4 update. I think all of the Tomcat clients are vulnerable right now and I'm waiting on Atlassian's updates across the board in their other products.
All good things come to an end - thanks to all our customers and partners who have been along the Hipchat and Stride journey with us. As of Feb 15th 2019, Hipchat Cloud and Stride have reached ...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events