How can I debug high CPU utilization on my remote agent?

I have a remote agent running on an EC2 box that is consuming vast amounts of resources, even when idle

This is the top output.

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                     
11808 ec2-user  20   0 6251m 1.6g  18m S 201.2 11.0  22726:39 /usr/java/jdk1.7.0_60/jre/bin/java -Xms512m -Xmx3096m -XX:PermSize=384m -XX:MaxPermSize=512m -jar /home/ec2-user/bamboo-agent-home/xml-data/build-dir/PEPPER-PBT517-BPC/pep

When I disable it, it doesn't go down.

When I request it to stop, vi the UI, it does not seem to want to stop.

The agent logs are not showing any errors.

I stopped the agent and noticed that the job was still there. I'm hypothesizing that this is actually an orphaned build that just got lost in the mix somehow. Is there any way I can confirm this? Has anyone seen this before?

2 answers

0 vote

The most likely reason is either a memory leak or your build logging a lot and the agent not being able to keep up. If you're able to, have a look at the heap dump from that agent.

BTW, using elastic agents is a lot better way of using EC2 in conjuction with Bamboo.

 

0 vote

Dan

It looks like your job PEPPER-PBT-BPC is running some processes in the background and doesn't kill them. You should go over your job configuration and see what could cause this.

If you use your own instance of Bamboo (not the Atlassian Cloud offering) you might want to check this plugin: https://marketplace.atlassian.com/plugins/com.atlassian.bamboo.plugin.bamboo-predator-plugin 

If only this plugin were available for Bamboo 5.10!  I also have an issue with just one Bamboo Agent that is consuming all available CPU, constantly, even though it's not building anything. 

Suggest an answer

Log in or Sign up to answer
Atlassian Community Anniversary

Happy Anniversary, Atlassian Community!

This community is celebrating its one-year anniversary and Atlassian co-founder Mike Cannon-Brookes has all the feels.

Read more
Community showcase
Renan Battaglin
Published May 18, 2017 in Bamboo

FAQ: How to Upgrade Bamboo Server

Bamboo 5.9 will no longer be supported after June 12, 2017. What does this mean? As part of our End of Life policy, Atlassian supports major versions for two years after the first major iteratio...

1,324 views 0 5
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you