Running many identical Bamboo agents with Docker

We're trying to run many unit-test jobs in parallel. Our Bamboo plan consists of one build job, followed by many (up to 40) unit test jobs, that may take long (up to 30 minutes each or more). We want to drastically reduce the time it takes to run these tests by having a lot of Bamboo agents running them in parallel.

We're using docker to spawn light-weight VMs, but we are wondering how to best use it:

Option 1) Many Bamboo agents running on the host machine. Each agent spawns a Docker container (Docker task in Bamboo), runs the test and then shuts down the container.

Option 2) Many detached Docker containers on the host machine, each running one Bamboo agent. We don't use the Bamboo Docker task, we just run the executable in each agent.

Have you faced the same problem? What option do you think will work best in the long term?

1 answer

0 vote

In Option 1, your builds will always start in a clean environment, so checkouts, Maven fetches etc will take longer to complete. This may be an advantage (stable builds) or a disadvantage. Option 2 has the reverse property.

In Option 2, you get the agent process isolation without modifications to your plans, which Option 3.

Note that you can also run multiple agents on the same host without Docker, just provide them with a separate home dir.

Suggest an answer

Log in or Sign up to answer
How to earn badges on the Atlassian Community

How to earn badges on the Atlassian Community

Badges are a great way to show off community activity, whether you’re a newbie or a Champion.

Learn more
Community showcase
Published May 18, 2017 in Bamboo

FAQ: How to Upgrade Bamboo Server

Bamboo 5.9 will no longer be supported after June 12, 2017. What does this mean? As part of our End of Life policy, Atlassian supports major versions for two years after the first major iteratio...

1,793 views 0 6
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you