Two Bamboo Servers vs. One Bamboo Server

[Jordan Packer]

We currently have two Bamboo instances in our company. One is used for all pre-prod (dev/test/stage) builds and deployments, the other is used strictly for production builds and deployments. The two Bamboo instances have different user access levels (developers in one instance, operations in the other instance), reside on different servers in different domains on the network, run under different user accounts on the server, and have different sets of build agents (local and remote).

I am evaluating the feasibility of moving everything to one Bamboo instance. Having them split in two creates several problems, and, from what I can tell, does not provide many benefits (the person who decided to do it this way in the first place is no longer available for questioning). Here is what I've discovered so far:

Problems:

• Plan cloning cannot occur all the way up the stack. Any plan created in one instance has to be manually re-created in the other instance. This causes problems when initially setting up plans, but also when making changes to the pre-prod plans that then have to be mimicked in the prod plans. Even the tiniest, slightest change in the pre-prod builds can, if forgotten, destroy confidence in the integrity of the production build plan.
• If we use the new Deployment Projects feature, the dashboard is not as helpful because one environment only shows pre-prod deployments/status, and the other environment only shows production deployments/status. This defeats the purpose of the Deployment Projects, in our opinion.
• Any resources needed on the build server to support a software build (Visual Studio, Java, additional libraries, etc.) have to be present and maintained on both build servers.
• We only have one instance integrating with JIRA, and not sure if we should/can integrate the other instance with JIRA (does the one-to-many connection cause problems, etc.).
• We have to pay for licenses/maintenance/support on two instances instead of just one.

Benefits:

• Security. Because the production builds all occur on a separate server in different domain under a different user account, we are able to harden that environment much more thoroughly and protect it from unwanted (or unauthorized) changes.

I have the resources available to me to make the build server very robust and powerful, so performance considerations are not a concern. I'm just trying to gather any other ideas as to why I might need to keep these instances separate. If it's just a security concern, I think I can convince management to combine them. Between utilizing the granular permissions capabilities and user training, I think we could avoid most security problems that they're worried about (and besides, if there are rogue deployments to Prod occurring, that's not something that should be fixed by technology, that's something that should be fixed by management).

All thoughts/questions/feedback/ideas are welcome. I anticipate your responses.

[Daniel Wester · Accepted solution]

You don't mention which Bamboo you're on. With Bamboo 5.0's deployment projects you can control access to the target environments. You can also reserve certain remote agents to be for deployments.. https://confluence.atlassian.com/display/BAMBOO/Agents+for+deployment+environments

Having 2 build servers creating the same builds is kinda scary since you can't really rely on that they're pulling the same dependencies. Depending on what your build system is - there is a potential that a version is bumped in between the builds and the 'dev' build won't really match the 'prod' build (even though it's the same source code). It is also easy to get version drift between the underlaying build components (for example your JDK might be 1.7.2_1 on on - but 1.7.2_3 on the other). The odds of these happening might be small(and even minute) - but when they happen - it's not fun to troubleshoot.

[Jordan Packer]

I completely agree with your points, thanks for backing me up there. As for the version of Bamboo we're using, it's version 5.2 so, yes, we could gain most of the security features we need from using the deployment projects (which is what I hope to do).

[shari_anderson]

Losing the ability to clone plans is IMHO enough reason to use 1 server. The cost savings on Bamboo and not having to duplicate / keep in sync the set of libraries to support builds is just a bonus. :)

[Ryan Armstrong]

I would love to hear how this turned out for you Jordan. I am researching the possibility of splitting our single build server into two servers. One idea on how to separate the server is, as you described, along the lines of production and development. I think, after reading this, that is not the route to go. However, it would be amazing if I could stir up some more conversation about the pros/cons of having two build servers vs. one server. Any feedback helps, thank you! 

[Jordan Packer]

Hey Ryan, glad to hear from you. As you can see in some of the other answers/comments in this thread, the pros for having one build server far outweigh the cons. We have been successfully running on one build server (with multiple remote agents) for over a year now. I would never go back. There are many reasons (configuration drift, maintenance, setup overhead, and more) why two environments was just not feasible. But above all those, one simple reason why one server is so great is because of the Deployment Projects. This area lets us have a clear picture of what releases are deployed in which environments at any given time. If you have one Bamboo environment for pre-production and another environment for production, then you no longer get this picture across all of your target deployment environments. Moving the artifacts across environments would also be a big pain. I would say that the only way I would consider going back to two environments would not be for splitting pre-prod from prod, but rather splitting teams/projects/responsibilities. Maybe one Bamboo environment would be dedicated to software teams, and another Bamboo environment would be dedicated to infrastructure (for example, Docker). Or any other combination that would make sense for your organization. But all in all, I would highly recommend _against_ splitting between pre-prod and prod. Hope this helps!

[Ryan Armstrong]

Hey Jordan, Thanks so much for the speedy response! As you pointed out, it is clear from this discussion that having two environments split along pre-prod and prod is more work than it is worth. I had thought about splitting the server along team lines, but again the same problems of configuration drift, maintenance, and setup overhead persist. At any rate, this has been very helpful and thank you again for sharing your experience.