Creating a CI/CD Pipeline for a Services Platform

[Chris Flynn]

Does anyone have any advice or know of any good resources to learn how to best implement a CI/CD pipeline. We are building a services platform solution with lots of smaller services that need to be built and deployed in a salable fashion. We use Java and Maven builds with Bamboo.

What other information can I provided to help facilitate a discussion or help people provide advice?

Thank you very much for your help.

[Minh Tran]

@Chris Flynn 

So i suggest you use bamboo spec api to create your Bamboo plans programmatically, please refer this for more information https://docs.atlassian.com/bamboo-specs-docs/6.4.0/specs-java.html#introduction

In theory Bamboo does not know your sub services and there is nothing called "auto-discover" sub-services in Bamboo

However, with Bamboo spec, you can define 2 Bamboo spec:

• First one is the bootstrap, this class will be used to create / update all of your bamboo specs that create the builds for all sub services
• Second one is the bamboo spec that will be used to automatically to explore there is a new sub service. You can define a list of sub services in this class, whenever you have a new sub service, you just add a new one in this class and it will generate a new bamboo spec for you. So this is programmatically way to do

In the second bamboo spec, we will have something something like this:

• define the default linked repository for each of sub service, this one needs to set up triggering based on commits. This will help trigger the builds based on your changes for each of sub service repository. That means a change in sub-service 1 will not cause a trigger for sub-service 2 and vice versa. The first task will use above default linked repository to check out your sub service source code
• Another task is to run your integration tests against the sub-service if having any
• This task is to pack up your service as a docker image or publish it to a central place for further deployment depending on your infrastructure
• This las task is to deploy the previous artifact to your expected production / staging / beta environment servers

I hope this clears your concern

Cheers,

Minh Tran

[Chris Flynn]

One thought going around within the team is that it would be ideal if there was a way to setup a plan per "system" or per "system + environment" that automatically processed new components when they are added to Subversion. Is it possible to do something let that and only rebuild components that were changed.

1. Is there a way to setup System A in Bamboo so it builds and deploys only Sub-service 1 when it is changed, but not Sub-service 2 or others
2. Is there a way to setup System A so that Bamboo "auto-discovers" Sub-service [n] and future components

Subversion Structure

System A

              |--> Sub-service 1

                              |--> src/main/java/com/company/code...

                              |--> pom.xml

              |--> Sub-service 2

                              |--> src/main/java/com/company/code...

                              |--> pom.xml

              |--> Sub-service [n]

                              |--> src/main/java/com/company/code...

                              |--> pom.xml

...

[Chris Flynn]

I will definitely have to check out that book, since you are not the first person to recommend it to me! :) Thank you very much for the quick response and help.

[Minh Tran]

@Chris Flynn I suggest this book for the state of the art of Continuous Delivery https://www.amazon.com.au/Continuous-Delivery-Jez-Humble/dp/0321601912

If you have any specific questions to discuss, please raise then we can discuss