Asynchronous build steps

[Sebastian_R]

Hi, with the upcoming macOS notarization requirement for mac applications, I am wondering how such a build step could look like in Bamboo.

The requirement for this build step is as follows: A zip file with all binaries gets send to Apple. An associated id, given by Apple, can be used to verify the status of the notarization. Once this step is done, the binaries can be modified with the resulting ticket and they are ready to ship.

The notarization session might take up several hours but is asynchronous. So I think one solution would be that a mac client starts compiling, sends the binaries to apple and continues working on another build. Once the notarization is done, the mac client can continue the build steps form the first build. Is that possible?

Any ideas on this is highly appreciated!

[Daniel Santos]

Hello @Sebastian_R

Thank you for carefully explaining the scenario you got into.

I'm not sure if what I'll share here is the best option to move forward,  but I wanted to share my 2 cents with you.

If you keep your build running, the agent handling it will not be able to take new builds, as you already know. It means that the best approach would be running the build in two parts:

• One before the notarization process
• Another after the notarization was processed

You could build the first part, starting the process with Apple and then notifying Bamboo when this process is done so the next part of the build could be processed.

Assuming you are running a build, not a deployment, this is what I would suggest:

1. Develop a daemon script to receive notarization requests from Bamboo and notify Bamboo back when the notarization process is done.
   The daemon would:
   1. Save the build key and the apple Notarizing ID
   2. Check if the request is processed, using what is shared here: Customizing the Notarization Workflow | Apple Developer Documentation.
   3. Resume the build using REST API and a custom variable to share the data that needs to be consumed in the next build stage

      curl --user <USER>:<PASSWORD> -X PUT <BAMBOO-URL>/rest/api/latest/queue/<PLAN-KEY-BUILD>?<BAMBOO_PLAN_VAR>=<VALUE>

2. Split the build into stages (automatic and manual):
   1. The first part of the build with AUTOMATIC stagesThe last stages of this group should be responsible for starting the Notarization process and sending the details to the daemon (build key and the apple Notarizing ID).
      The build key can be build using the following variable expression: 

      $bamboo_planKey-$bamboo_buildNumber

      Reference: Bamboo variables - Atlassian Documentation
      The Notarizing ID will come from the request to Apple.
   2. The second part of the build has MANUAL stages
      This stage and the next ones will process the part of your build after apple notarizing process.

I hope this idea can move you forward or at least help you to think on different approaches to build your pipeline.

[Sebastian_R]

Sorry for the late reply. Thanks a lot for the ideas! I will try them out

[Daniel Santos]

You are welcome! =]

[Jimmy Seddon]

Thanks @Daniel Santos! I completely forgot about using stages, this actually helps me out with a different issue I'm trying to solve.

[Daniel Santos]

That is great to hear! I'm happy to help.

[kiranpatel11]

@Daniel Santos

What is the best way to tackle aysnc task during deployment ?

My scenario is to trigger a perf test post deployment of the application, if the perf test is concluded successful via callback, I would like to mark the deployment for that env sucessful, and trigger a deployment to next environment.

[Daniel Santos]

Hi @kiranpatel11

I have very limited time to investigate this, now that I'm on a different project, but I'll try to engage someone to give you insights on this.

[Gerhard Forster]

Hi @kiranpatel11,

Please allow me to jump in for my colleague as I tested out a couple of thing locally and think that I have quite a nice solution for you. At least it works well for me.

Firstly, let me wrap up what I understand that your use case looks like. So I believe you have this situation:

A)  Build (or builds, eventually chained per Bamboo's plan dependency feature).
B)  Deployment to a QA environment.
C)  Wish to run further tests, you mention perf test here.
D1) On success of this / these test(s) => deploy the artifact from (A) to another, let's call it production environment.
D2) On fail => NOP, so just do nothing further.

I hope that this describes well what you aim to achieve. So here an approch of a solution - and there are a bunch of alternatives around, I'd say.

A -- B) In your build plan(s) you set build triggers just as needed for the build, this might be polling or better commit triggers set up for your connected repositories.
You build / produce there what's necessary and share its result as an artifact directly with both deployment environments, so QA and production per an Artifact Download task.
-- B--) In your QA deploymenyt environment you set deployment trigger "After successful build plan". By that, any build with success will kick of the defined tasks of this QA deployment and per the rules of this deployment project's Release versionsing crerate an appropriate release for QA (assuming that none of the tasks fail).
B -- C) Within your production depolyment environment you configure another deployment trigger, namely "After successful deployment" and you set it as being triggered by your QA environment. That said, whenever the QA deployment terminates with success, this new deployment is automatically kicked off.
-- C --) Now you run the test(s) that you want to conduct in this production deployment environment, be it per Bamboo Script or any other dedicated task(s).
-- D1 -) If this / these test task(s), in your case perf, is / are succeeding, it will directly use the artifact that's shared with it, the same that got shared with the QA environment and deploy it according to the same Release versioning scheme as QA did before.
-- D2 -) If this or these task(s) failed, the whole deployment would turn a FAIL and nothing would be relaeased. So in other words, noghing had happend.

You see that in this case study I made use of some triggers. To add to the above said, Bamboo comes along with a set of Deployment triggers and a bunch of Build triggers. Triggers are a great means of stimultating a chain of activities. Right as you see that above.

Now when you study these two KBs from above, you'll find that one trigger is missing and this missing one might be essentail. Also for your senario, as you might want to return to a build plan once your QA deployment succeeded, there run the perf test and on success of this (and only then) trigger off your second deployment, now to your production environment.

What's "missing" (and you'll see in short that it's not missing actually) is a "Trigger plan after successful deployment". We provide this currently as a Bamboo plugin that you need to install from here: Atlassian marketplace -- After Deployment Trigger for Bamboo. Once done, you can use it just like any other Bamboo trigger.

See here a feature request BAM-14878 -- Add ability to trigger plans from Deployment Projects so to make this trigger find its way into a future Bamboo release. Please set yourself as watcher on this suggestion ticket and/or vote for it. Note that suggestions as well as bugs are directly monitored by our Devs. So your vote will clearly help increase visibility for your wish and need of having that feature integrated to Bamboo.

Please quote this message with "Accept answewr" if it was of help for you and if not, just share your concern or any further ask with us, preferably in this same thread.

Thanks a lot and cheers,
Gerhard

[kiranpatel11]

Thanks Gerhard.