I'm trying to set up a plan with stages which runs integration tests. I've set-up my structure as described in the Bamboo Best Practice - Using stages article.
I have multiple, separated integration tests projects which are allowed to run in parallel and if one fails, the other ones should be allowed to continue or start. This means that combining everything in 1 stage is not really an option, because if the build fails, none of them should run, but if one fails, the other ones should be allowed to run.
Since I'm using Bamboo Cloud, I have little to no control over where the artifacts show up or how fast the upload happens. And it's slow. Very slow. Unacceptable slow.
My artifact also contains some sample files. They are part of my project setup, required to run the integration tests and thus embedded in the artifact. The total size of my artifact is thus around 150 MB, which I can get down to 80 MB when I create a ZIP file instead of uploading the individual items.
But that still takes about 10-15 minutes to publish. And that doesn't work for me. This means the total time of a single test build takes around 20 minutes, where the actual build and the test run only take about 2 minutes. And that's not acceptable if you want to run them as part of a continuous build plan.
I came across BAM-15572 which mentions a possible work around by moving the instances to the US West region, but it still isn't ideal.
So I was wondering:
Instead of using the zip file as the artifact, instead push the artifact yourself to S3 (or some other fast/shared storage you have access to) via some scripting. Then write the URL to that artifact in a simple text file. Use that text file as your artifact. Then it's published quickly and you read the URL from the artifact in a script in further stages to pull the zip.
I have no direct experience with large artifacts myself, but I do publish a simple file as an artifact and I've never experienced slowness with it.
Since I've moved the AMI to the US West region, my upload times have decreased. It only took 10 second to upload a 60 MB artifact, which at the moment is acceptable. But I will take your advice into consideration. At some point it would be advisable to keep artifacts around longer than 30 days (which I believe is the default Bamboo Cloud retention policy), so uploading it into my own storage container would be more advisable.
Atlassian Summit is an excellent opportunity for in-person support, training, and networking.Learn more
...there's anything I've learnt from working, it's that people are lazy! No offense to anyone reading this, but it's true and we can all admit it. The easier you make something for someone, the more...
Connect with like-minded Atlassian users at free events near you!Find a group
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no AUG chapters near you at the moment.Start an AUG
You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs