Use artifact in second stage without publish

Hello,

I'm trying to set up a plan with stages which runs integration tests. I've set-up my structure as described in the Bamboo Best Practice - Using stages article.

  1. Stage 1: build & publish a shared artifact
    1. Default Job
  2. Stage 2: dependent on artifact, run test
    1. Test Job 1
    2. Test Job 2

I have multiple, separated integration tests projects which are allowed to run in parallel and if one fails, the other ones should be allowed to continue or start. This means that combining everything in 1 stage is not really an option, because if the build fails, none of them should run, but if one fails, the other ones should be allowed to run.

Since I'm using Bamboo Cloud, I have little to no control over where the artifacts show up or how fast the upload happens. And it's slow. Very slow. Unacceptable slow.

My artifact also contains some sample files. They are part of my project setup, required to run the integration tests and thus embedded in the artifact. The total size of my artifact is thus around 150 MB, which I can get down to 80 MB when I create a ZIP file instead of uploading the individual items.

But that still takes about 10-15 minutes to publish. And that doesn't work for me. This means the total time of a single test build takes around 20 minutes, where the actual build and the test run only take about 2 minutes. And that's not acceptable if you want to run them as part of a continuous build plan.

I came across BAM-15572 which mentions a possible work around by moving the instances to the US West region, but it still isn't ideal.

So I was wondering:

  • Is there a better strategy?
  • Can I improve the upload speed?
  • Can I somehow use the build result from stage 1 in the following stages without the need to publish the artifact?

1 answer

Instead of using the zip file as the artifact, instead push the artifact yourself to S3 (or some other fast/shared storage you have access to) via some scripting. Then write the URL to that artifact in a simple text file.  Use that text file as your artifact.  Then it's published quickly and you read the URL from the artifact in a script in further stages to pull the zip.

I have no direct experience with large artifacts myself, but I do publish a simple file as an artifact and I've never experienced slowness with it.

 

Good luck

Since I've moved the AMI to the US West region, my upload times have decreased. It only took 10 second to upload a 60 MB artifact, which at the moment is acceptable. But I will take your advice into consideration. At some point it would be advisable to keep artifacts around longer than 30 days (which I believe is the default Bamboo Cloud retention policy), so uploading it into my own storage container would be more advisable.

Suggest an answer

Log in or Join to answer
Community showcase
Renan Battaglin
Published May 18, 2017 in Bamboo

FAQ: How to Upgrade Bamboo Server

Bamboo 5.9 will no longer be supported after June 12, 2017. What does this mean? As part of our End of Life policy, Atlassian supports major versions for two years after the first major iteratio...

1,097 views 0 5
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you
Atlassian Team Tour

Join us on the Team Tour

We're bringing product updates and pro tips on teamwork to ten cities around the world.

Save your spot