It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Use artifact in second stage without publish

Hello,

I'm trying to set up a plan with stages which runs integration tests. I've set-up my structure as described in the Bamboo Best Practice - Using stages article.

  1. Stage 1: build & publish a shared artifact
    1. Default Job
  2. Stage 2: dependent on artifact, run test
    1. Test Job 1
    2. Test Job 2

I have multiple, separated integration tests projects which are allowed to run in parallel and if one fails, the other ones should be allowed to continue or start. This means that combining everything in 1 stage is not really an option, because if the build fails, none of them should run, but if one fails, the other ones should be allowed to run.

Since I'm using Bamboo Cloud, I have little to no control over where the artifacts show up or how fast the upload happens. And it's slow. Very slow. Unacceptable slow.

My artifact also contains some sample files. They are part of my project setup, required to run the integration tests and thus embedded in the artifact. The total size of my artifact is thus around 150 MB, which I can get down to 80 MB when I create a ZIP file instead of uploading the individual items.

But that still takes about 10-15 minutes to publish. And that doesn't work for me. This means the total time of a single test build takes around 20 minutes, where the actual build and the test run only take about 2 minutes. And that's not acceptable if you want to run them as part of a continuous build plan.

I came across BAM-15572 which mentions a possible work around by moving the instances to the US West region, but it still isn't ideal.

So I was wondering:

  • Is there a better strategy?
  • Can I improve the upload speed?
  • Can I somehow use the build result from stage 1 in the following stages without the need to publish the artifact?

1 answer

Instead of using the zip file as the artifact, instead push the artifact yourself to S3 (or some other fast/shared storage you have access to) via some scripting. Then write the URL to that artifact in a simple text file.  Use that text file as your artifact.  Then it's published quickly and you read the URL from the artifact in a script in further stages to pull the zip.

I have no direct experience with large artifacts myself, but I do publish a simple file as an artifact and I've never experienced slowness with it.

 

Good luck

Since I've moved the AMI to the US West region, my upload times have decreased. It only took 10 second to upload a 60 MB artifact, which at the moment is acceptable. But I will take your advice into consideration. At some point it would be advisable to keep artifacts around longer than 30 days (which I believe is the default Bamboo Cloud retention policy), so uploading it into my own storage container would be more advisable.

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Bamboo

Unable to add or edit Bitbucket Cloud repository in Bamboo

On 31 May, a GDPR-related change went live in the Bitbucket Cloud API that resulted in users not being able to create or edit Bitbucket Cloud Linked repositories in Bamboo. This API update removed t...

307 views 2 6
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you