plan build is not using the wrong artifact download.

[Fred Kendall]

I have a plan build sequence.  B and C have a dependency on Plan A.  

Plan B and C should be using the last artifact set from A, but it is using 37 and not the fresh 38.  

The plans are in sequence.  B should not start until A is complete.  The fresh Artifacts from A are not available yet, thus Plan B is using the previous set of artifacts. 

This makes all of my sequential builds useless.  B and C should be using the fresh new A artifacts, not the older previous set. 

Bamboo Server is 6.5 build 60509

[Fred Kendall · Accepted solution]

We may try to process a first build as the first dependency, followed by a insignificant (not having dependencies)  plan between each of the subsequent builds with dependencies. This will allow us to kick off a set of builds in the evening a check them in the morning. 

[Eduardo Alvarenga]

Hello @Fred Kendall

Thank you for reaching out!

Were you able to reproduce that behaviour on a recent Bamboo release such as 7, 8 or 9? Bamboo 6.5 has set as end-of-life years ago and is not supported anymore.

Regarding your Plan, are you using Build dependencies? Depending on how your dependent plans require the artifacts you may have to explore some Dependencies Block strategies (probably not available on Bamboo 6.5)

Thank you,

Eduardo Alvarenga
Atlassian Support APAC

--please don't forget to Accept the answer if the reply is helpful-- 

[Fred Kendall]

We are a DoD facility and are not able to get updates safely.  We are using version 6.5.  I've checked the Dependencies Block strategies and I don't see how this would help us.  We have the individual plan builds being executed in the correct sequence on a single agent.  The agent will only process one plan at a time.  I've gone back and checked previous sequential build and I see the same problem behavior.   

We do not see the problem is we process each plan build individually and allow about 15 minutes between builds.  It seems that the artifacts directories and not identified as available/existing until after the next sequenced plan is initiated automatically. 

I tried to place a unique short 4 minute plan build between the dependency builds and still saw the problem.  I tried the Clean working directory task at the top of each plan build to allow a break between  the actual requirement for the dependent predecessor and the successor. This did not work.  

We may try to process a first build as the first dependency, followed by a insignificant (not having dependencies)  plan between each of the subsequent builds with dependencies. This will allow us to kick off a set of builds in the evening a check them in the morning.  

The incident that bit me in the ass was a two week old dependency was used in a new build.  Developers indicated that they had resolved the issue and that I must have use the incorrect dependencies.   I'm informed them that I was not a idiot and that I would not let that happen.  Well, I had too  eat my hat.  The Bamboo Artifacts had indeed been outdated.

Can you assure me that this is not an issues in updated versions of Bamboo?  I can make an attempt for an upgrade.

[Eduardo Alvarenga]

Hello @Fred Kendall

It seems there is something going on in the way you are triggering the builds. Can you describe your internal queuing process and how you actually trigger the Plans in sequence?

As the dependent plans are downloading previous artifacts. It may be the case of the second build and onwards are getting triggered whilst the first plan is still running. The artifact list will only get updated once the Plan run is finished, not during it.

The dependent plans will build the artifact list when triggered, so if there still are no new artifacts present, it will consider the last one available.

When you use build dependencies, Bamboo will guarantee that the dependent Plans will be triggered only when the initial Plan is completed.

So, we need to understand in detail how your Plan trigger and dependencies process work in case you are using any non-standard/outside approach.

There is a fixed bug that may be related to your issue, so it would be important to validate if a newer release will solve the issue.

•  https://jira.atlassian.com/browse/BAM-19985 

Best regards,

Eduardo Alvarenga
Atlassian Support APAC

--please don't forget to Accept the answer if the reply is helpful-- 

[Fred Kendall]

I take 10 minutes to queue up the jobs on a single agent.  The agent will only one plan build at at time.  We are using this thread to control the single series process flow.  It seems that the Artifacts are assigned at the time the plan is put in the queue.  Not taking an effort to check on the newest (fresh) artifact.  

The first dependency takes about 45 minutes to pull the source code and process the build.  It was assumed that the next build in the sequence would find the new artifact and use it.  This is not the case.  I'm assuming that the plans are triggered after the previous build is completed and the agent can trigger the next, next, next....

Do you have another way to automate the triggering of each build.  The single agent method is not working for us.

[Eduardo Alvarenga]

Hello @Fred Kendall

Thank you for the explanation.

It is still not clear how you QUEUE/Trigger the builds. Are you doing this manually for each Plan? Are you using Build dependencies?

Can you explain how the process kicks in and how you actually instruct Bamboo to start the builds and their dependencies? We are looking for the actual commands or configuration instructions by using Bamboo's internal methods or external ones.

Eduardo Alvarenga
Atlassian Support APAC

[Fred Kendall]

We are not using "Build Dependencies".

I have about 20 unique build plans in the Bamboo GUI interface.  The requirements, on the each of the build plans, limit the build to the use of a single build server using an agent. 

I select "Run" on the first dependency, Bamboo connects to the agent (on a remote Windows Server 2016 OS).  The first task in each plan is "Source Code Checkout".  It is directed to a repository for the source of code.  We have "Force Clean Build" Checked.  

A new directory is created on the remote agent and the source code is populated as expected on the remote agent build server.  

The next task is to run an ANT script to start the build on the agent.  The build completes successfully.  Bamboo copies the artifacts to the bamboo artifact directory, unique for this build plan.  A new directory is created JOB1\build-000006.  

When the Artifacts have completely copied from the agent server to the Bamboo artifacts directory, the build plan is completed.  The agent is now free to process the next build plan.   Again, the first Build Plan is complete (according to Bamboo). 

------------

Bamboo allows the second build plan, that was manually added to the queue from the Bamboo interface, to begin execution.  Note:  I've selected run on 8 build plans, they will only execute one at a time on the single agent build server.  I've gone home, anticipating that the Bamboo server will continue to process each build plan in the order that I've manually selected "Run"

Second manually started build plan, Bamboo connects to the agent (on a remote Windows Server 2016 OS).  The first task in each plan is "Source Code Checkout".  It is directed to a repository for the source of code.  We have "Force Clean Build" Checked.  

A new directory is created on the remote agent and the source code is populated as expected on the remote agent build server.  

The next task is to "Artifact Download".  The anticipation is that JOB1\build-000006 will be copied to the remote agent Windows 2016 Server.  This is not the case, It has copied JOB1\build-000005 files that were generated a few weeks ago.  

The next task is to run an ANT script to start the build on the agent.  The build completes successfully using the WRONG Artifacts.  Bamboo copies the second applications artifacts to the bamboo artifact directory, unique for this build plan.  A new directory is created JOB1\build-000032.

The third build plan runs and should be using artifacts from build plan 1 "JOB1\build-00006 and build plan 2 artifacts "JOB1\build-000032.  NOT THE CASE.  It has used plan 1's "JOB1\build-00005" and build plan 2's "JOB1\build-00031".  Third build plan completes successfully, but at been using the WRONG artifacts.  

At some point (about 4 build plans latter) The correct artifacts "Plan 1's JOB1\build-00006" and Plan 2's JOB\build-000032" are used.  

Bottom line is that the artifact sequence is not refreshing after a build is complete.  It is updated at some random time, post build.  

Yes,  Each build plan was run by Manually selecting "RUN".  Eight builds are added to the queue manually within at 10 minute windows.  I'm anticipating that when I return the office in the morning, that 8 hours of builds are completed successfully.

Office Phone: 719-554-5543

[Fred Kendall]

Note:  we are using Bamboo 6.5 which you do not support.  I need to know of the newer versions of Bamboo, will update the Artifact data sequence and flush the availability of Artifacts sooner (during the subsequent builds)

[Eduardo Alvarenga]

Hello @Fred Kendall

Thank you very much for the detailed explanation of your pipeline. It definitely helped us understand the big picture!

Bamboo queueing works based on the information it has during the queueing phase, not during the actual Plan build. That's why your Plans are referencing the previous artifacts as the "future" artifacts didn't even exist at that stage.

To get that sorted you will have to either:

1. Use Bamboo Plan build dependencies. This way the first build will trigger the dependent one and once queued, it will reference the latest artifact
2. Use external artifact storage. That will depend on you to store the files externally and reference them. You can use a remote, shared CIFS/NFS/SCP/etc filesystem or even a specific product such as JFrog Artifactory or Sonatype Nexus

We have just raised a feature request for that:

• BAM-21997 - Allow Artifact Download task to reference the latest available Artifact 

We encourage you to vote for it and add yourself as a watcher to be kept informed as to the state of the feature request moving forward. That way, if our development team updates the ticket, you'll be notified via email. You are also free to add a comment with your business-specific requirement so the developers can have a better understanding of the impact on your business.

There are a number of factors that determine how Atlassian prioritizes Feature requests. You can learn more about this by reading our Implementation of New Features Policy.

If you have any questions about the feature request or our feature request implementation policy, please let us know.

Kind regards,

Eduardo Alvarenga
Atlassian Support APAC

--please don't forget to Accept the answer if the reply is helpful--