Build Duration 369 minutes!

Question

Hi,

We have a build plan that usually takes less than 30 minutes. Sometimes it finishes successfully but only after a long delay (6 hours!) and the logs indeed show only the last 30 minutes, so it looks like the build itself ran normally. This seems to happen irregularly with random idle periods

How can I diagnose what happened during the first 5 and 1/2 hours?

We're using spot instances on AWS, with a 5 minutes timeout on the bid, but before I blame it on AWS, I'd like to know what happened as viewed by Bamboo.

How can I get more detailed information about spawn AWS instances, like:

* was the job running on a spot or regular instance?
* the time between requesting an instance and getting an instance
* the time between getting an instance and the bamboo agent responding
* any other logs on Bamboo side about scheduling of jobs..

Thanks

Arjun Panday · Answer

Thanks a lot, I sure hope that could be it.

By any chance, do you know when the next OnDemand update is planned?

Krystian Brazulewicz · Answer

Arjun, the situation you're describing fits this issue: https://jira.atlassian.com/browse/BAM-15187 Fix for this issue will be in the next Atlassian OnDemand (Cloud) update. Please verify if the problem is gone after the upgrade. If it is not then please contact our Support

Arjun Panday · Answer

Thanks Krystian,

Well, Bamboo reports a duration of 369 minutes in the build result summary, but the build logs show only 30 minutes of activity. The first log line says:

simple 23-Nov-2014 03:47:03 ... started building on agent Elastic Agent on i-xxx

and the build job completes at 23 Nov 2014, 4:14:04 AM

Now, I'm unclear whether the duration reported takes the queue time into account, so I'm not sure whether the job was in the queue for 6 hours or hanging in a "starting phase" somewhere in between.

Either way, with a max of 10 elastic intances running in parallel, I don't see how this could happen, unless all our builds are stuck for hours.

The Bamboo server is on Atlassian Cloud, not our own.