Build had to be cancelled: it was marked as in progress in DB

[Tech Noboundary]

Bamboo 8.1.3

CPU 16 cores

RAM 16GB

OS Windows Server 2016

Recently, we experienced a good number of failures with this error "Build XXXX had to be cancelled: it was marked as in progress in DB".

I have manually updated BUILDRESULTSUMMARY table with 

• LIFE_CYCLE_STATE = 'NotBuilt'
• BUILD_STATE = 'Unknown'
• DELTA_STATE = 'None'
• and valid BUILD_CANCELLED_DATE and BUILD_COMPLETED_DATE

Somehow, Bamboo is still constantly query the DB and showing the build in progress.  This blocks builds. 

I get through the DB schema.  It is not clear how to stop the query and the wrong report.  Would like to ask our community for help. 

Regards,

[Eduardo Alvarenga]

Hello @Tech Noboundary 

Making changes directly to the database is not supported and not recommended at the risk of data corruption. We advise you to roll back the changes the soonest. Make sure to do it whilst Bamboo is not running.

As your issue require further investigation, we advise you to open a ticket with Atlassian at https://support.atlassian.com/contact.

Please do not post any internal information to this forum as it is public.

Cheers,

Eduardo Alvarenga
Atlassian Support APAC

--please don't forget to Accept the answer if the reply is helpful-- 

[Tech Noboundary]

Thanks.  We encountered this issue rather frequently (1 or 2 per week).  What could be the root cause?  We have been scaling up with Bamboo quickly now and will hit 500 agent level this year.  This can be a big issue for us. 

[Eduardo Alvarenga]

Hello @Tech Noboundary

Check the following KB article for some possible causes for the issue:

• Build <key> had to be cancelled: it was marked as in progress in DB but Bamboo has no record of this build

Additionally, monitor your server's CPU and memory usage. It is also recommended to enable Garbage Collection logging so you can monitor the JVM memory consumption:

• Monitor Memory usage and Garbage Collection in Bamboo 

Finding the exact root cause will require investigation. For that please raise a ticket with Atlassian.

Cheers,

Eduardo Alvarenga
Atlassian Support APAC

--please don't forget to Accept the answer if the reply is helpful-- 

[Tech Noboundary]

Thanks.  I instructed an engineer to reach out to Atlassian tomorrow for the direct support.  The issue blocks two teams on their develop branches now. 

We are on 8.1.3 which is well above 5.11.3.  Sounds that we shall not hit this issue, right? 

I will enable garbage collection logging. 

[Tech Noboundary]

Will a restart of our Bamboo service recover this issue?