Bamboo Deployment Timeout

Subhi Andrews August 28, 2019

We have a situation where we invoke Ansible Playbook command from Bamboo Deployment and on rare occasions, it just waits on one of the steps in the playbook, forever. Since the Deployment never fails or succeed, we don't get notifications. Things get queued for that Deployment environment we don't notice until there's some error or failure in Production. 

It seems there's a timeout available for Build jobs that are based on the history of how long it takes to complete the plan. Is there something similar available for Deployment Environments? How can I potentially solve this problem?

1 answer

0 votes
Daniel Santos
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 29, 2019

Hi @Subhi Andrews

I'll make some assumptions here:
    You are running the build in a Unix-like remote agent
    You are running the Ansible Playbook command from a script task.

Possible solution

  • Use the command timeout to define the maximum time you think the Ansible Playbook could take and still succeed.
  • Add the option set -e at the beginning of your script task. That will make the build fail for any command returning an error, including the result of timeout command when it kills a command after reaching the max time defined.

Example

timeout -k 30 <TIME> <ANSIBLE_COMMAND>

In case of timeout, the command above will send to Ansible the SIGTERM and if the command stays running for 60s more, then it will send the SIGKILL.


I hope this helps.

Subhi Andrews August 30, 2019

We have added ansible-playbook as a capability and calling it has a command instead of a script. Is there a solution for that? Since we might have to make changes in many different jobs?

Daniel Santos
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 30, 2019

Hi @Subhi Andrews

yes, you can use the Build monitoring feature, but that will consider the whole build time and will not work on local agents.

If you use remote agents that should work fine. =]

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events