Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,325,435
Community Members
 
Community Events
168
Community Groups

Pipelines repository settings: graceful shutdown / only admin can stop pipeline

Edited

Our use case

In our system anybody can start a pipeline and approve a deployment. A deployment to production can be manually approved after a series of robust tests and build phases across multiple branches.

We have a continuous deployment system running in Bitbucket Pipelines, and the last step is performed by Terraform. Terraform provisions our infrastructure as it is defined in code across our repositories, and programmatically makes changes to our infrastructure if there are any differences in configuration.

 

Problem statement

Terraform, along with many other deployment systems, can only operate one instance/session per environment at a time. This is facilitated by a remote locking mechanism which is read/written to by all deployers and pipelines.

If a Bitbucket Pipelines pipeline step is "stopped", it does not gracefully shut-down. Ordinarily Terraform will be gracefully stopped (for example if Ctrl+C is hit in terminal) and the remote lock is released.

With Bitbucket Pipelines, anybody can force immediate termination of the pipeline script which results in our infrastructure half-updated and the state remains locked.

 

Solutions

This problem is not limited to Terraform, it applies to any deployment system which relies on a locking mechanism. Below are my suggestions for resolving this problem:

  1. Repository Settings -> Pipelines -> Toggle: "Stop" will issue a shutdown command to the process, consuming build minutes until the process terminates.
  2. Repository Settings -> Pipelines -> Toggle: Only admins can stop a pipeline. 

 

If anybody else is experiencing a similar issue, has a workaround, or otherwise has any feedback, then please contribute to this discussion :) 

1 comment

we have the same issue with our deployment pipeline. We are using "helm" to deploy services to kubernetes and if someone is stopping the pipline during "helm update" then the process terminates and causing problems for further deployments because of "pending-install" state.

It would be great to have a possibility within bitbucket-pipelines to have something like a "shutdown-hook" to be able to react on the case that someone stops the pipeline in the middle of working, or at least a way on how to disable the "stop" button for some pipelines.

Comment

Log in or Sign up to comment
TAGS
Community showcase
Published in Bitbucket

Git push size limits are coming to Bitbucket Cloud starting April 4th, 2022

Beginning on April 4th, we will be implementing push limits. This means that your push cannot be completed if it is over 3.5 GB. If you do attempt to complete a push that is over 3.5 GB, it will fail...

2,873 views 3 9
Read article

Atlassian Community Events