Bitbucket Pipeline Paused the pipeline

[venkatcss]

Hi Team,

I have setup the Bit bucket pipeline in my environment .

When multiple users commits  one by one .. Second commit  pipeline is going to  pause  mode.

 first commit pipeline is only running.. But here I want to run the latest commit pipeline need to run..

I hope you got it my problem.. Please help me on this issue.

Bitbucket-pipeline.yml

---
image: "node:10.15.0"
pipelines:
branches:
development:
-
step:
deployment: test
name: "install and build"
caches:
- node
script:
- "apt-get update -y"
- "apt-get install -y zip"
- "cd admin/front-end"
- "npm install"
- CI=false
- "npm run build"
- "zip -r /tmp/artifact.zip *"
trigger: automatic
artifacts:
- admin/front-end/build/**
-
step:
image: "python:3.5.7"
name: test
caches:
- pip
script:
- "apt-get update -y"
- "pip install boto3==1.9.197"
- "apt-get install -y zip"
- "zip -r /tmp/artifact.zip *"
- "python codedeploy_deploy.py"

Thanks & regards,

venkat

[Rafael Pinto Sperafico]

Hi @venkatcss ,

Pipelines will automatically check if there is a deployment in progress before starting a new one to the same environment. If there is already a deployment in progress, later pipelines deploying to the same environment will be paused. 

More information can be found on Deployment concurrency control.

Kind regards,
Rafael

[venkatcss]

Hi rsperafico,

Thanks for your reply Yes I agreed it's happening for me.. But if it's happens like this my 

It will causes problems for my all code.. so I want to run latest commit pipeline .

is it possible or not?

But as per your information after paused the pipeline do i need to resume the pipeline manually?

Regards,

Venkat

[Rafael Pinto Sperafico]

Hi @venkatcss ,

As the documentation suggests (Deployment concurrency control), you can manually:

1. Rerun the whole pipeline from the beginning
2. Resume the paused pipeline at the step it became paused

When setting up bitbucket-pipelines.yml, you can define if the build will get trigger automatically or manually. Based on your YAML, builds will be triggered automatically getting the latest commit from your development branch.

The build is based on a commit, so in the screenshot you have provided the build will be executed against commit 3ff2c6f regardless if there are new commit.

Perhaps you should consider making use of tags, running builds only when a tag is created or updated. In this way, you will define what should run and what should not.

Please, refer to the following documentation for further information:

• 2 additional ways to trigger your builds in Pipelines
• Get started with Bitbucket Pipelines
• Run pipelines manually

Kind regards,
Rafael

[Shawn Hempel]

@Rafael Pinto Sperafico wouldn't it make sense to automatically resume after the current deployment completes?

In a Continuous Integration model, our expectation is that every commit will eventually make it to the integration environment without any manual intervention.

It should not be necessary to use tags or manually resume the pipeline. The logic we (and I suspect most people) would want is for concurrency to automatically resume the pipeline after the deployment completes.

[Rafael Pinto Sperafico]

Hi @Shawn Hempel,

In a Continuous Integration model, our expectation is that every commit will eventually make it to the integration environment without any manual intervention.

The eventually in comment above suggests that not every commit will result on a build, due to that, there are already exceptions being made.

Not every commit should trigger a build, only desirable ones. For instance, someone reviewed your source code asked you or one developer to comment an implemented method. This would result on a new commit, therefore, there would not be any difference between an existent build result and this new commit.

If the build without comments took 100min, now with comments added to the implemented method this new commit would be using another 100min from Standard or Premium plan you may have.

...The logic we (and I suspect most people) would want is for concurrency to automatically resume the pipeline after the deployment completes.

Resuming after the deployment completes shall no longer be concurrency.

When running a concurrent deployment, you or another developer could be overriding resources that should be consumed by your build, causing undesirable result(s), making it difficult to track.

Not to mention, if your build resulted in failure, the concurrency you have suggested could consume undesirable minutes from your plan.

Hope the above shed some light.

Kind regards,
Rafael

[timaev]

Good day @Rafael Pinto Sperafico, let me describe a situation for which some automatic deployment concurrency control could be desirable. When someone from the project team merges multiple PR's at once or several of us doing it simultaneously there will only be one arbitrary commit reaching and completing the deployment and in most cases the most recent commit will get its deployment step paused. Afterwards we always have to remember to manually check and resume most recent PR deploy.

Something like a following approach could be nice: if a deployment step detects same deployment already active then enter into pending state and once the active one finishes begin the queued deployment but only if the given commit is the HEAD. Right now we have a hack to do something like that with:

master:
 - step: ...
 - step: ...
 - step:
  name: Ensure most recent commit
  script:
   - git fetch
   - '[ `git rev-parse origin` = `git rev-parse HEAD` ]'
 - step:
  name: Deploy to production
  deployment: production
  script:
   - ...

[Shawn Hempel]

@Rafael Pinto Sperafico I'm fine with wasting minutes in our plan due to a failed build. That's our fault if it happens - we don't need our deployment pipeline to protect us from ourselves.

Our builds take 3 to 6 minutes. I don't expect that will change significantly.

Not every commit should trigger a build, only desirable ones.

This concept is anathema to CI/CD. All commits to the CI branch are desirable and should be built. Otherwise they should not have been committed to the CI branch.

I think I didn't describe the desired model very well.

I don't want every individual commit to result in its own individual build. I want every commit to eventually be included in a build (without manual intervention.)

So for instance, if I commit right now a build/deployment will be triggered. If my colleague commits one minute later, the in-progress deployment should be allowed to complete and then immediately once it finishes, a new pipeline should start which includes all commits since the previously deployed commit.

It seems to me the obvious solution would be to hold off on starting a new pipeline until the in-progress deployment completes. However I recognize that pipelines and deployments are somewhat disconnected, so that may not be feasible.

The arguments you've raise are reasonable from a certain perspective, but none of them really apply to us. As such, the inability to opt-in to a behavior we want is proving frustrating. Pipelines is a terrific product and I would very much like to keep using it. Hopefully a way to support this fairly simple model can be conceived. Certainly many other CI/CD tools have found a way.

[Rafael Pinto Sperafico]

Hi @Shawn Hempel and @timaev ,

Thank you for the information provided.

I would encourage you to comment on https://jira.atlassian.com/browse/BCLOUD-16304 - queuing and automatic resuming of paused deployment steps, as it seems to describe the need you have demonstrated on this thread.

Kind regards,
Rafael

[Pierre Herman]

A quick comment from a lambda user.
I find pretty disturbing the a paused pipeline stays in the "paused" state indefinitely... I see pipelines "paused" from weeks ago. We should be able to cancel that and change its state to "cancelled" or "abandoned". What would happen is someone resumes a paused pipeline when dozens of others have already been run successfully?

[sravani.kambhampati]

I am looking for a way to run the paused pipelines.  We don't have 

any option to pause the failed builds

getting this while trying to run manually:

the pipeline ran with temporary variable values so we can not re run bitbucket pipelines

[ryan_boog]

There are great suggestions on https://jira.atlassian.com/browse/BCLOUD-16304 but they are being ignored. This issue is bigger than you think. What can users do to bring more attention to this so some sort of action is taken?

[john.lister]

We are using Mono-Repos, so a single commit can potentially trigger multiple distinct deployments, but Bitbucket has no concept of splitting or understanding these. We currently therefore have to have a single step that determines the changed components and deploys them all.

Even with a standard repo, I'd reiterate the above comments, if I have a deployment in progress and someone else commits that triggers a deployment I'd definitely expect that to start once the previous one finishes. Other CI/CD pipelines (azure for example) perform this way. Clearly deploying at the same time is bad, but once one is finished the next one queued should start. As previously said, of the first deployment one fails and causes the next one(s) to fail , then that is my responsibility to resolve

[Cord Slatton]

At least send a notification if a pipeline is paused so it can be looked into.

[psychobolt]

For my monorepo setup, I decided to use dotenv-vault (https://www.dotenv.org) and dotenv-cli for running my scripts. This allows me to configure my pipeline to not run into the parallel limitations that bitbucket imposes. I also find the flexibility of sharing secrets locally and on the CI per deployment in this approach.