Pipeline with deployments and several manual steps.

Question

Hi everyone,

I have been struggling for a while to understand how to write a pipeline for this seemingly simple scenario:

We have two environments (deployments): staging and production.

Each of them have 3 independent steps to run, all of them to be triggered manually and independently (because we may need to time them based on daily traffic patterns  and/or sometimes we don't want to run all steps in that deployment).

So it looks like something like this:

Test, build  & etc       ->        Staging             ->       Production

| -> Step1 (manual)       | -> Step1 (manual)

| -> Step2 (manual)       | -> Step2 (manual)

| -> Step3 (manual)       | -> Step3 (manual)

I have tried with stages, but that seems to be not possible because you cannot have manual steps within the stage.

I also tried to use parallel to be able to run each manual step independently each from other, but then I cannot use "deployment" tag because they are not allowed to be repeated.

Not sure what I am missing. I am coming from GL where this was fairly easy to be set.

Thanks in advance.

Theodora Boudale · Answer

Hi @aiviso,

I'm afraid that this is not possible with a single pipeline. Stages are the way to have multiple steps with the same deployment environment in a single pipeline, however, you can't have a manual trigger in a stage's step or skip a step in a pipeline to run the steps independently. This is why I recommended a custom pipeline for each of these steps:

* https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipelines

An example yaml file would be the following:

pipelines:
  branches:
    master:
      - step:
          script:
            - echo "Test & Build"
  custom:
    staging-1:
      - step:
          deployment: staging
          script:
            - echo "Staging 1"
    staging-2:
      - step:
          deployment: staging
          script:
            - echo "Staging 2"
    staging-3:
      - step:
          deployment: staging
          script:
            - echo "Staging 3"
    production-1:
      - step:
          deployment: production
          script:
            - echo "Production 1"
    production-2:
      - step:
          deployment: production
          script:
            - echo "Production 2"
    production-3:
      - step:
          deployment: production
          script:
            - echo "Production 3"

This way the branches: master pipeline will run automatically when there are new commits on master branch.

You can then trigger any of the custom pipelines you want independently.

You can trigger a custom pipeline in one of the following ways:

* On the Pipelines page of the repo, there is a Run pipeline option at the top (next to Schedules). You can select a branch and then the custom pipeline you want to run.
* While viewing the Branches page of the repo, you can select the 3 dots under the Actions column for a branch > Run pipeline for a branch.

This option and the previous one will run the custom pipeline for the latest commit of the branch. If you want to run the custom pipeline for a previous commit of master and not the latest one. see the next option.

* You can trigger it for a specific commit if you open the commit on Bitbucket website and select Run pipeline from the right sidebar (at the bottom)

Kind regards,
Theodora

aiviso · Answer

Hi @Theodora Boudale , thanks for the answer. The question, though is still how to use environments in these steps? As far as I can see I can only define each environment once in the pipeline, correct? So if I have 3 steps for staging and 3 steps for production, that will not gonna work?

Besides that, I want to trigger the pipeline and execute 'test & build' stage on branch.

Something like this:

pipelines:
  branches:
    master:
      - step:
          script:
            - echo "Test & Build"
      - parallel:
          steps:
            - step:
                deployment: staging
                trigger: manual
                script:
                  - echo "Staging 1"
            - step:
                deployment: staging
                trigger: manual
                script:
                  - echo "Staging 2"
      - parallel:
          steps:
            - step:
                deployment: production
                trigger: manual
                script:
                  - echo "Production 1"
            - step:
                deployment: production
                trigger: manual
                script:
                  - echo "Production 2"

but that is not possible because of repeated deployment clause with the same value. But again, I cannot move deployment clause to parallel because parallel does not support it.

Not sure how to solve it. Seems to be fairly simple use case :(

Theodora Boudale · Answer

Hi @aiviso,

If you want all six steps to be triggered manually and independently from each other, you can create a custom pipeline for each of these steps:

* https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipelines

You can then execute only the custom pipelines that you want.

If a step in a pipeline is manual, then all the previous steps of this pipeline need to complete successfully, in order to trigger this specific step.

We have a feature request for the ability to skip steps that you can vote for and watch: https://jira.atlassian.com/browse/BCLOUD-20833

And another one for manual steps in a stage: https://jira.atlassian.com/browse/BCLOUD-22223

In the meantime, custom pipelines would allow you to trigger only the steps you want.

Please feel free to reach out if you have any questions.

Kind regards,
Theodora