Pipeline with parallel manual steps, each with success-dependent subsequent steps

[Steven Hayhurst]

I want to build a pipeline with a step structure like the following:

• build package
  • validate package against sandbox 1
    • deploy package to sandbox 1
  • validate package against sandbox 2
    • deploy package to sandbox 2

• Build package is the automatic step.
• I would like the subsequent four steps to be manually triggered.
• The validate jobs should be able to be run in parallel with any other job of any type (validate or deploy)
• A deployment job should only be able to be run if it's validate step passed successfully
• The deployment jobs should also be able to be run in parallel; that is, alongside any of the other deploy/validate jobs
• Each validate/deployment pair of jobs should be able to be run regardless of other validate/deployment steps failing, ie I should be able to validate and deploy to sandbox 1 regardless of whether validating/deploying to sandbox 2 is successful or not

I've used this kind of structure previously with Azure (I didn't build it) but I can't figure out if its possible here.  Am I going to have to split this out to multiple pipelines?  I thought I would be able to do this when I found out about after-script, but seems I can't put a step inside it.  Here's what I thought I might be able to do:

I should add that I am completely new to yaml and pipelines in general, literally started using it yesterday

pipelines:

  pull-requests:

    default:

      - step:

          name: 'Generate deployment packages'

          script:
            ...

      - parallel:

        - step:

            name: 'Validate Against Sandbox 1'

            trigger: manual

            script:
              ...

            after-script:

              - step:

                  name: 'Deploy to Sandbox 1'

                  trigger: manual

                  script:
              if [[ BITBUCKET_EXIT_CODE -eq 1 ]]; then exit 1; fi
                    ...

        - step:

            name: 'Validate Against Sandbox 2'

            trigger: manual

            script:
              ...

            after-script:

              - step:

                  name: 'Deploy to Sandbox 2'

                  trigger: manual

                  script:
              if [[ BITBUCKET_EXIT_CODE -eq 1 ]]; then exit 1; fi
              ...

I also thought Stages might help, but you can't manually trigger stage steps :(

Thanks

[Theodora Boudale]

Hi Steven and welcome to the community!

I'm afraid that this cannot be done with a single pipeline at the moment.

The validate jobs can be parallel and they can be executed independently. However, the after-script is not a separate step and it cannot be triggered manually, it will always run when the step finishes.

You can look into using different custom pipelines:

• https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipelines

A custom pipeline does not run automatically, but it can be triggered manually. You can see on this page how you can run a custom pipeline manually:

• https://support.atlassian.com/bitbucket-cloud/docs/pipeline-triggers/

You could have:

• a custom pipeline for the build job
• a custom pipeline for the validate and deploy to sandbox 1 (with two steps)
• a custom pipeline for the validate and deploy to sandbox 2 (with two steps)

Please keep in mind that these can be triggered independently, so someone could trigger the last custom pipeline without the first one having been executed.

It is also important to take into account that custom pipelines cannot be triggered on pull requests. They can be triggered only for a branch or a specific commit.

Please feel free to reach out if you have any questions.

Kind regards,
Theodora

[Steven Hayhurst]

Thank you for the response.  Since you put "at the moment" in your response, can I read into that that it might be something possible in future?  This is the sort of setup I've used previously in a previous company who used Azure so it's definitely possible in other products.

Thanks again

[Theodora Boudale]

Hi Steven,

This setup should be possible if we implement the following two features:

• Support parallel stages (feature request: https://jira.atlassian.com/browse/BCLOUD-22215)
• Support manual trigger for a step in a stage (feature request: https://jira.atlassian.com/browse/BCLOUD-22223)

With these two features implemented, it should be possible to do the following:

• Use a stage for each pair of validate and deploy jobs
• Make the stages parallel so they can run independently
• Add a manual trigger to the deploy jobs

You can add your vote to both feature requests (by selecting the Vote for this issue link) to express your interest. You are more than welcome to leave any feedback there, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

Implementation of features is done as per our policy here and any updates will be posted in the feature requests.

Kind regards,
Theodora