Bitbucket pipeline: Can we re-use/import stage in branches block

Mrunal N

Our query is regrading Bitbucket pipeline.

We have defined steps(install,build,test and deply) in a stage under definition. Now under pipelines -> branches we want it to be called/re-use. But seems branches requires steps only because we are getting following error while triggering the pipeline:

There is an error in your bitbucket-pipelines.yml at [pipelines > branches > {develop,staging} > 0 > stage > steps > 0]. The step section is empty or null.

 

Pipeline code for reference:

# Default base image : pipeline wide

image: node:15.1.0-alpine

 

# pipeline wide definations

definitions:

  # stage - to group different steps of process

  stage: &install-test-build-deploy

    name: install-test-build-deploy

    runs-on:

      - self.hosted

      - linux.shell

    caches:

      - node

    artifacts:

      - build/**

    steps:

      - parallel:

          steps:

            - step: &lint

                name: Lint

                script:

                     # Run your linter of choice here

                     - npm install eslint

          - step: &install

                name: install

                script:

                    - npm install

 

      - step: &build

           name: build

           script:

              - npm run build

 

      - step: &test

           name: test

           script:

              - npm run test

 

      - step: &deploy

           name: deploy

           script:

              # update s3 bucket with latest changes via sync command

              - aws s3 sync --delete build s3://${BUCKET}

 

              # update cdn content by creating invalidation

              - aws cloudfront create-invalidation --distribution-id ${DIST} --paths "/*"

 

# Pipeline workflow Configuration

pipelines:

  default:

    - step:

        <<: *lint

  branches:

    '{staging}':

      - step:

          <<: *install-test-build-deploy

          deployment: staging

    '{qa}':

      - stage:

          <<: *install-test-build-deploy

          deployment: qa

    '{production}':

      - stage:

          <<: *install-test-build-deploy

          deployment: production

          # trigger: manual

Please guide...

Theodora Boudale

Hi @Mrunal N,

There are some syntax errors in the definition of the stage. Specifically:

1. Right after the keyword definitions, you need to define the stage as follows:

definitions:

stages:
# stage - to group different steps of process
  - stage: &install-test-build-deploy

2. Caches and artifacts should be added to the step that generates them and not to the stage.

3. Parallel steps are not supported yet on a stage (we do have a feature request that you can vote for here https://jira.atlassian.com/browse/BCLOUD-22214)

4. The runs-on definition needs to be specified on the step level and not on the stage.

If you want all of your steps to run on a runner, you need to add this definition to all steps.

If caches and artifacts are generated during the first step, if you want to run all steps on a self.hosted runner, and if your stage includes all 5 steps, then a valid stage definition could look as follows:

definitions:
  stages:
  # stage - to group different steps of process
    - stage: &install-test-build-deploy
        name: install-test-build-deploy
        steps:
          - step: &lint
              name: Lint
              runs-on:
                - self.hosted
                - linux.shell
              script:
                # Run your linter of choice here
                - npm install eslint
              caches:
                - node
              artifacts:
                - build/**
          - step: &install
              name: install
              runs-on:
                - self.hosted
                - linux.shell
              script:
                - npm install
          - step: &build
              name: build
              runs-on:
                - self.hosted
                - linux.shell
              script:
                - npm run build
          - step: &test
              name: test
              runs-on:
                - self.hosted
                - linux.shell
              script:
                - npm run test
          - step: &deploy
              name: deploy
              runs-on: 
                - self.hosted
                - linux.shell
              script:
                # update s3 bucket with latest changes via sync command
                - aws s3 sync --delete build s3://${BUCKET}
                # update cdn content by creating invalidation
                - aws cloudfront create-invalidation --distribution-id ${DIST} --paths "/*"

I see that you have used an image definition at the top of your yml file (image: node:15.1.0-alpine). Please keep in mind that if you use a Linux Shell Runner for all steps, then the builds do not run in Docker containers and the image doesn't make a difference.

Linux Shell Runners use Bash to run pipeline steps on your Linux machine (host device). This allows the runner to execute applications on the host, but does not provide a clean build environment for every step. Any side effects generated by the step (such as installing any applications, starting a database service, or editing a file outside of the build directory) would potentially affect the next step to be run (including new pipeline runs). To compensate for this, the runner tries to empty the build directory empty after each step. It is your responsibility to make sure the scripts you run in each step won’t have a major impact on other steps.

If you want to run the builds in Docker containers, you can look into configuring a Linux Docker Runner instead:

• https://support.atlassian.com/bitbucket-cloud/docs/set-up-and-use-runners-for-linux/

Kind regards,
Theodora