How can I define reusable scripts and run them in parallel for a branch

Eric Poe September 16, 2024

Given the following pipeline configuration, the Pipeline Validator claims it is good but the pipeline gives an error: `[pipelines > branches > FOO-10751-snapshot-used-as-clean-source > 0 > parallel > steps > 0]. The step section is empty or null.`

  • Am I wrong? How can I fix this?
  • Is the pipeline runner wrong?
  • Is the Pipeline Validator wrong?

 

definitions:
steps: # Note: the "&" is a yaml anchor that allows reusing this particular step
- step:
name: &PHP-Code-Quality
image: foo/php-ci:1
script:
- |
php bunch-of-stuff.php
- step:
name: &JS-Code-Lint
image: node:18
script:
- |
npx bunch-of-stuff
- step:
name: &Update-Snapshot
runs-on:
- server.test
- self.hosted
- linux.shell
script:
- bunch-of-stuff.sh
pipelines:
default:
- parallel:
steps:
- step: *PHP-Code-Quality
- step: *JS-Code-Lint
- step: *Update-Snapshot
branches:
FOO-10751-snapshot-used-as-clean-source:
- parallel:
steps:
- step: *PHP-Code-Quality
- step: *JS-Code-Lint
- step: *Update-Snapshot

 

Eventually, I will want to use a specific branch for staging a release via Pipelines. This is a preliminary step towards that goal.

2 answers

1 accepted

2 votes
Answer accepted
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 18, 2024

Hi Eric and welcome to the community!

There is a bug with the validator and it doesn't show any errors when yaml anchors are used.

I think the culprit for the error you see is that you are using the anchors in the step name, instead of after - step:

Can you try adjusting the definitions as follows, and let me know if it works?

definitions:
steps: # Note: the "&" is a yaml anchor that allows reusing this particular step
- step: &PHP-Code-Quality
name: PHP-Code-Quality
image: foo/php-ci:1
script:
- |
php bunch-of-stuff.php
- step: &JS-Code-Lint
name: JS-Code-Lint
image: node:18
script:
- |
npx bunch-of-stuff
- step: &Update-Snapshot
name: Update-Snapshot
runs-on:
- server.test
- self.hosted
- linux.shell
script:
- bunch-of-stuff.sh

Kind regards,
Theodora

Eric Poe September 18, 2024

That was it. Thank you!

Like Theodora Boudale likes this
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 20, 2024

You are very welcome! Please feel free to reach out if you ever need anything else!

1 vote
Peter Falck Grony
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
September 18, 2024

Hello Eric,

I didn't know about the Pipeline Validator, so I can't speak to who is wrong between that and the runner. However, I can share that the pipelines in my company looks like this to re-use steps:

aliases:
- &build_and_test
name: Build and test
image: osrf/ros:noetic-desktop-full-bionic
script:
- apt update -y
- apt install git ssh -y
- git clone git@bitbucket.org:<redacted>/bitbucket-ros.git
- bitbucket-ros/prepare-workspace.bash
- bitbucket-ros/build.bash
- bitbucket-ros/test.bash

- &deploy
name: Deploy
image: osrf/ros:noetic-desktop-full-bionic
deployment: production
artifacts:
- "*.deb"
script:
- apt update -y
- apt install git ssh -y
- git clone git@bitbucket.org:<redacted>/bitbucket-ros.git
- bitbucket-ros/binarize.bash
- bitbucket-ros/upload.bash
- bitbucket-ros/docs.bash
- bitbucket-ros/trigger-upstream-projects.py

pipelines:
default:
- step:
<<: *build_and_test
- step:
<<: *deploy
trigger: manual

pull-requests:
'**':
- step:
<<: *build_and_test

tags:
'**':
- step:
<<: *build_and_test
- step:
<<: *deploy


Biggest difference seems to be that we have `aliases` instead of definitions at the top and we use `<<: *anchor` to get the anchor. 

Note: I have found, when trying things out I found on the internet, that Bitbucket is not following standards for Markdown and Yaml - It's a mess. And Yaml in itself is also a mess... You may enjoy reading: The yaml document from hell 

Eric Poe September 18, 2024

Nice. I'm not familiar with `<<:` so I'll look into it.

Like Peter Falck Grony likes this

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events