You're enrolled in our new beta rewards program. Join our group to get the inside scoop and share your feedback.
Join groupJoin the community to find out what other Atlassian users are discussing, debating and creating.
I have lot's of duplicate code in my pipeline yaml. I tried the 'YAML anchors and references' approach suggested in Q&A https://community.atlassian.com/t5/Bitbucket-questions/How-to-reuse-steps-for-different-branches/qaq-p/875615 but it doesn't work and I get an error when I want to launch my pipeline.
The error is sometling like:
There is an error in your bitbucket-pipelines.yml at [pipelines > custom > build-internal > 0 > step > script > 1]. To be precise: Missing or empty command string. Each item in this list should either be a single command string or a map defining a pipe invocation.
Validate your bitbucket-pipelines.ymlView bitbucket-pipelines.yml
Below is a code sample to test:
image: maven
clone:
depth: 500
commonStep: &commonStep
- step:
script:
- echo "How come this causes a -Configuration error- when I try to execute a pipeline while it did pass the in-browser YAML validation?"
pipelines:
custom:
build-internal:
- step:
name: Build maven internal
caches:
- maven
script:
- echo "Lot's of code redundancy"
- <<: *commonStep
deploy-internal-tst:
- step:
name: Deploy to test
deployment: test
script:
- echo "Lot's of code redundancy"
- <<: *commonStep
What is going wrong?
Hi @Samuel
I think the problem you are facing is caused by indentation. The common step behaves like a full step, not only an extra command in the script section.
I understand you when you say that there is no validation error in the syntax but there are compilation errors. This is caused by a problem to validate files using anchors. This problem is already reported here:
I've run this test and it worked as expected. Please notice that I used the default section to simplify running it locally.
image: maven
clone:
depth: 500
commonStep: &commonStep
step:
script:
- echo "How come this causes a -Configuration error- when I try to execute a pipeline while it did pass the in-browser YAML validation?"
pipelines:
default:
- step:
name: Build maven internal
caches:
- maven
script:
- echo "Lot's of code redundancy"
- <<: *commonStep
You will see that
- <<: *commonStep
is aligned with the above step and it will create an extra step when compiling.
I also built another 2 tests examples to show a possible way to use anchors.
The first one is similar to the one shared in that question you mentioned:
commonStep: &commonStep
step:
script:
- echo "common step"
commonScript: &commonScript
script:
- echo "common script"
pipelines:
default:
- <<: *commonStep
- step:
<<: *commonScript
The second one is similar to what is shared in our docs:
definitions:
step: &myCommonStep
script: &myCommonScript
- echo "definition command"
pipelines:
default:
- step: *myCommonStep
- step:
script: *myCommonScript
- step:
<<: *myCommonStep
name: my new name
Reference: YAML anchors - Atlassian Documentation
I tested both of them.
I hope that helps.
Thanks Daniel, your reply got me going.
Rewriting my example the now working version looks like this:
# Docker image
image: maven
clone:
depth: 500
commonStep1: &commonStep1
- step:
name: Step 1a
script:
- echo "Lots of duplicate code"
- echo "Let's see the file structure"
- mkdir -p test
- echo "." > test/test-file2.txt
- ls -la test/
artifacts:
- test/*.txt
commonStep2: &commonStep2
- step:
name: Step 1b
script:
- echo "Even more duplicate code"
- echo "Let's see the file structure"
- mkdir -p test
- ls -la test/
pipelines:
custom:
build-internal:
- step:
name: Step 1
script:
- echo "Let's get started"
- mkdir -p test
- echo "." > test/test-file1.txt
- ls -la test/
- <<: *commonStep1
- <<: *commonStep2
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Samuel
I'm glad you were already finding your own way!
I wanted to share those examples as a reference for other users as well.
Thank you for taking the time to create this question and following up on it, I appreciate that.
Have a good one!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Daniel Santos ,
I am trying to reuse a common script and add more script commands on the same step, but I am not able to do it. Is it possible? Here's what I am trying to do, from your example:
definitions:
scripts:
script: &commonScript
- echo "common script"
pipelines:
default:
- step:
script:
<<: *commonScript
- echo "another script"
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I was having the same problem, this solves for me. take a try.
definitions:
script: &commonScript
echo "common script"
pipelines:
default:
- step:
script:
- *commonScript
- echo "another script"
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Diônitas Mendes dos Santos I tried your solution after a plenty of tries but I get a configuration error at the beginning of the pipeline. Even if BB validator says that everything is fine with my pipeline.
PS: I'm using parallel steps
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Khalifa EL BANAN can you provide an example of your pipeline as I got it working. Note: I did see an error when I tried to make the reusable script a list and instead needed to make them a string and use &&
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Yes sure, in general my pipeline looks like the following:
definitions:
script: &myCommonScript
- first script
- second script
pipelines:
pull-requests:
feature/*:
- step:
name: 'first step'
script:
- some script
- parallel:
- step:
name: 'parallel step 1'
script:
- *myCommonScript
- a further script
- step:
name: 'parallel step 2'
script:
- *myCommonScript
- another script
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
The issue is likely that
&myCommonScript
is a list. When you try to use that in a script such as
- step:
name: 'parallel step 1'
script:
- *myCommonScript
- a further script
what you actually have is a list of scripts within a script. My recommendation would instead be changing your pipeline to have the common script be a single operation such as
definitions:
script: &myCommonScript
first script &&
second script
pipelines:
pull-requests:
feature/*:
- step:
name: 'first step'
script:
- some script
- parallel:
- step:
name: 'parallel step 1'
script:
- *myCommonScript
- a further script
- step:
name: 'parallel step 2'
script:
- *myCommonScript
- another script
There may be a cleaner way of doing this but I have not found it
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
It doesn't work.
There is an error in your bitbucket-pipelines.yml at [pipelines > branches > TEST-DEV > 0 > step > script > 0]. To be precise: This section should be a string or a map (it is currently defined as a list).
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Jim, note in my case a not using a list, is just a string. if you need more than one script in commonScript, you can do some thing like that
definitions:
script: &commonScript
echo "common script 1";
echo "common script 2";
pipelines:
default:
- step:
script:
- *commonScript
- echo "another script"
the yaml parse will treat as if it were a big string
echo "common script 1";echo "common script 2";
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Thanks @Diônitas Mendes dos Santos
i will post my bitbucket-pipelines.yml. I check with bitbucket validatator. it said "valid", but i tried it with bitbucket pipeline. It reported error.
--------------------
There is an error in your bitbucket-pipelines.yml at [pipelines > branches > TEST-DEV > 0 > step > script > 0]. To be precise: This section should be a string or a map (it is currently defined as a list).
-------------------
```
image : nikolaik/python-nodejs:python3.8-nodejs15
definitions:
step: &myCommonStep
script: &myCommonScript
- curl https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip
- unzip awscliv2.zip
- ./aws/install
- ls -lrta
- npm install -g serverless
- python --version
- serverless --version
- aws --version
- export roleARN=$ROLE_ARN_DEV
- aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
- aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
- aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID --profile loop-role
- aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY --profile loop-role
- echo "role_arn=$roleARN" >> ~/.aws/credentials
- echo "source_profile=default" >> ~/.aws/credentials
- cat ~/.aws/credentials
- aws configure set source_profile default --profile loop-role
- aws configure set role_arn $roleARN --profile loop-role
- aws configure set region ap-southeast-2 --profile loop-role
- aws configure set output json --profile loop-role
- aws configure list
- cat ~/.aws/credentials
- cat ~/.aws/config
pipelines:
branches:
TEST-DEV:
- step:
name: serverless build
caches:
- node
script:
- *myCommonScript
- npm install -g serverless-pseudo-parameters
- npm install -g serverless-python-requirements
- serverless deploy --stage dev --verbose --aws-profile loop-role
```
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
# I tried to use code block, but after copy-paste, it disappear.
# This pipeline is to use serverless and assume role to deploy an application
Thanks
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
That is not possible through bitbucket-pipelines.yml default coding. What you could do instead is to create a script with the command you want to run and call that script in multiple steps.
There is a feature request that might address your request:
I hope that helps.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Can we use after-script in definitions as below and leverage *step_report_publish throughout yml
definitions:
steps:
- step: &report_generation
name: Connect Final results
size: 2x
script:
- tests_merge.sh --merge results
after-script: &step_report_publish
- tests_publish.sh --dir target/site --merge results
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi everyone, We are looking to learn more about development teams’ workflows and pain points, especially around DevOps, integrations, administration, scale, security, and the related challeng...
Connect with like-minded Atlassian users at free events near you!
Find an eventConnect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.
Host an eventYou're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.