Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Deleted user
Level
0 / 0 points
Next:
badges earned

Your Points Tracker
Challenges
Leaderboard
  • Global
  • Feed

Badge for your thoughts?

You're enrolled in our new beta rewards program. Join our group to get the inside scoop and share your feedback.

Join group
Recognition
Give the gift of kudos
You have 0 kudos available to give
Who do you want to recognize?
Why do you want to recognize them?
Kudos
Great job appreciating your peers!
Check back soon to give more kudos.

Past Kudos Given
No kudos given
You haven't given any kudos yet. Share the love above and you'll see it here.

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Bitbucket pipeline reuse of code

I have lot's of duplicate code in my pipeline yaml. I tried the 'YAML anchors and references' approach suggested in Q&A https://community.atlassian.com/t5/Bitbucket-questions/How-to-reuse-steps-for-different-branches/qaq-p/875615 but it doesn't work and I get an error when I want to launch my pipeline.

The error is sometling like:

Configuration error

There is an error in your bitbucket-pipelines.yml at [pipelines > custom > build-internal > 0 > step > script > 1]. To be precise: Missing or empty command string. Each item in this list should either be a single command string or a map defining a pipe invocation.

Validate your bitbucket-pipelines.ymlView bitbucket-pipelines.yml

 

Below is a code sample to test:

image: maven

clone:
depth: 500

commonStep: &commonStep
- step:
script:
- echo "How come this causes a -Configuration error- when I try to execute a pipeline while it did pass the in-browser YAML validation?"

pipelines:
custom:
build-internal:
- step:
name: Build maven internal
caches:
- maven
script:
- echo "Lot's of code redundancy"
- <<: *commonStep

deploy-internal-tst:
- step:
name: Deploy to test
deployment: test
script:
- echo "Lot's of code redundancy"
- <<: *commonStep

What is going wrong?

3 answers

1 accepted

2 votes
Answer accepted

Hi @Samuel

I think the problem you are facing is caused by indentation. The common step behaves like a full step, not only an extra command in the script section. 

I understand you when you say that there is no validation error in the syntax but there are compilation errors. This is caused by a problem to validate files using anchors. This problem is already reported here:

I've run this test and it worked as expected. Please notice that I used the default section to simplify running it locally.

image: maven

clone:
depth: 500

commonStep: &commonStep
  step:
    script:
      - echo "How come this causes a -Configuration error- when I try to execute a pipeline while it did pass the in-browser YAML validation?"

pipelines:
  default:
    - step:
        name: Build maven internal
        caches:
          - maven
        script:
          - echo "Lot's of code redundancy"
    - <<: *commonStep

You will see that

 - <<: *commonStep

is aligned with the above step and it will create an extra step when compiling.

 

I also built another 2 tests examples to show a possible way to use anchors.

The first one is similar to the one shared in that question you mentioned:

commonStep: &commonStep
  step:
    script:
      - echo "common step"

commonScript: &commonScript
  script:
    - echo "common script"

pipelines:
  default:
    - <<: *commonStep
    - step:
        <<: *commonScript

The second one is similar to what is shared in our docs:

definitions:
  step: &myCommonStep
    script: &myCommonScript
      - echo "definition command"

pipelines:
  default:
    - step: *myCommonStep
    - step:
        script: *myCommonScript
    - step:
        <<: *myCommonStep
        name: my new name

Reference: YAML anchors - Atlassian Documentation

I tested both of them.

I hope that helps.

Thanks Daniel, your reply got me going.

Rewriting my example the now working version looks like this:

# Docker image
image: maven

clone:
depth: 500

commonStep1: &commonStep1
- step:
name: Step 1a
script:
- echo "Lots of duplicate code"
- echo "Let's see the file structure"
- mkdir -p test
- echo "." > test/test-file2.txt
- ls -la test/
artifacts:
- test/*.txt
commonStep2: &commonStep2
- step:
name: Step 1b
script:
- echo "Even more duplicate code"
- echo "Let's see the file structure"
- mkdir -p test
- ls -la test/
pipelines:
custom:
build-internal:
- step:
name: Step 1
script:
- echo "Let's get started"
- mkdir -p test
- echo "." > test/test-file1.txt
- ls -la test/

- <<: *commonStep1
- <<: *commonStep2
Like # people like this

Hi @Samuel

I'm glad you were already finding your own way!
I wanted to share those examples as a reference for other users as well.

Thank you for taking the time to create this question and following up on it, I appreciate that.

Have a good one!

Hi @Daniel Santos ,

I am trying to reuse a common script and add more script commands on the same step, but I am not able to do it. Is it possible? Here's what I am trying to do, from your example:

definitions:
scripts:

  script: &commonScript
    - echo "common script"

pipelines:
  default:
    - step:
script:
        <<: *commonScript
- echo "another script"
Like # people like this

@rgbp

I was having the same problem, this solves for me. take a try.

definitions:
script: &commonScript
echo "common script"

pipelines:
default:
- step:
script:
- *commonScript
- echo "another script"
Like # people like this

@Diônitas Mendes dos Santos I tried your solution after a plenty of tries but I get a configuration error at the beginning of the pipeline. Even if BB validator says that everything is fine with my pipeline.

PS: I'm using parallel steps

Like raymond Kelly likes this

 @Khalifa EL BANAN  can you provide an example of your pipeline as I got it working. Note: I did see an error when I tried to make the reusable script a list and instead needed to make them a string and use &&

Like Khalifa EL BANAN likes this

Yes sure, in general my pipeline looks like the following:

definitions:
script: &myCommonScript
- first script
- second script

pipelines:
pull-requests:
feature/*:
- step:
name: 'first step'
script:
- some script

- parallel:
- step:
name: 'parallel step 1'
script:
- *myCommonScript
- a further script
- step:
name: 'parallel step 2'
script:
- *myCommonScript
- another script

The issue is likely that 

&myCommonScript

is a list. When you try to use that in a script such as

        - step:
name: 'parallel step 1'
script:
- *myCommonScript
- a further script

what you actually have is a list of scripts within a script. My recommendation would instead be changing your pipeline to have the common script be a single operation such as

definitions:
script: &myCommonScript
first script &&
second script

pipelines:
pull-requests:
feature/*:
- step:
name: 'first step'
script:
- some script

- parallel:
- step:
name: 'parallel step 1'
script:
- *myCommonScript
- a further script
- step:
name: 'parallel step 2'
script:
- *myCommonScript
- another script

 There may be a cleaner way of doing this but I have not found it

Like # people like this
Like raymond Kelly likes this

It doesn't work. 

 

Configuration error

There is an error in your bitbucket-pipelines.yml at [pipelines > branches > TEST-DEV > 0 > step > script > 0]. To be precise: This section should be a string or a map (it is currently defined as a list).

Hi Jim, note in my case a not using a list, is just a string. if you need more than one script in commonScript, you can do some thing like that

definitions:
script: &commonScript
echo "common script 1";
echo "common script 2";


pipelines:
default:
- step:
script:
- *commonScript
- echo "another script"

the yaml parse will treat as if it were a big string



echo "common script 1";echo "common script 2";
Like Jim Fang likes this

Thanks @Diônitas Mendes dos Santos

i will post my bitbucket-pipelines.yml.  I check with bitbucket validatator. it said "valid", but i tried it with bitbucket pipeline. It reported error.

--------------------

There is an error in your bitbucket-pipelines.yml at [pipelines > branches > TEST-DEV > 0 > step > script > 0]. To be precise: This section should be a string or a map (it is currently defined as a list).

-------------------

```
image : nikolaik/python-nodejs:python3.8-nodejs15

definitions:
step: &myCommonStep
script: &myCommonScript
- curl https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip
- unzip awscliv2.zip
- ./aws/install
- ls -lrta
- npm install -g serverless
- python --version
- serverless --version
- aws --version
- export roleARN=$ROLE_ARN_DEV
- aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
- aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
- aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID --profile loop-role
- aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY --profile loop-role
- echo "role_arn=$roleARN" >> ~/.aws/credentials
- echo "source_profile=default" >> ~/.aws/credentials
- cat ~/.aws/credentials
- aws configure set source_profile default --profile loop-role
- aws configure set role_arn $roleARN --profile loop-role
- aws configure set region ap-southeast-2 --profile loop-role
- aws configure set output json --profile loop-role
- aws configure list
- cat ~/.aws/credentials
- cat ~/.aws/config

pipelines:
branches:
TEST-DEV:
- step:
name: serverless build
caches:
- node
script:
- *myCommonScript
- npm install -g serverless-pseudo-parameters
- npm install -g serverless-python-requirements
- serverless deploy --stage dev --verbose --aws-profile loop-role

```

# I tried to use code block, but after copy-paste, it disappear.
# This pipeline is to use serverless and assume role to deploy an application

Thanks 

what if you need to share code across multiple single steps

Hi @Charlie Jonas

That is not possible through bitbucket-pipelines.yml default coding. What you could do instead is to create a script with the command you want to run and call that script in multiple steps.

There is a feature request that might address your request:

I hope that helps.

This ticket is closed without any solution for reuse script in multiple steps

Can we use after-script in definitions as below and leverage *step_report_publish throughout yml

definitions:
steps:
- step: &report_generation
name: Connect Final results
size: 2x
script:
- tests_merge.sh --merge results

after-script: &step_report_publish
- tests_publish.sh --dir target/site --merge results

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

Calling any interview participants for Bitbucket Data Center

Hi everyone,  We are looking to learn more about development teams’ workflows and pain points, especially around DevOps, integrations, administration, scale, security, and the related challeng...

485 views 5 4
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you