Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

is it possible to dynamically create parallel steps based on artifact of previous step?

Eytan Yammer April 8, 2021

We have a monorepo and need to be able to build multiple containers in parallel. However, how many we need to build will change each time to pipeline is run. so I have the following logic:


generate list of containers to build --> generate parallel steps --> run the parallel steps


pipelines:  default:      
generate list of changed services          
the list      
script: create parallel steps    
parallel:        \
stepbuild container #1        
stepbuild container #2


is this possible in bitbucket pipelines?

2 answers

1 accepted

0 votes
Answer accepted
Justin Thomas
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 15, 2021

@Eytan Yammer Currently Bitbucket Pipelines doesn't support dynamic parallel steps. Are the max containers a constant? If that is the case, I would suggest having a step for all the containers and skipping them if it's not required to build the container based on the artifact. 

Eytan Yammer April 17, 2021

Thanks. For the time being the number of container won't be changing every day but it will be growing steadily over the next several months. 


What I've decided to do was the following. 


pipelines:  default:    


   - step:         

      nameBuild container1




        - path/to/container1/code/**

   - step:         

      nameBuild container2




        - path/to/container2/code/** 


this will have to do for the time being. Thanks for the help. 

Like # people like this
Rob Bradshaw August 22, 2023

@Eytan Yammer apologies, very old post. Does this work to dynamically skip/run steps based on files that were written to path/to/container/code/** during build step(s) or files were already in those paths when commited to your branch?


I ask because the way I understand, bb changesets only work against the original repo cloned code, when pipeline first ran, and not against any files added during the build step.


We have reason to skip steps via changesets in case when files were not changed in specific apps within our disparate mono repo app /dist/ folders

0 votes
Bill Lazar March 31, 2023

This was answered two years ago. Is there any change in Pipelines since to support this? We have a growing number of tests and manually managing splits to keep PR build times reasonable is unwieldy, to say the least.

Suggest an answer

Log in or Sign up to answer
AUG Leaders

Atlassian Community Events