is it possible to dynamically create parallel steps based on artifact of previous step?

[Eytan Yammer]

We have a monorepo and need to be able to build multiple containers in parallel. However, how many we need to build will change each time to pipeline is run. so I have the following logic:

generate list of containers to build --> generate parallel steps --> run the parallel steps

pseudo-code

pipelines:  default:      
  - step:          
    script:          
      - generate list of changed services          
    artifact:          
      - the list      
  - step:        
    script: create parallel steps    
  - parallel:        \
    - step: build container #1        
    - step: build container #2

is this possible in bitbucket pipelines?

[Justin Thomas · Accepted solution]

@Eytan Yammer Currently Bitbucket Pipelines doesn't support dynamic parallel steps. Are the max containers a constant? If that is the case, I would suggest having a step for all the containers and skipping them if it's not required to build the container based on the artifact. 

[Eytan Yammer]

Thanks. For the time being the number of container won't be changing every day but it will be growing steadily over the next several months. 

What I've decided to do was the following. 

pipelines:  default:    

- parallel:      

   - step:         

      name: Build container1

   condition:

    changesets:                

      includePaths:

        - path/to/container1/code/**

   - step:         

      name: Build container2

   condition:

    changesets:                

      includePaths:

        - path/to/container2/code/** 

this will have to do for the time being. Thanks for the help. 

[Rob Bradshaw]

@Eytan Yammer apologies, very old post. Does this work to dynamically skip/run steps based on files that were written to path/to/container/code/** during build step(s) or files were already in those paths when commited to your branch?

I ask because the way I understand, bb changesets only work against the original repo cloned code, when pipeline first ran, and not against any files added during the build step.

We have reason to skip steps via changesets in case when files were not changed in specific apps within our disparate mono repo app /dist/ folders

[Bill Lazar]

This was answered two years ago. Is there any change in Pipelines since to support this? We have a growing number of tests and manually managing splits to keep PR build times reasonable is unwieldy, to say the least.