pipeline: how to create a dependency between steps

Admin Admin January 23, 2022

Hello,

 is there a way to make the "deploy" step wait for the "build" step?

issue:

 the download of the docker image for scp pipe takes 40 seconds and the actual copy of the artifacts - 2.

 

It would be good to have all the downloading done while "build" is in progress.

 

bitbucket_pipelines.yml


image: atlassian/default-image:3

pipelines:
default:
- step:
name: Build and Test
caches:
- node
script:
- rm -rf node_modules/
- rm package-lock.json
- npm install
- npm run build
artifacts:
- node_modules/**
- .nuxt/**
- package.json
- step:
name: Deploy npm_modules artifacts using SCP
deployment: test
script:
- pipe: atlassian/scp-deploy:1.2.1
variables:
USER : $DEP_USER
SERVER : $DEP_SERVER
REMOTE_PATH : '/tmp/test/'
LOCAL_PATH : 'node_modules/'
DEBUG: 'true'

2 answers

1 accepted

0 votes
Answer accepted
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 25, 2022

Hi @Admin Admin,

It is not possible to configure what you are asking, non-parallel steps are always executed sequentially and the previous step needs to complete for the next one to start. Parallel steps would not be a solution for your use case, as it is not possible for parallel steps to use artifacts by steps in the same parallel set.

What you could do to speed up the process would be to use a docker cache, this way the pipe's Docker image will be cached and then used in subsequent builds. You can modify the second step as follows:

- step:
name: Deploy npm_modules artifacts using SCP
deployment: test
caches:
- docker
script:
- pipe: atlassian/scp-deploy:1.2.1
variables:
USER : $DEP_USER
SERVER : $DEP_SERVER
REMOTE_PATH : '/tmp/test/'
LOCAL_PATH : 'node_modules/'
DEBUG: 'true'

The cache will be saved after the first successful build and subsequent builds will use the cache for the pipe's Docker image. Any cache which is older than 1 week will be cleared automatically and repopulated during the next build:

https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/

If cloning the repo during a pipelines build takes a long time, you could try adjusting the depth of the clone; the default is 50, but if you don't need that you could specify a lower value as follows in the yaml file

clone:
depth: 5 # include the last five commits

Kind regards,
Theodora

Admin Admin January 25, 2022

@Theodora Boudale 

 not what i was _hoping_ to hear. But thank you for clarifying the question for me.

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 26, 2022

You are very welcome, please feel free to reach out if you ever need anything else.

Kind regards,
Theodora

0 votes
Pramodh M
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
January 23, 2022

Hi @Admin Admin 

It's already dependent, If your step fails, succeeding steps will not be executed!!

There's a configuration to run steps in parallel

https://bitbucket.org/blog/speed-build-parallel-steps-pipelines

Thanks,
Pramodh

Admin Admin January 23, 2022

i do NOT want the dependent step to fail.

I'd like the dependent step to wait until the artifacts are created...

Pramodh M
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
January 24, 2022

Hi @Admin Admin 

Since the Image where you are building the npm project, and SCP is happening in another pipe. This should go one after the other.

Thanks,
Pramodh

Admin Admin January 24, 2022

@Pramodh M,

 I maybe not be clear, apologies.

I'd like to kick off the deployment step before the artifacts in the build step are ready.

What I'm trying to do is :

1. step build and step deployment: downloads repos and dockers simultaneously

2. build is building;

3. deployment is waiting for the build to finish before starting the deployment.

Pramodh M
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
January 24, 2022

Yes @Admin Admin 

I understood, Since we are using Pipe to transfer the file in the step, the docker image will be pulled at the moment in time.

If you are using a self-hosted runner there is a workaround, but with Atlassian, runners it's not efficient (You could try having npm build && docker pull image-name to keep the image before you run the next step where it uses docker image)

Thanks,
Pramodh

Like Admin Admin likes this

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
AUG Leaders

Atlassian Community Events