Usage of Deployment variables on multiple steps

[Nicola Salvo]

Hi,

I have broken down my pipeline in multiple steps to speed it up, and I'm using deployment variables to manage secrets:

pipelines:
  default:
    - step: 
        name: Step1
        deployment: test
        script:
          - echo $DEPLOYMENT_CREDENTIALS > /tmp/keyfile.json
    - parallel:
        - step:
            name: Step2
            deployment: test
            script:
               - echo $DEPLOYMENT_CREDENTIALS > /tmp/keyfile.json
        - step:
            name: Step3
            deployment: test
            script:
               - echo $DEPLOYMENT_CREDENTIALS > /tmp/keyfile.json

It looks like my pipeline it invalid, as it doesn't allow to specify "deployment" multiple times, however if I specify it only once the other steps do not find the variables. 

Is it possible to have multiple steps which make use of deployments ? Otherwise any suggestion on how to implement such scenario?

Cheers

Nicola

[Christian Koschmieder de Juan]

this feature is A MUST

[Nitin Goyal]

we also facing same issue. we have steps like build docker image and push to ECR different AWS account. and another step we are publishing to the Kubernetes (again on different Kubernetes based on deployment).

[jmgarciadelmoral]

``` - step: &deploy_set_env
name: "Deploy: Set Env"
clone:
enabled: false
artifacts:
- .env
script:
- printenv | xargs -L 1 echo export > .env```

and then in the following steps:

```

- step: &deploy
name: "deploy"
script:
- |
source .env

...```

[Anuj]

I used this approach but getting error in sourcing the .env file

bash: export: `%s': not a valid identifier

[Jay Kravetz]

This worked well. However, the ability to tell multiple steps to use the same environment is necessary

[Roger Viaplana]

Same issue.

When will we have a solution?

[Deleted user]

Our company has decided to slowly migrate to an alternative version control supplier solely because this issue has been around for multiple years and all the suggested work arounds have the potential to expose the pipeline variables. 

[Christian Koschmieder de Juan]

You cant use the same deployment in different steps. It is supossed to be used one time. So, what you can do is to create one deployment for each step and use it in the pipeline.

Like...

Lab-deploy-1 to execute a script

Lab-deploy-2 to execute a seconds script

Until they allow us to repeat deployments in steps thats all you can do.

[Matt]

Any updates on this please? 

[mihai.dobre]

I'm facing almost the same issue. I was looking for some sort of environment variable that bitbucket is pushing by default (like BITBUCKET_DEPLOYMENT_ENVIRONMENT) in order to get the names of the steps in a pipeline. And I was thinking to use the one I mentioned but .... I need to declare fake environments in settings. Please let us use `deployment` or expose the name of the steps in an environment variable so that we don't need to repeat ourselves.

[Zogoo]

Is there any other way to achieve this?

[Deleted user]

Here we are facing the same issue. This will be really nice to have.

[undPohlen]

Then please vote for the feature: https://jira.atlassian.com/browse/BCLOUD-18261

[Deleted user]

Done. Thank you for pointing me that.

[Philip Hodder]

Hello Nicola,

Currently, you can only define a particular deployment environment once per pipeline. In the example you have provided, you have used it in three steps. Which is why you're get errors in your pipeline.

You could try one of the following strategies. They have various trade-offs though.

1. Consolidate the steps into a single one. Though, you'll lose the performance improvements you're trying to add.
2. Create 3 separate environments, then reference each one once in each step. However, this will need duplicate environment variable configuration across the environments. And will also cause three Test environments to show in the dashboard, for what I assume is a single deployment?
3. Set the Deployment credential as a repository-level variable. This will make it accessible to any pipeline that runs in the repository. But could have security concerns depending on how much trust other users with write access have.

You might also find these feature request interesting to follow: 

• Multi step deployments
• Passing environment variables to later steps

If neither of those cover what you'd like, you can also create a new feature request.

It would be helpful for us if you could add details of your use-case to one of those feature requests.

Thanks,

Phil

[cvivieca]

Hi Phil, 

I have the same question. Is it not yet there other way to achieve this? 

Link: https://community.atlassian.com/t5/Bitbucket-questions/Use-duplicate-environment-in-the-same-pipeline-Bitbucket/qaq-p/1286798

[Carlos Júlio]

You can try to use artifacts: https://jira.atlassian.com/browse/BCLOUD-15849?error=login_required&error_description=Login+required&state=323be04c-4b8b-4aa1-bbd9-20f9f81a25d2

[Deleted user]

I have the same issues here. I have three steps to separate functionality but not be able to define the same deployment. This means that the variables that are need to all steps must either be defined elsewhere (very ugly and approach) or create a huge step (also ugly).

And I would like another problem that appears with this deployment limitation. I was using a python image in one step to achieve certain goals. And on the next step I was using another image. Having to merge the multiple steps to one means that I will have to do with one image for all of my goals in this step.

This should be fixed asap.

[kevinpiac]

Facing the exact same issue there. 
I've added more verbose env variables such as DEV_VARIABLE_NAME, PROD_VARIABLE_NAME in order to keep my multiple steps process.

Please consider fixing this ASAP :) 

[simeon-uxiliary]

It would be great to be able to reuse the deployment variables.

[Oleksandr Diudiun]

More than two years and still nothing?

[Doroftei Andrei]

Still waiting for a fix.

[Son Le]

I'm having the exact same issue - waiting for a fix.

[Valentin Funk]

Exactly the same issue, no acceptable solution to this yet.

[Manish Manandhar]

On the same boat, please allow deployment variables to be shared across steps.
Artifacts wont work for secret variables.

[naveen.kummari]

Wow, more than 3 years and no update

[Tomáš Prítrský]

Stages were introduced, where you can group multiple steps
if you define the "deployment:" variable at stage, the env variables will be shared across all steps within this stage.

[Clinical.ly's Atlassian Admin]

I find it odd that your company has spent so much time and resources creating features like Dynamic Pipelines, yet have not solved the problem of reusing environment variables across multiple steps. Yes, people have come up with viable workarounds, so is that good enough for Atlassian?

[Luke Phillips]

I can not believe this request has been around for 5+ years and still not done. Bitbucket Pipelines are way behind the competitors.

[j]

In my case we have a mono-repo with multiple services. We have separate deployment steps for each service, which are triggered conditionally depending on whether there was a change in the directory of the specific folder. 

With this limitation and other limitations on bitbucket pipelines, we are pretty much stuck:

1. We cannot put deployment on multiple steps. 

2. We cannot put the steps in a stage with a deployment at that level, because stages do not allow conditional changesets. So a small change to one service will result in a deployment to all services. 

3. We cannot set the environment variables in a first step depending on the deployment. The only way to do it is to export them into an artifact, but deployment-dependent secrets will become clear text. The functionality to delete/expire artifacts at the end of a pipeline was never implemented. 

All solutions suggested so far don't work.

So many feature requests trying to go around these limitations:

https://jira.atlassian.com/browse/BCLOUD-21188

https://jira.atlassian.com/browse/BCLOUD-15849

https://jira.atlassian.com/browse/BCLOUD-20294

[Rahul Arora]

Same setup ! Pipelines have been a nightmare .. my script is full of workaround for both 1 and 2. Workaround are likely better than trying to figure out Dynamic Pipelines ! Useless UI upgrades over necessities has been a trend for a while so probably the long term solution is github.

[Drew Gallagher]

Looking for this solution right now for our team. It would be very helpful!

[Christian Koschmieder de Juan]

Create different deployment in the branch. Like lab-dep-1 lab-dep2 and use it for the steps in the same deployment. Is the only thing you can do.

[Adrien Carré]

I would like this to be implemented as well.

Ideally we could define a deployment variable for multiple steps for an entire pipeline

[Sergiy Prognimak]

Not sure what it cause so much problems for so long. Solution is very simple

Just make it like this. First grab  env variables from deployment definition into artifact, then use them in next steps.

steps:

   step_init: &build_init_step

     script:

      - touch environment.sh

      - echo "export REPOSITORY=$REPOSITORY" >> environment.sh

      - echo "export BUILD_TAG=$BUILD_TAG" >> environment.sh

      - echo "export TARGET_HOST=$TARGET_HOST" >> environment.sh

   artifacts:

    - environment.sh

   step_deploy: &build_deploy_step

     script:

      - source environment.sh

pipelines:

  branches:

    'master':

   - step:

     name: Build "Test" environment images

     deployment: Test

     <<: *build_init_step

   - parallel:

     <<: *build_images_parallel_test

   - step:

     name: Deploying to "Test"

     <<: *build_deploy_step

[j]

This solution puts deployment-specific secrets, like passwords which will be different between environments, in clear text in the artifacts of your pipeline.

[Bryant Foresman]

Anyone on this thread, look into stages, which were recently implemented; they provide multi step deployments, but have some limitations, like no parallel steps in them, and no manual steps in them: https://support.atlassian.com/bitbucket-cloud/docs/stage-options/

[Drew Gallagher]

Are there any plans for stages to run in parallel eventually? 

[Tomas Maggio]

How is it possible that everything they do have so many limitations?

[Aurimas Navardauskas]

what if I have two stages:

- plan

- apply (manual trigger)

for both stages I need env vars specific to respective environment, does not seem doable?

[Maksym Cierżniak]

Exacly, we'd like to have deployment configured for each environment where we can use deployment variables and first runt `terraform plan` to see what will be the outcome and then have manual step for actually applying the plan @Philip Hodder . I assume that it is currently not possible to achieve, right ? 

[j]

Stages don't even allow steps with conditions inside them. The reason I have separate steps is that some of them are not triggered if no changes were done to that specific module. But I want to be able to run all steps to different deployment environment. Can't put them in a stage because stages don't allow conditions in steps. 

[Ba Satti]

wow wow