Access private runner host environment variables

clockworknet November 11, 2021

Hi,

We are utilising self hosted runners, with a runner located in each of our environments (dev, staging, prod, etc). Each of the runner hosts is built with a handful of global environment variables configured, that provide some general configuration for the environment in question (e.g. AWS_DEFAULT_REGION, ENVIRONMENT_NAME, etc).

Is there a way to pass these variables into the container used to run a step on those runners, the use case being that steps would inherit configuration, simply by virtue of the runner they are assigned to? (Similar to setting and using environment variables on individual Jenkins nodes) I would rather avoid having to duplicate this configuration inside BitBucket or having to maintain functionality inside the container to pull this info in from another source, if possible.

2 answers

0 votes
Andrew Gibbs February 4, 2022

I would likewise love to be able to get an environment variable from the outer orchestrator container to the inner step container so I could make use of IAM Role derived credentials within my pipelines instead of having to use static credentials from an IAM User...

https://community.atlassian.com/t5/Bitbucket-questions/use-IAM-role-credentials-from-private-runner-executing-inside/qaq-p/1934915

0 votes
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 16, 2021

Hi @clockworknet and welcome to the community.

In order to provide isolation and reproducibility, a runner doesn’t provide access directly to the host file system, so I'm afraid that what you're asking is not possible.

You can either set these variables in Bitbucket or pass the values in a file and then source the file during the build.

Kind regards,
Theodora

clockworknet November 16, 2021

Hi Theodora,

 I am suspecting that what I am asking about is not possible, but to be clear I am not asking for filesystem access. I am asking whether it is possible to pass environment variables that are set in the runner host globally, into the container started for each build step - the equivalent of:

`docker run -e SOME_GLOBAL_ENV_VAR ...`

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 19, 2021

Hi @clockworknet,

Thanks for the clarification, I didn't realize that you were talking about passing the environment variables in the 'docker run' command for the runner.

Please allow me to check with my team internally and I'll get back to you on that.

Kind regards,
Theodora

clockworknet November 21, 2021

Hi Theodora,

Thanks for looking into this further.

It is not the runner container that I am concerned about getting the environment variables into. I start that container and so have full control over the options that are used to start it.

It is each container that is started by the runner process, to handle a deployment step. I guess that if it were possible, those variables would need to be first passed into the runner container, but I am interested in whether it is possible to then get that process to pass them on to each container it starts.

Norbert C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 23, 2021

Hi @clockworknet 

Let me step in, I'm Norbert and I'm also a Bitbucket Cloud Support Engineer, it's nice to meet you.

I would like to inform you that the functionality what you would like achieve is unfortunately not possible. The only way how you can pass the environmental variables to the runner container is to set those variables in your Bitbucket Pipelines settings and once it's set up, those variables can be used in your build.

Please let me know if my explanation was clear and whether if you have any further question?

I'm looking forward to hear from you.

Have a great day ahead!

Best Regards,
Norbert
Atlassian Bitbucket Cloud Support Engineer

Ajay _view26_
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
November 25, 2021

HI @Norbert C ,


Could you provide a sample example? 

I presume you are referring to using them as variables inside the pipeline.

Cheers

Ajay

Norbert C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 26, 2021

Hi Ajay,

Yes, of course please find the example :) .

I've created an "asdf" environmental variable with the value of "testtesttest" in Pipelines:

test1.png

When I started this Pipelines, as you can see I'm using Runners and when I echo the "asdf" environmental variable, it shows the "testtesttest" value:

test2.png

Please let me know whether if this is the functionality what you're looking for?

Best Regards,

Norbert 
Atlassian Bitbucket Cloud Support

clockworknet November 26, 2021

Thanks @Norbert Cfor clarifying the situation - it would be a nice feature to have at some point if possible.

Norbert C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
December 1, 2021

Hi @clockworknet 

 

Can you clarify please what feature would you like to have implemented? Would you like to be able to pass the environmental variable via the docker run command? 


Once I have the clarification, I'm going to open a feature request for you.

 

I'm looking forward to hear from you.

 

Best Regards,
Norbert

Atlassian Bitbucket Cloud Support 

clockworknet December 1, 2021

Hi @Norbert C 

Our particular use case:

 

  • we use existing automation and cfg mgmt to build our private runners hosts (in our case AWS EC2 instances), and deploy the BB Runner container instance (there is one small manual step in registering the runner in BB, where we need to copy the various tokens from BB)
  • as part of this automated provisioning, we set a series of environment variables on the EC2 host. The values are typically derived from the AWS environment (e.g. tags added to the EC2 instance/param store values, etc) or otherwise stored in our existing cfg mgmt systems

We would like a way for those host based environment variables to be accessible inside the container used to run each deployment step. With this, we would be able to pass configuration to the deployment step implicitly, simply based on which runner the step was executing.

Without this feature, our options are to create duplicate config inside of BitBucket, however that is inefficient and can also result in having to store/manage values that we would prefer remain inside our environments. Alternatively, we can re-run the same process on each deployment step that we ran when the host was booted to replicate the behaviour and set the env vars inside the step container, but in the context of a fast feedback system, that is somewhat slow.

I guess the 'runner' container would need to be involved given I presume it is responsible for starting the 'deployment step containers' so perhaps being able to pass it a series of env vars with a specific prefix (much like TFVARS in Terraform ) that it will in turn set on the step containers could be an option?

Norbert C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
December 6, 2021

Hi @clockworknet 

I appreciate your detailed explanation. Unfortunately this is not available right now, but I opened the following feature request:

BCLOUD-21523 - Allow Bitbucket runner to access host's environmental variable


Our development team will give a first-hand update on that ticket if there's any progress made so I would suggest keeping a watch and vote for it.

 

Do note however that there's no ETA on enhancement request, and all enhancements are implemented with this policy in mind: Implementation of New Features Policy

Best Regards,
Norbert
Atlassian Bitbucket Cloud Support

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events