Empty variables

[Terence Pires]

Hi,

I'm trying to use variables in a pipeline but, even though they are defined and have a value defined, pipeline treats them as empty.

I'm using a mix of repository and deployment variables, of which here is the list:

Repository vars:

• EMAIL_PASSWORD_UNSECURE
• EMAIL_PASSWORD (secured var)
• EMAIL_HOST

Deployment vars:

• APP_URL
• EMAIL_SUBJECT

Deployment vars do not display, where repository vars do.

Also, I tried to use the secured var EMAIL_PASSWORD var in a atlassian/email-notify:0.4.4 pipe but I always get an error, so I assume they are not interpreted well either.

Here's part of my pipeline

branches:
  '*':
    - step:
        name: Test
        script:
          - echo $EMAIL_HOST # displays well
          - echo $EMAIL_PASSWORD # displays well
          - echo $EMAIL_PASSWORD_UNSECURE # can't be used in a pipe
          - echo $EMAIL_SUBJECT # does not display
          - echo $APP_URL # does not display

 Thanks

[Terence Pires · Accepted solution]

Problem was solved with the update of the email-notify package to version 0.4.5

[Terence Pires]

Hi @Theodora Boudale thanks for your answer.

My previous message only mentioned part of the pipeline, here is the full version, in which,  as you can see, I am using a deployment name. 

# This is a sample build configuration for JavaScript.
# Check our guides at https://confluence.atlassian.com/x/14UWN for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: node:latest

definitions:
  steps:
    - step: &env
        name: Env
    - step: &build-test
        name: Build
        caches:
          - node
        script:
           [... build steps]
        artifacts:
          - artifact-client.tar.gz
    - step: &deploy-test
        name: Deploy
        script:
           [... deploy steps]

  branches:
    '*':
      - step:
          <<: *env
          script:
            - cat .env.test > .env
            - echo $EMAIL_PORT
            - echo $EMAIL_HOST
            - echo $EMAIL_PASSWORD
            - echo $EMAIL_PASSWORD_UNSECURE
            - echo $EMAIL_SUBJECT
            - echo $EMAIL_RECIPIENTS
            - echo "A new version of has been deployed on ${APP_URL}"
            - pipe: atlassian/email-notify:0.4.4
              variables:
                USERNAME: $EMAIL_USERNAME
                PASSWORD: $EMAIL_PASSWORD
                FROM: $EMAIL_FROM
                TO: 'user@mail.com'
                HOST: $EMAIL_HOST
                PORT: $EMAIL_PORT
                SUBJECT: $EMAIL_SUBJECT
                BODY: 'A new version of has been deployed on ${APP_URL}'
          artifacts:
            - .env
      - step:
          *build-test
      - step:
          <<: *deploy-test
          deployment: test

Error output

Traceback (most recent call last):  File "/pipe.py", line 181, in <module>    pipe = EmailNotify(pipe_metadata=metadata, schema=schema, check_for_newer_version=True)  File "/usr/local/lib/python3.7/site-packages/bitbucket_pipes_toolkit/core.py", line 239, in __init__    self.variables = self.validate()  File "/usr/local/lib/python3.7/site-packages/bitbucket_pipes_toolkit/core.py", line 296, in validate    env[key] = yaml.safe_load(value)  File "/usr/local/lib/python3.7/site-packages/yaml/__init__.py", line 162, in safe_load    return load(stream, SafeLoader)  File "/usr/local/lib/python3.7/site-packages/yaml/__init__.py", line 114, in load    return loader.get_single_data()  File "/usr/local/lib/python3.7/site-packages/yaml/constructor.py", line 43, in get_single_data    return self.construct_document(node)  File "/usr/local/lib/python3.7/site-packages/yaml/constructor.py", line 47, in construct_document    data = self.construct_object(node)  File "/usr/local/lib/python3.7/site-packages/yaml/constructor.py", line 92, in construct_object    data = constructor(self, node)  File "/usr/local/lib/python3.7/site-packages/yaml/constructor.py", line 420, in construct_undefined    node.start_mark)yaml.constructor.ConstructorError: could not determine a constructor for the tag '$EMAIL_PASSWORD'  in "<unicode string>", line 1, column 1:    $EMAIL_PASSWORD    ^  

[Theodora Boudale]

Hi @Terence Pires,

Thank you for your reply.

I can see in the yaml file that you posted that the deployment keyword is used for the following step:

- step:
    <<: *deploy-test
    deployment: test

The step that echoes the variables is different than the above step and it does not have the deployment keyword. Any deployment variables you have defined in the 'test' environment will be available only in the script of the step with 'deployment: test'.

I will reach out to the development team regarding the error you are seeing with the pipe and I'll get back to you when I hear from them.

Kind regards,
Theodora

[Halyna Berezovska]

@Terence Pires hello!

I am seeing that your EMAIL_PASSWORD var cannot be loaded as yaml.

What I can say to debug it more:

- please, use the latest email-notify version 0.4.5, we did some changes and we wonder if it fixes it or not

- try this typing:  PASSWORD: '${EMAIL_PASSWORD}'

- if still fails, locally try in Python console or in any python environment with YAML your password and see if yaml error will happen.

Regards, Galyna

[Terence Pires]

Hello Galyna,

Yes, updating to 4.4.5 solved my problem. I can now send an email using both repository and deployment vars.

thanks for your help

[Halyna Berezovska]

@Terence Pires then I know what bug it was. Thank you for raising this!

Have a great day!

[Theodora Boudale]

Hi @Terence Pires and welcome to the community.

Deployment variables are available only for deployment steps for the environment they were defined for.

E.g. if you have defined the deployment variables for the environment named staging, they will be available in a step that includes the following in its definition:

deployment: staging

The step in the part of your yml is missing that, which is why the deployment variables are not available.

Regarding the issue with the variable EMAIL_PASSWORD in atlassian/email-notify:0.4.4 pipe, I assume that you used this as a value of the PASSWORD variable of the pipe? Could you please attach here the full output of the pipe when you use this variable (please make sure to mask private/sensitive data in the output, if any, prior to sharing) so we can better help you?

Kind regards,
Theodora