Why are my deployment variables not propagating through my pipeline correctly?

Shane McNamara_vhba1h

I have a pipeline that dockerizes a Node.js application, pushes the image to AWS ECR and then deploys the image to AWS ECS. 

I am using atlassian/aws-ecr-push-image:1.2.0 to push to ECR and atlassian/aws-ecs-deploy:1.0.3 to deploy to ECS. Both of these pipes require the following AWS variables:

AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION

but state in the README that they can be omitted if defined in the context.

I have defined these variables as deployment variables, which appears to work correctly for atlassian/aws-ecr-push-image:1.2.0 but not for atlassian/aws-ecs-deploy:1.0.3.

 

This fails on the ECS deploy step, stating it is missing the required AWS variables:

 - pipe: atlassian/aws-ecr-push-image:1.2.0
  variables: 
   IMAGE_NAME: $IMAGE_NAME 
   TAGS: '${COMMIT_HASH} latest' 
- pipe: atlassian/aws-ecs-deploy:1.0.3 
 variables: 
  CLUSTER_NAME: $ECS_CLUSTER_NAME 
  SERVICE_NAME: $ECS_SERVICE_NAME 
  TASK_DEFINITION: 'my_task_definition.json'
  DEBUG: "true"
  FORCE_NEW_DEPLOYMENT:"true" 

 

Even though both pipes have the exact same expected AWS variables, the ECR pipe receives the AWS variables as expected, but the ECS pipe does not. Explicitly setting the variables causes both pipes to work as expected:

- pipe: atlassian/aws-ecr-push-image:1.2.0
 variables: 
  IMAGE_NAME: $IMAGE_NAME 
  TAGS: '${COMMIT_HASH} latest' 
- pipe: atlassian/aws-ecs-deploy:1.0.3 
 variables: 
  AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
  AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
  AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
  CLUSTER_NAME: $ECS_CLUSTER_NAME 
  SERVICE_NAME: $ECS_SERVICE_NAME 
  TASK_DEFINITION: 'my_task_definition.json'
  DEBUG: "true"
  FORCE_NEW_DEPLOYMENT:"true" 

Is this expected behaviour? Can the variables only be passed through one pipe from context? What am I doing wrong here?  

Halyna Berezovska

@Shane McNamara also, try to split these executions into two steps, not one.

BEsides that, this looks like a bug inside pipeline infra, we will report this, thank you