It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

For loop script syntax error Edited

I'm trying to use the AWS CLI to invalidate my CloudFront distribution. This consists of two commands, one to create the invalidation and then one to wait for it to complete.

To do this, I need to capture the invalidation ID that AWS sends back to me and pass it to the second command. After lots of Googling I have settled on the below script. Whilst not pretty, it serves it's purpose.

pipelines:
branches:
Stage/development:
- step:
name: Invalidate
image: cgswong/aws:aws
script:
- for /f usebackq %F in (`aws cloudfront create-invalidation --distribution-id "##############" --paths /* --query "Invalidation.Id"`) do aws cloudfront wait invalidation-completed --distribution-id "##############" --id %F

The command I've written works perfectly on my machine (Windows 10) but when it's run as part of the Pipeline, I get the following error:

bash: /opt/atlassian/pipelines/agent/tmp/bashScript6992979577238911185.sh: line 5: syntax error near unexpected token `usebackq'

I've tried putting usebackq in quotes but that didn't help - same error.

Could someone help with this please or offer another solution?

 

2 answers

1 accepted

0 votes
Answer accepted

Hi @Andy_Furniss

You will need to convert your command from CMD (Windows) to bash (Linux). The format of a loop in bash would be something like:

for F in $(command1); do command2 $F ;done

The command1 ideally should return values with quotes separated by space or line breaks. The quotes are only needed if the names have spaces in them because space is the separator of elements here.

The command2 would just embed the variable F on any part of it using $F to return the variable value.

I hope that helps.

Thanks a lot @Daniel Santos !

Sorry for the late reply, I haven't had a chance to get back on this task yet but I will let you know if I manage to get it working with your suggestion!

I figured it might be because it needed to be done with bash but I have zero bash experience so I couldn't figure out how to do it! Hopefully your method works :)

Like Daniel Santos likes this

No problem, take your time.
Let me know if you face any issues to make that work.

Hi Daniel. I think I've made some progress but I've got stuck on how to pass the variable to the --id option of the AWS CLI invalidation-completed command. I believe that this must be an issue with the syntax of my command which you can see below:

for F in $(aws cloudfront create-invalidation --distribution-id ############## --paths /* --query "Invalidation.Id"); do aws cloudfront wait invalidation-completed --distribution-id ############## --id $F ;done

 The above outputs "Waiter InvalidationCompleted failed: The specified invalidation does not exist." which is obviously a AWS error.

The strange thing is, if I replace the second command with an echo, it outputs a valid invalidation ID, like so:

for F in $(aws cloudfront create-invalidation --distribution-id ############## --paths /* --query "Invalidation.Id"); do echo $F ;done

 This outputs a string wrapped in double quotes, like this: "C3JTO0AJ91B4Q0".

Is there an issue with how I'm using the $F variable in my command? Perhaps something I'm not aware of with Bash?

Also, I've noticed that if I manually run the second command with the value given when just echo'ing the output, it works. So it's just something wrong with how the value is being passed to the second command it seems.

Sorted it! I had to add:

--output text 

to the create-invalidation command so that it's return value didn't have the double quotes. Passing this to the second command was then successful.

I wonder if there's a way to handle the value when it's wrapped in double quotes though? Just for future reference :)

This is great!

I was not able to work on this yesterday, but I'm really glad that you were able to work this out.

If you think this answer can help others, please mark it as accepted.

Thank you for sharing your findings here =]

Have a good one!

you can try this post about loop in bash

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket Pipelines

What We Learned When We Researched Open Source Vulnerabilities in 7 Popular Coding Languages

...hey are a part of us, shaping how we interact with the world around us. The same holds true for programming languages when we think about how different kinds of vulnerabilities raise their heads in t...

1,041 views 0 3
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you