It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

old azcopy in Azure Storage Deploy v0.5.0

hello, i cant use pipeline v0.5.0 to deploy files to container '$web' which is default name for static web container in Azure storage.

pipeline uses 7.2.0 version of azcopy, and this bug was fixed in 7.3.0 version of azcopy

https://github.com/Azure/azure-storage-net-data-movement/issues/133

 

please update azcopy to new version 

 

thanks

Greg

 

 

2 answers

1 accepted

1 vote
Answer accepted

Hi Greg,

Thanks for bringing this issue to our attention. We've been in touch with Microsoft and they've now released a new version of the pipe (1.1.1) that uses azcopy 7.3.0.

Hello, the issue seems to stay. Are there any additional problem known? 

Could you post the exact yaml snippet you're using to invoke the pipe, along with any relevant error output that you get when executing it?

Snippet:

 - pipe: microsoft/azure-storage-deploy:1.1.1
variables:
SOURCE: 'build/'
DESTINATION: 'https://xxxxxxxxxxxx.blob.core.windows.net/$web/'
DESTINATION_SAS_TOKEN: $AZURE_STORAGE_SAS_TOKEN
EXTRA_ARGS: '--verbose --set-content-type'

The problem seems to be rooted in the use of "$web". I tried various ways to escape the '$' but only with an environment variable i was able to get the pipe running without any error and correct behaviour. 

Like Martin Buchleitner likes this

I'm glad you've got it working. Quoting/escaping of special characters like '$' in pipe variables can be quite tricky because there are always at least two rounds of processing that can cause the characters to be interpreted as something other than a literal value. In the case of this pipe, the way it is currently implemented, there are actually three rounds of processing (two to pass the value into the pipe and a third round within the pipe itself).

Please have a look here for general information on quoting and escaping of pipe variables: https://community.atlassian.com/t5/Bitbucket-Pipelines-questions/How-do-I-properly-quote-Pipelines-pipe-variables-and-escape/qaq-p/1021713 This describes the first two rounds of processing.

So based on that information, it would appear that you need to do this to get the correct value into the pipe:

DESTINATION: 'https://xxxxxxxxxxxx.blob.core.windows.net/\$web/'

However as I mentioned this particular pipe actually does another round of processing which will cause the "$web" value to be treated as an environment variable expansion. So I believe the correct quoting/escaping for your example (for version 1.1.1 of the pipe) would be this:

DESTINATION: 'https://xxxxxxxxxxxx.blob.core.windows.net/\\\$web/'

I know that this is confusing and difficult to get right (I'm not even 100% certain myself that my suggestion will work without testing it). We're currently looking at how we can improve the pipe implementations to minimise the amount of escaping that's required.

Like # people like this

A potentially useful information regarding this:

I've managed to fix the issue by setting my destination environment variable to:

https://xxxxxxxxxxxx.blob.core.windows.net/\$web/

Without backslash or with three of them it did not work.

 

Addititionally, I'd suggest using EXTRA_ARGS variable from this pipe, and setting your environment variable for that to

--set-content-type

As that will keep original file type (otherwise you might ran into an issue that when opening your static website, it starts a download, because without that argument it sets default blob type to application/octet-stream, which you probably do not want).

 

However, all of this seems to be rather useless if your build is producing files with different hashes (in file name), like create-react-app / WebPack does - because thispipe  does not delete those previous files (and offers no flag to do so that I've found), you'd end up with all older files taking up your storage for no reason.

Hi Ilija,

Thanks for providing that extra information.

To clarify for others who might be confused (as I was initially), my example above is appropriate if you're configuring the destination (containing the dollar sign) directly in the yaml file. I'm guessing what Ilija did is put the destination (containing the dollar sign) into a repository variable using the Bitbucket UI which means that the first two rounds of processing (yaml and variable expansion) don't apply. Which is why only a single backslash is required to survive the final round of processing within the pipe.

0 votes

Hello Grzegorz,

Thank you for finding and reporting this issue with Bitbucket pipelines and azcopy version 7.2. I have submitted a feature request for this to have it reviewed. You may find the created feature request at Issue #18211.

We would suggest you vote and watch this issue so you may receive future updates on its status.

Regards,
Stephen Sifers

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Bitbucket Pipelines

Building a Bitbucket Pipe as a casual coder

...ipe.sh :  #!/bin/bash source "$(dirname "$0")/common.sh" enable_debug extra_args="" if [[ "${DEBUG}" == "true" ]]; then extra_args="--verbose" fi # mandatory variables R...

4,410 views 4 22
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you