Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,414,272
Community Members
 
Community Events
170
Community Groups

old azcopy in Azure Storage Deploy v0.5.0

hello, i cant use pipeline v0.5.0 to deploy files to container '$web' which is default name for static web container in Azure storage.

pipeline uses 7.2.0 version of azcopy, and this bug was fixed in 7.3.0 version of azcopy

https://github.com/Azure/azure-storage-net-data-movement/issues/133

 

please update azcopy to new version 

 

thanks

Greg

 

 

2 answers

1 accepted

2 votes
Answer accepted

Hi Greg,

Thanks for bringing this issue to our attention. We've been in touch with Microsoft and they've now released a new version of the pipe (1.1.1) that uses azcopy 7.3.0.

Hello, the issue seems to stay. Are there any additional problem known? 

Could you post the exact yaml snippet you're using to invoke the pipe, along with any relevant error output that you get when executing it?

Snippet:

 - pipe: microsoft/azure-storage-deploy:1.1.1
variables:
SOURCE: 'build/'
DESTINATION: 'https://xxxxxxxxxxxx.blob.core.windows.net/$web/'
DESTINATION_SAS_TOKEN: $AZURE_STORAGE_SAS_TOKEN
EXTRA_ARGS: '--verbose --set-content-type'

The problem seems to be rooted in the use of "$web". I tried various ways to escape the '$' but only with an environment variable i was able to get the pipe running without any error and correct behaviour. 

Like Martin Buchleitner likes this

I'm glad you've got it working. Quoting/escaping of special characters like '$' in pipe variables can be quite tricky because there are always at least two rounds of processing that can cause the characters to be interpreted as something other than a literal value. In the case of this pipe, the way it is currently implemented, there are actually three rounds of processing (two to pass the value into the pipe and a third round within the pipe itself).

Please have a look here for general information on quoting and escaping of pipe variables: https://community.atlassian.com/t5/Bitbucket-Pipelines-questions/How-do-I-properly-quote-Pipelines-pipe-variables-and-escape/qaq-p/1021713 This describes the first two rounds of processing.

So based on that information, it would appear that you need to do this to get the correct value into the pipe:

DESTINATION: 'https://xxxxxxxxxxxx.blob.core.windows.net/\$web/'

However as I mentioned this particular pipe actually does another round of processing which will cause the "$web" value to be treated as an environment variable expansion. So I believe the correct quoting/escaping for your example (for version 1.1.1 of the pipe) would be this:

DESTINATION: 'https://xxxxxxxxxxxx.blob.core.windows.net/\\\$web/'

I know that this is confusing and difficult to get right (I'm not even 100% certain myself that my suggestion will work without testing it). We're currently looking at how we can improve the pipe implementations to minimise the amount of escaping that's required.

Like # people like this

A potentially useful information regarding this:

I've managed to fix the issue by setting my destination environment variable to:

https://xxxxxxxxxxxx.blob.core.windows.net/\$web/

Without backslash or with three of them it did not work.

 

Addititionally, I'd suggest using EXTRA_ARGS variable from this pipe, and setting your environment variable for that to

--set-content-type

As that will keep original file type (otherwise you might ran into an issue that when opening your static website, it starts a download, because without that argument it sets default blob type to application/octet-stream, which you probably do not want).

 

However, all of this seems to be rather useless if your build is producing files with different hashes (in file name), like create-react-app / WebPack does - because thispipe  does not delete those previous files (and offers no flag to do so that I've found), you'd end up with all older files taking up your storage for no reason.

Hi Ilija,

Thanks for providing that extra information.

To clarify for others who might be confused (as I was initially), my example above is appropriate if you're configuring the destination (containing the dollar sign) directly in the yaml file. I'm guessing what Ilija did is put the destination (containing the dollar sign) into a repository variable using the Bitbucket UI which means that the first two rounds of processing (yaml and variable expansion) don't apply. Which is why only a single backslash is required to survive the final round of processing within the pipe.

The only option that worked for me was three slashes and no environment variables used.


It would be great if @microsoft could add this as a warning to their storage pipeline so people don't continually run into this issue.@Stephen Sifers 

0 votes

Hello Grzegorz,

Thank you for finding and reporting this issue with Bitbucket pipelines and azcopy version 7.2. I have submitted a feature request for this to have it reviewed. You may find the created feature request at Issue #18211.

We would suggest you vote and watch this issue so you may receive future updates on its status.

Regards,
Stephen Sifers

Suggest an answer

Log in or Sign up to answer
TAGS

Atlassian Community Events