I have a plan/job that runs on a WIndows OS based agent and produces a shared artifact comprise of multiple files.
On another plan/job that runs on a Linux based agent I use the Artifact Download task to copy these artifacts into the build.
The problem is that instead of the directory tree being recreated the Artifact Download task creates "files" in the root of the build folder that have literal backslash embedded.
It would seem that the notion of "path separator" is not conveyed properly via this mechanism.
Example:
The produced (on Windows) shared artifact might be: foo/bar/data.txt
The staged (on Linux) artifact ends up being a file named foo\bar\data.txt
This, of course, is complete nonsense.
Is there some option / trick that needs to be enable to make this work?
TIA
Dave Cattley
1. Wait for Atlassian to fix it.
or
2. Use some other tool / container (e.g. ZIP, TAR, whatever) to pack the artifact files into a single object and unpack it on the destination.
Hi David,
There was a bug in the past (reported for our cloud instances - In that time we had Bamboo cloud) which has the exact behavior you are describing now. This is the one: Artifact folders produced on Windows are being published as files.
I was trying to reproduce the same issue in a recent Bamboo version (6.7.2) and I was unable to reproduce it.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Version: 6.6.1
The artifact is defined with a pattern (like the following):
Location: <blank>
Pattern: **/foo/bar/*.*
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
[aside] I lament the demise of Bamboo Cloud ...
After some strategy analysis our company released that Bitbucket Pipelines would be a better fit for a Cloud offer and this is why Bamboo is only offered as self hosted instances.
Getting back to the artifact thing, I've tried to reproduce this issue also in Bamboo 6.6.1 using a remote agent on Windows 10 and consuming the artifact on a Linux agent with multiple artifact definitions based on your description but no luck yet.
I've opened a support request for you so we can investigate this further and get more details from your server that could not be shared in this channel. You should receive a confirmation email from our support website.
I would kingly ask you to share here the resolution you get from our support channel so other users that come to this thread will be able to take advantage of our work together and save their time.
Thank you!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
More Info:
The artifacts in question are stored on S3. The issue appears to be that when a pattern such as:
**/foo/bar/*.*
matches an artifact for example:
dir1/dir2/foo/bar/file.ext
on a Linux Agent the resulting object in the S3 bucket is named as one would expect:
<s3-bucket-root>/<bamboo-artifact-name-path>/dir1/dir2/foo/bar/file.ext
The problem occurs when a Windows agent produces the same thing the "match" looks like this:
dir1\dir2\foo\bar\file.ext
and the object in S3 is named
<s3-bucket-root>/<bamboo-artifact-name-path>%5cdir1%5dir2%5cfoo%5cbar%5cfile.ext
where the "\" characters returned from path translation of the pattern match in Windows are now treated as literal characters in a filename instead of path separators.
My best guess is that the S3 artifact handler should be performing some sort of path normalization to the "path separator" expected by S3 (which evidently is not accepting "\").
When a Linux Agent downloads those artifacts it really has little choice to but to serialize them locally with the filename they have. Someone could have actually *meant* to produce a filename with a backslash in it in the Linux world (though they should probably be exiled to a far away planet for doing so reserved for clever folks that also put backspace in their filenames as a "security measure" ...).
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
AWS/S3 has this to say about putting a backslash in an object key name:
Characters to Avoid
Avoid the following characters in a key name because of significant special handling for consistency across all applications.
Backslash ("\")
Left curly brace ("{")
Non-printable ASCII characters (128–255 decimal characters)
Caret ("^")
Right curly brace ("}")
Percent character ("%")
Grave accent / back tick ("`")
Right square bracket ("]")
Quotation marks
'Greater Than' symbol (">")
Left square bracket ("[")
Tilde ("~")
'Less Than' symbol ("<")
'Pound' character ("#")
Vertical bar / pipe ("|")
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Thank you for sharing all those details here they are very insightful.
I'm following your support ticket with the Bamboo team and as soon we get a conclusion there we will be able to share the cause of this issue here. I can see it is related to how Bamboo is handling the Artifact storage to S3 on windows machines. We just need to check in what conditions and open a bug for this.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
By the way, I agree that your workaround is the best way to move forward at the moment (an archive).
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Daniel Santos what ever became of this? Anything? Has it been fixed in some release since 6.9.1 ?
thx
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Putting this into a script task after the Artifact Download task will "fixup" (move) the artifact files to the expected location(s) in the local file system.
#!/bin/bash
for _src in `find . -type f -regex '.*[\\]+.*'` ; do
_dst="${_src//\\//}"
mkdir -vp $(dirname $_dst)
mv -v "$_src" "$_dst"
done
I am a shell script novice so YMMV but this seems to have done what I need it to do.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.