bitbucket-upload-file:0.1.8 unzips tar.gz on upload

[Mark Nelissen]

I try to make available the resulting tar.gz from the pipeline build in the Download area of the Bitbucket repository. For this, I use a packing step with the following script:

- mkdir -p plugin/webapp/dist 
- cp -a webapp/dist plugin/webapp 
- cp plugin.json plugin 
- tar -czvf plugin.tar.gz plugin 
- pipe: atlassian/bitbucket-upload-file:0.1.8 
  variables: 
    BITBUCKET_USERNAME: $BITBUCKET_USERNAME 
    BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD 
    FILENAME: 'plugin.tar.gz'

The plugin.tar.gz does appear in the download area, but when I download it (from the UI), it is no longer gzipped.

The problem lies in the plugin (or the download section), since if I download the artifact directly from the build pipeline step, the plugin.tar.gz is correctly gzipped.

[ktomk]

@Mark Nelissen Interesting insight and just to put it upfront, this is more a comment than an answer, I think someone from Atlassian should also comment on it as they have much more insight and knowledge.

From the description as I read it it sounds like as if - and I guess a bit here that you download it with a browser - the gz-ipped artifact is unpacked while downloading. I can imagine that this is perfectly the case as many webservers are configured this way. This allows to store pre-gz-ipped artifacts statically on a webserver so that the webserver must not gz-ip them on-the-fly before downloading (gz compression is one of the cornerstones for compressed transfer encoding in HTTP, there are others, just saying).

If this assumption/guess is what is happening here, then there is little to do without having changes on the webserver configuration which somewhat is out of scope.

The only work-around at hand - no guarantee that it works, it's merely a try - is to change the file extension from `.tar.gz` to something different, maybe `.tgz` to keep some commons or to `.tar.gz.bin`. But even thought, if the HTTP client inspects the stream and identifies it as being a gz stream (which works by just inspecting the very first bytes), it might still decompress the stream before persisting on the clients local file-system.

To further inspect I would suggest to tinker with `curl` instead of a browser as first of all it has many options to control such kind of behavior and also options to gather much more information a browser would never show (e.g. `-v` for verbosity). This depends a bit how comfortable you feel with command line usage and the `curl` utility in specific.

If your project is public and you can share an URL or two, it would be easier to say more here after running some commands. Nevertheless as I wrote at the beginning of my comment, I somewhat feel that someone from Atlassian can shed more light here.

/cc  @Halyna Berezovska ping. This might be of interest either for you or one of your fellow colleagues.

[Mark Nelissen]

Thank you for your detailed explanation.

The project is not public, unfortunately, since it is a client project.

I do feel comfortable with the command line, but using it kind of defeats the use of the bitbucket pipeline for packaging the deliverable: simplicity. If I resort to the command line, I might as well build the solution locally (we do have a script that does all the building equivalent of the pipeline). It would only take 2-3 commands. Especially since the download is not public, making the curl call more complex.

The reason to push the deliverable to the Download section, is to be able to perform all actions (including deploying the deliverable) through the browser, without having to also include the command line (because in that case it is faster to just build locally). I have a similar usability issue if I publish the deliverable as artefact in the build pipeline: the artefact which is itself a single .tar.gz, is wrapped in a secondary .tar.gz. On macOS, when using the default unpacker (so without resorting to command line tar commands), it unpacks both levels, and I end up with the content of the .tar.gz on my disk, instead of the .tar.gz that is inside the .tar.gz from the pipeline artefact packaging.

I am aware that a solution could be for this project to try to add another step to the build that does the deploying for me, but since the project is coming to an end, I don't have the time to investigate the necessary API's on the target solution to make this possible. Anyway, I've openend this issue more as a kind of bug report, since it is not OK that a file published to the download section is modified from the file you upload. Especially not for a rather common file type for packaging final product build like .tar.gz.

[Halyna Berezovska]

@Mark Nelissen thanks for reporting this! we will investigate on improve cases for this pipe to make this work also

Regards, Galyna

[Halyna Berezovska]

hello @Mark Nelissen , we have investigated the case. *.tar.gz are uploading and then downloading fine with UI and wget tool. Perhaps, the type of your browser or content inside your archive matter.  If you still have problems with this, tell us, we will try to help

Regards, Galyna

[Mark Nelissen]

Hi @Halyna Berezovska ,

The problem is indeed gone. When downloading the file from the download section, uploaded through the atlassian/bitbucket-upload-file:0.1.8 pipe, it now retains the correct compression format. I am no longer capable of reproducing the behaviour.

Kind regards,

Mark