How to get the result of Bitbucket pipeline script in next script?

[Michael Brizic]

I'm interested in learning the pattern or approach available that would allow me to do the following:

script:
  - pipe: custom-acct/custom-pipe:1.0.0
  - <check result of above pipe or some return value>

I am interested in letting the custom pipe succeed so that the script execution continues but in the next step of the script I would like to know some return value of executing the pipe, or perhaps check an environment variable. 

[Tom Bradshaw · Accepted solution]

Hi Michael,

We have a directory you can use to pass information out of pipes in the form of files placed in this directory. To obtain the result of a script execution make sure the pipe outputs the result to a file in this directory and you'll be able to read the result from that file in the next command or step.

You can find more information about the directory structure and accessing it here https://confluence.atlassian.com/bitbucket/advanced-techniques-for-writing-pipes-969511009.html#Advancedtechniquesforwritingpipes-Sharinginformationbetweenpipes

If you have any questions let me know.

Cheers,
Tom

[Michael Brizic]

Hi Tom,

Thanks, I thought that might be the approach however, when I'm writing/editing my pipeline file those variables mentioned in the document that you linked to don't appear available. But, I went ahead and wrote a file to the "shared" directory anyway and then attempted to access it like so:

- cat "$BITBUCKET_PIPE_SHARED_STORAGE_DIR/bb-commit-has-changes.txt"

But the response was that the directory returned by that variables was empty and thus the file I was "catting" was not found:

+ cat "$BITBUCKET_PIPE_SHARED_STORAGE_DIR/bb-commit-has-changes.txt"cat: /bb-commit-has-changes.txt: No such file or directory

Am I accessing incorrectly? Do I have access to that directory on a simple script line or do I have to be in another pipe??

[Tom Bradshaw]

Hey Michael,

Sorry about that. I didn't realise this but we don't include the storage dir variables in the list of variables we expose normally.

The location of the shared storage dir (the one you were using) is

/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes

To use one specific to that pipe it's the same path but appended with "<pipeRepo>/<pipeName>"

I'll also see if we can get that variable exposed as part of the calling step but I can't offer any guarantees on if and when that will happen.

Cheers,

Tom

[Michael Brizic]

Thanks.

Giving me that actual path helped, but I would have expected given your response that the file I created would have been under that path with <pipeAcct>/<pipeName> appended. However this is what I see:

+ ls -al /opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipestotal 16
drwxr-xr-x. 3 165536 165536 4096 Jun 18 11:14 .
drwxr-xr-x. 3 165536 165536 4096 Jun 18 11:14 ..
-rw-r--r--. 1 165536 165536    5 Jun 18 11:14 bb-commit-has-changes.txt
drwxr-xr-x. 3 165536 165536 4096 Jun 18 11:14 $DOCKERHUB_USERNAME

So if I cat the location of the path you gave with <pipeAcct>/<pipeName> appended I get this:

+ cat "opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/$DOCKERHUB_USERNAME/bitbucket-commit-has-changes-of-interest/bb-commit-has-changes.txt"
cat: opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/$DOCKERHUB_USERNAME/bitbucket-commit-has-changes-of-interest/bb-commit-has-changes.txt: No such file or directory

So it seems I must append <pipeAcct>/<pipeName> in my pipe as well if I expect the file I create to show up under that path, right?

Also, I pass in my <pipeAcct> name explicitly however, the pipeline is converting that to $DOCKERHUB_USERNAME and as you can see above, it is not resolving, returning a value. What gives? 

[Tom Bradshaw]

Hi Michael.

I have to apologise. I've looked into this more and I have given you incorrect information. The $BITBUCKET_PIPE_STORAGE_DIR and $BITBUCKET_PIPE_SHARED_STORAGE_DIR locations are used to pass information between docker pipes.

There is a much easier strategy that you can use to export your results.

Each pipe has the build directory mounted into the pipe itself. You can create a file in the pipe and the file should be placed into the same location in the build directory. You could even pass the location as a parameter to the pipe.

As for the reason this shows up in the shared storage directory it is because you are using the $BITBUCKET_PIPE_SHARED_STORAGE_DIR parameter as the location which does not have the <pipeAcct>/<pipeName> appended. If you use $BITBUCKET_PIPE_STORAGE_DIR it should have this information appended already.

If the $DOCKERHUB_USERNAME variable value matches your <pipeAcct> value we will mask it as we do with every secured variable value in a pipeline.

Cheers,

Tom.

[Michael Brizic]

Ok everything works for me based on all the above comments (and it all makes sense).

Thank you for all the clarifications and follow-up!

[Rupert Broad]

How does this work when the variable $BITBUCKET_PIPE_SHARED_STORAGE_DIR is not exported?

given you reply I would expect echo $BITBUCKET_PIPE_SHARED_STORAGE_DIR to produce a directory path but as it doesn't exist nothing happens