Bitbucket YML - External Linux Runner - How do I upload artifacts to Bitbucket?

[James Merkle]

I use an external Linux runner that builds the C/C++ code.

Desired outcome:
When the build is complete, I want to get the artifacts uploaded from the runner to the cloud so they show up in the "Downloads" area.

Links to similar topics:
https://support.atlassian.com/bitbucket-cloud/docs/deploy-build-artifacts-to-bitbucket-downloads/
https://support.atlassian.com/bitbucket-cloud/docs/use-artifacts-in-steps/
https://support.atlassian.com/bitbucket-cloud/docs/publish-and-link-your-build-artifacts/

The "pipe: atlassian/bitbucket-upload-file:0.1.2" method appeared the most promising, but it appears the build needs to take place within the Bitbucket cloud, not an external runner for this method to work.

Defining artifacts within the YML also appears to require the build to take place within the Bitbucket cloud for them to show up in "Downloads"

Using "curl" to deploy artifacts doesn't seem to perform the desired outcome .

Since I have multiple runners, one for building - using "Windows Tools", and another using Linux, I'd like to find a solution that both external runners can use.

Suggestions?

[Theodora Boudale · Accepted solution]

Hi James and welcome to the community!

Pipes can be used in builds running on Atlassian infrastructure and on Linux Docker runners.

If you are using a Linux Shell runner, you can use our API to upload a file to the Downloads section of a repo:

• https://developer.atlassian.com/cloud/bitbucket/rest/api-group-downloads/#api-repositories-workspace-repo-slug-downloads-post

You can include the curl command that uses our API in the yml file.

If you're still facing issues, could you please provide the following info so I can better help you?

• Please confirm if you are using a Linux Shell runner and the version of the runner you are using
• Share the command you are running in your yml file (after sanitizing any private/sensitive data) and the full output of the command

Please keep in mind that if you upload a file with the same name as an existing file in the Downloads section, then the existing file will be replaced.

Kind regards,
Theodora

[James Merkle]

Using external Ubuntu 22.04.3 LTS with runner version 1.518 (Linux Shell)

Command:
curl -v -X POST "https://api.bitbucket.org/2.0/repositories/merkles/nucleo-f103rb_blinky/downloads" --form files=@"textfile1.txt"

Output of the curl command:

+ curl -v -X POST "https://api.bitbucket.org/2.0/repositories/merkles/nucleo-f103rb_blinky/downloads" --form files=@"textfile1.txt"
Note: Unnecessary use of -X or --request, POST is already inferred.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 18.205.93.6:443...
* Connected to api.bitbucket.org (18.205.93.6) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS header, Certificate Status (22):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [88 bytes data]
* TLSv1.2 (OUT), TLS header, Finished (20):
} [5 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS header, Finished (20):
{ [5 bytes data]

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* TLSv1.2 (IN), TLS header, Certificate Status (22):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [155 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [15 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2772 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [79 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=California; L=San Francisco; O=Atlassian US, Inc.; CN=*.bitbucket.org
* start date: Apr 28 00:00:00 2023 GMT
* expire date: Mar 28 23:59:59 2024 GMT
* subjectAltName: host "api.bitbucket.org" matched cert's "*.bitbucket.org"
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
* SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x559cde6e8e90)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
> POST /2.0/repositories/merkles/nucleo-f103rb_blinky/downloads HTTP/2
> Host: api.bitbucket.org
> user-agent: curl/7.81.0
> accept: */*
> content-length: 227
> content-type: multipart/form-data; boundary=------------------------bcba21fd8f27154b
> 
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* We are completely uploaded and fine
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [202 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 32)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
< HTTP/2 401 
< server: envoy
< vary: Authorization, Origin
< www-authenticate: Basic realm="Bitbucket.org HTTP"
< cache-control: max-age=0, no-cache, no-store, must-revalidate
< content-type: text/html; charset=utf-8
< x-b3-traceid: 48dabbe416cd9526
< x-usage-output-ops: 0
< x-used-mesh: False
< x-dc-location: Micros-3
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< date: Fri, 03 Nov 2023 16:29:26 GMT
< x-request-id: 48dabbe416cd9526
< x-usage-user-time: 0.017692
< x-usage-system-time: 0.000000
< x-served-by: a7a2391e24ef
< expires: Fri, 03 Nov 2023 16:29:26 GMT
< x-xss-protection: 1; mode=block
< x-envoy-upstream-service-time: 29
< x-view-name: bitbucket.apps.downloads.api.v20.handlers.DownloadsHandler
< x-b3-spanid: 55d9ff5b03a70564
< x-static-version: 0a88acc2851f
< x-content-type-options: nosniff
< x-accepted-oauth-scopes: repository:write
< x-render-time: 0.013230085372924805
< x-trace-id: 48dabbe416cd9526
< x-usage-input-ops: 0
< x-version: 0a88acc2851f
< x-request-count: 3218
< x-frame-options: SAMEORIGIN
< x-cache-info: not cacheable; request wasn't a GET or HEAD
< content-length: 0
< 
{ [0 bytes data]

100 227 0 0 100 227 0 776 --:--:-- --:--:-- --:--:-- 774
* Connection #0 to host api.bitbucket.org left intact

=======================================================
The verbose output of the curl command appears to indicate the file was
uploaded as expected, but no file appears in the "Downloads" area.
=======================================================
 
Feel free to visit this test repository.  I've made it public.
https://bitbucket.org/merkles/nucleo-f103rb_blinky

=======================================================
Question:  Does this API allow an upload to the repository 'Downloads' area,
independent of the YML file?
Could I push a file at any time from a command prompt?

[Theodora Boudale]

Hi James,

Thank you for the info.

I see in the output of the curl command a status code of 401 which indicates an authorization issue because there are no authentication credentials in the call.

If you are a repo admin, you can generate a Repository Access Token from the repo's Repository settings > Access tokens. The token needs to have Repository - Write permissions for this API call to succeed. Copy the token that is generated.

Then, from Repository settings go to Repository variables (section PIPELINES) and create a secured variable with name e.g. access_token and value the value you copied from the earlier step.

Afterwards, adjust the API call in your yml file as follows:

curl -v -X POST "https://api.bitbucket.org/2.0/repositories/merkles/nucleo-f103rb_blinky/downloads" --form files=@"textfile1.txt" --header "Authorization:Bearer ${access_token}"

Please feel free to let me know if the call works then.

Does this API allow an upload to the repository 'Downloads' area, independent of the YML file?
Could I push a file at any time from a command prompt?

Yes, indeed you can. You can run API calls from your own computer or from another server that has access to the internet (or is at least accessible from Bitbucket's IP addresses, in case it is behind a firewall).

Please feel free to let me know if you have any questions.

Kind regards,
Theodora

[James Merkle]

Using the Repository Access Token did the trick!
Thank you!

[Theodora Boudale]

That's good to hear and you are very welcome!

Please feel free to reach out if you ever need anything else!