How can run pipes on self-hosted runners behind proxy?

[Aaron.Luo]

We are using self hosted runners behind proxy.

Sometimes pipes need to connect to Internet. For example, "bitbucketpipelines/checkstyle-report" needs to report to http://api.bitbucket.org; "aquasecurity/trivy-pipe" needs to download database before scan images.

How can we setup HTTP_PROXY and HTTPS_PROXY for pipes?

[Patrik S]

Hello @Aaron.Luo ,

Thank you for reaching out to Atlassian Community!

At this moment, Pipelines Runners do not support using proxy, but we already have a feature request to implement that functionality, which you can find in the following link :

• https://jira.atlassian.com/browse/BCLOUD-21158

 I would suggest you to add your vote there, since this helps both developers and product managers to understand the interest. Also, make sure you add yourself as a watcher in case you want to receive first-hand updates from that ticket. Please note that all features are implemented with this policy in mind.

As a workaround, you could try forcing the docker daemon to use proxy while starting the runner container by including the following configuration in `~/.docker/config.json` :

  "proxies": {
    "default": {
      "httpProxy": "http://username:password@proxy2.domain.com",
      "httpsProxy": "http://username:password@proxy2.domain.com"
    }
  }

Source : https://docs.docker.com/network/proxy/

Hope that helps. Let us know in case you have further questions about this particular topic.

Thanks, @Aaron.Luo .

Kind regards,

Patrik S

[Aaron.Luo]

Thanks @Patrik S for your response.

The workaround didn't work. I tried to include the proxy configuration in `~/.docker/config.json` in the docker daemon then tested bitbucketpipelines/git-secrets-scan:0.6.0 and bitbucketpipelines/checkstyle-report:0.3.1 pipes, both failed.

I looked through the source code of both pipes. Seems both pipes tried to go through the bitbucket-pipelines-auth-proxy container to connect to api.bitbucket.org. I don't know if it's possible to have HTTP_PROXY and HTTPS_PROXY configured upon the bitbucket-pipelines-auth-proxy container.

[Patrik S]

Hello @Aaron.Luo ,

After checking internally, our development team confirmed that they still need to do some work before Runners behind a proxy is available.

In this case I would suggest you to keep a watch on the feature request BCLOUD-21158 to receive first-hand updates about any progress on the implementation.

Thank you, @Aaron.Luo .

Kind regards,

Patrik S

[Aaron.Luo]

Thanks @Patrik S . I've already commented under BCLOUD-21158. Will keep watching it.

[Aaron.Luo]

Hi @Patrik S ,

I noticed that BCLOUD-21158 has been closed and the pipeline-runner does pickup HTTP_PROXY/HTTPS_PROXY/NO_PROXY. But bitbucket-pipelines-auth-proxy still doesn't support running behind a corporate proxy.

Seems the bitbucket-pipelines-auth-proxy is a Nginx reverse proxy to a list of backend_upstreams such as “bitbucket.org” etc. We might be able to update the auth-proxy image by updating /etc/nginx/nginx.conf.template so it could redirect request to a corporate proxy server with this workaround. A sample of code snippet is as follows: 

http {
  ……

  upstream corp_proxy {
    server ${HTTPS_PROXY}
  }

  server {

    listen 29418;
    server_name bitbucket.org;

    set $backend_upstream "https://corp_proxy/${BITBUCKET_PORT}";
    set $backend_host "bitbucket.org";


    location ~ '^/(${BITBUCKET_WORKSPACE}/${BITBUCKET_REPO_SLUG}|${BITBUCKET_REPO_OWNER_UUID}/${BITBUCKET_REPO_UUID})' {
      proxy_pass $backend_upstream;
      proxy_set_header Host $backend_host;
      ……
    }
  ……

  }
  ……

}

Are you able to create another bug ticket for this and bring this with your team?

We were trying to use this solution in our internal, but our private docker registry must be authenticated. Unfortunately since BCLOUD-21417 can only support private docker registry without authentication, we didn't get a chance to test.

Appreciate if this issue could get resolved in any way.

Kind regards,

Aaron

[Patrik S]

Hello @Aaron.Luo 

Thank you for bringing this to our attention.

I've confirmed internally that indeed using a private docker registry that requires authentication is currently not possible when overriding runner's docker images. At this moment it's only possible to override Runner's images using unauthenticated registries.

As per your request, I went ahead and created the below feature request to implement that functionality :

• https://jira.atlassian.com/browse/BCLOUD-21753

Feel free to add a comment there giving more details about your use-case, so our developers have a better context when prioritizing the implementation. Also , make sure to add yourself as a watcher in the ticket, so you receive a notification whenever there's any update.

Let me know if you have any further questions.

Thanks, @Aaron.Luo .

[Aaron.Luo]

Hi @Patrik S ,

Thank you for creating the JIRA ticket.

I've added myself as a watcher. I've also added our use case and some thoughts for implementation. Hope this would help.

Kind regards,

Aaron

[Aaron.Luo]

Hi @Patrik S ,

Thanks for your team's effort to support BCLOUD-21753.

The custom AUTH_PROXY_IMAGE, CLONE_IMAGE and PAUSE_IMAGE can be pulled from our internal registry which requires authentication now. While custom CLONE_IMAGE runs as expected, AUTH_PROXY_IMAGE still runs with some issues.

We expect that AUTH_PROXY_IMAGE could get HTTP_PROXY, HTTPS_PROXY and NO_PROXY settings from $HOME/.docker/config.json within bitbucket-pipelines-runner when it's got running upon a build as we have different proxy settings in different environments / k8s clusters (See the above comment I posted on Mar 07, 2022). I've tested that the proxy settings are passed into a container when I ran a container with the `docker run some-image (eg. our custom AUTH_PROXY_IMAGE)` command. But seems that bitbucket-pipelines-runner didn't pickup $HOME/.docker/config.json as a standard docker client default behaviour while it runs the AUTH_PROXY_IMAGE upon a build.

This is the blocker for us to use any third party pipes that needs to report back to Code Insights in Bitbucket cloud. Is it possible to get this fixed with a high priority please? Thanks in advance.

Kind regards,

Aaron