pipeline: build and push docker image with custom HTTPS root-ca-certificate?

Question

Hi there,

I have a hopefully not so uncommon issue, so that someone can easily point me to a working solution (googl'ing did not help so far...):

We have bitbucket repostory that needs to build and push an docker image to a docker registry.

As the docker registry we want to use is a company-local-LAN system, we configured a self-hosted runner in docker-style.

Pipeline yml looks like this:

image: atlassian/default-image:3

pipelines:

custom: 
      dockerbuild:
        - step:
            name: "docker build"
            runs-on:
              - self.hosted
              - linux        
            script:
            # Prepare Docker Image with ZScaler certificates etc...
              - | 
                bash <( curl --header "Authorization: Bearer $PIPELINE_HELPER_BEARERTOKEN" -sL --url "https://api.bitbucket.org/2.0/repositories/mycompany/pipelinehelperscripts/src/HEAD/PrepareDockerImage" )

# do docker stuff
              - docker login dockerreg.mycompany.com -u $ISE_DOCKER_USER -p $ISE_DOCKER_PASS
              - docker build -t dockerreg.mycompany.com/translationservice
              - docker push dockerreg.mycompany.com/translationservice
            services:
              - docker

For securit reasony, the company is running Z-Scaler service, which hooks into every HTTPS connection with its own root-ca-certificate.

For "normal linux commands" like wget or curl, we need to put the z-scaler root ca certificate into a /usr/local/share/ca-certificates/ and run "update-ca-certificates". That's what the bash/curl command in the pipeline is doing...

But the docker commands totally ignore the certificates I added.
I also tried to apply the certificates to the docker-host system. 
To make it clear:
On both system (docker host and docker container) I can use the certificates with wget, curl, ...

The docker-host system can successfully login to our registry without certificate issues.

But only on docker container level (run by the pipeline runner) the applied root-ca-certificate of Z-Scaler is *NOT* recognized by any docker command.

I'm pretty sure that other people with self-signed certificates do face the same problem.
So, is there someone out there who can give me a hint?

br,

Alex

[update]
What I found out so far:
If I run "docker version" inside the pipeline, I see that docker is somehow not using the docker server from the docker-host. That would explain why my docker container does now know the root-ca like my docker-host.
I found this documentation: https://support.atlassian.com/bitbucket-cloud/docs/run-docker-commands-in-bitbucket-pipelines/#Using-an-external-Docker-daemon

But: I have no clue how I can tell my pipeline to use the docker daemon from my docker-host system...

Alexander Christian · Answer

I built my own image based in docker:dind image. I stopped playing with certificates and just configured docker's daemon.json to skip certificate checking (=insecure registry).

Pipeline yaml starts like this:

image: atlassian/default-image:3

definitions:
  services:
    docker: 
      image: 
        name: dockerreg.mycompany.com/mydind:latest
        username: $DOCKER_USER
        password: $DOCKER_PASS

Now I can use the docker service in the pipeline with MY docker variant... Works quite well so far. Now fighting with certificate issues on a different level :-)

Alexander Christian · Answer

One further question: Doesn't need DIND the "--priviledged" flag when started? See step 3: Docker Inside Docker. Docker has undoubtedly transformed the… | by Shivam kushwah | Medium

So, even if I create my own DIND image with my required CA, how can I tell the pipeline that this image needs to run in priviledged mode? Or is "priviledged" mode automatically applied when having the "service: - docker" added to yaml?

According to the pipeline documentation (Run Docker commands in Bitbucket Pipelines | Bitbucket Cloud | Atlassian Support), this flag is not allowed due to security reasons. But the documentation also says:

These restrictions, including the restricted commands listed below, only apply to the pipelines executed on our cloud infrastructure. These restrictions don't apply to the self-hosted pipeline Runners.

But the documentation does not tell me how to set the priviledged flag when doing DIND (which is obviously required when running docker commands in docker) when using own docker runner (=not in cloud).

Alexander Christian · Answer

Looks like an option. What I found out regarding the used DIND image: Pipeline says on startup Screenshot 2024-07-18 094942.png

So for DIND, an kind of custom DIND image is used. Woud be great to know what the dockerfile says so that I know how it is build. But I did not find the source to it. Would be great I you have some more details on it. In the meantime I'll try put my own DIND image together.