Use of Docker Compose and Python 3.11 in bitbucket pipeline

[Pasquale Lafiosca]

To test my repo, I need to run:
```

docker compose build

docker compose up -d
```
inside the pipeline. I also need Python 3.11.

First, I am not sure what image to use.
After several attempts I went back to the recommended `atlassian/default-image:4` and manually installing python at the beginning. I don't like it but it works.

Second, I tried to use the docker service but apparently there is no docker compose in it.
Failed miserably.

Third, I manually installed all the dependencies using the commands as per official Docker documentation https://docs.docker.com/engine/install/ubuntu/
However, I get the error command not found: tee, even after installing the necessary package with apt install -y --reinstall coreutils

What is the easiest way to test my code that depends on docker compose V2 AND Python 3.11???

Thanks.

[Theodora Boudale · Accepted solution]

Hi Pasquale and welcome to the community!

Docker compose is available in the image atlassian/default-image:4 as docker-compose. Please keep in mind that Docker compose v2 (which is preinstalled in atlassian/default-image:4) won't work with buildkit. Docker compose with buildkit builder by default requires privileged mode, which is disabled in Bitbucket Pipelines for security reasons. If you want to use Docker compose v2, you will need to disable buildkit by adding the command export DOCKER_BUILDKIT=0 in your yml file, in the Pipelines step that uses docker-compose. Restricted Docker commands and options are listed in this page:

• https://support.atlassian.com/bitbucket-cloud/docs/run-docker-commands-in-bitbucket-pipelines/

Installing Docker Engine during your build won't work. Every pipelines step runs in a Docker container, and unless you use the docker service that Pipelines provide your build won't have access to a Docker daemon.

You can build your own Docker image that includes both python:3.11 and docker-compose, and use that as a build container in your Pipelines builds. You can install Docker Compose standalone (in images that don't include it) by following the instructions on this page:

• https://docs.docker.com/compose/install/standalone/

You can use as base image python:3.11, or you could use a different one, e.g. alpine, ubuntu, debian and then install Python 3.11, docker-compose and any other tools you need by adding the necessary commands in the Dockerfile. You can then build and push this image to a Docker registry and reference it in your yml file. We support public and private Docker images including those hosted on Docker Hub, AWS, GCP, Azure and self-hosted registries accessible on the internet:

• https://support.atlassian.com/bitbucket-cloud/docs/use-docker-images-as-build-environments/

Finally, if you want to run docker commands that are restricted in Pipelines, you can use a self-hosted runner instead for a specific Pipelines step:

• https://support.atlassian.com/bitbucket-cloud/docs/runners/

If you use a Linux Docker runner, you will need to define a custom dind image for the Docker service, so that you won't run into the same restrictions as cloud-based builds:

• https://support.atlassian.com/bitbucket-cloud/docs/configure-your-runner-in-bitbucket-pipelines-yml/#Custom-docker-in-docker-image

Please feel free to let me know if you have any questions.

Kind regards,
Theodora

[Pasquale Lafiosca]

Thanks. I'll look into this.

[Theodora Boudale]

You are very welcome. Please feel free to reach out if you have any questions!

[Sahir Maharaj]

Hello @Pasquale Lafiosca

From my experience, using Docker Compose and Python 3.11 in Bitbucket Pipelines requires careful image selection and setup. The recommended atlassian/default-image:4 works, but it’s inefficient to manually install Python.

Consider creating a custom Docker image that includes both Python 3.11 and Docker Compose pre-installed to streamline your pipeline.

For the error with tee, ensure that all required dependencies are installed in your pipeline’s environment. If the issue persists, consult Docker documentation or contact Atlassian support at https://support.atlassian.com/contact/#/ for guidance on optimizing your setup.

[Pasquale Lafiosca]

Thanks. I think custom docker images are the way to go.