Hey @Antoine Drabble, welcome to Atlassian community
My question is : what is the difference in syntax between bitbucket-pipelines and docker-compose?
There are no syntax differences on the level of YAML, both Docker Compose and Bitbucket Pipelines are using the same YAML file-format.
However they are entirely different on the level of the Docker Compose and Bitbucket Pipelines own syntax and both have very different semantics. Just copying over from one file to another won't do it as you already have found out.
If I may take the YAML from your question as an example, Docker Compose container representation is services where in Bitbucket pipelines it is foremost pipelines, the meaning of services in definitions in Bitbucket Pipelines may appear similar but is different (compare the docs of Docker Compose and Bitbucket Pipelines for the full picture of the differences).
For example: In Bitbucket Pipelines you can create a pipeline that is running docker compose. Some references about this:
Alternative to that is having a pipeline with the build image (minio/mc:latest in your case?) making use of a service defined from the service image (minio/minio:edge).
What is the entrypoint might eventually become the script of the pipeline.
Example 1: Minio Service in Bitbucket Pipelines
Porting your Docker Compose example to Bitbucket Pipelines as a quick command-line example (making use of the pipelines utililty and the shell, it includes the Bitbucket Pipelines yaml as a here document):
$ <<BBPL pipelines --file -
---
# Example pipelines file with minio service
pipelines:
default:
- step:
name: minio service example
image: minio/mc:latest
script:
- mc config host rm local
- mc config host add --api s3v4 local http://127.0.0.1:9000 minio minio123
- mc rb --force local/ciip-support/ || true
- mc mb local/ciip-support/
services:
- minio
definitions:
services:
minio:
image: ktomk/pipelines:minio-server # from minio/minio:edge
variables:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: minio123
BBPL
+++ step #1
name...........: "minio service example"
effective-image: minio/mc:latest
container......: pipelines-1.minio-service-example.default.pipelines
container-id...: fb424c4dcfda
+++ copying files into container...
+ mc config host rm local
Removed `local` successfully.
+ mc config host add --api s3v4 local http://127.0.0.1:9000 minio minio123
Added `local` successfully.
+ mc rb --force local/ciip-support/ || true
mc: <ERROR> Unable to validate target `local/ciip-support/`. Bucket `ciip-support` does not exist.
+ mc mb local/ciip-support/
Bucket created successfully `local/ciip-support/`.
➜ pipelines git:(test) ✗
Example Notes:
- First code-block is the command, `$` symbolizes the shell prompt
- Second code-block is the output of that command
- The image `docker.io/minio/minio:edge` can not be used as-is for a Bitbucket Pipelines service as far I could find out quickly. The reason is the `command:` part in the Docker Compose file, the `CMD ["minio", "server", "/data"]` can be moved into a dedicated service image (here ktomk/pipelines:minio-server, I'll add two-line dockerfile below)
- Technically running both `mc config host rm local` and `mc rb --force local/ciip-support/` is not necessary in Bitbucket Pipelines as the Containers are always starting fresh. This is the reason why the bucket removing command is suffixed with `|| true` as the command to remove the bucket ("rb") does always fail and would otherwise stop the pipeline script. (Nevertheless it's fine to have these when keeping containers with the pipelines utility which has an option for that and it speeds up writing the example, so I have kept it in)
This example shows how it could look like in Bitbucket Pipelines compared to the Docker Compose example/port I could gather from the code-block in your question.
As noted, the Minio service docker container image needs to be tweaked a bit to work out of the box with Bitbucket Pipelines. Here a Dockerfile:
FROM docker.io/minio/minio:edge
CMD ["minio", "server", "/data"]
Reason is that Bitbucket Pipelines for the service definitions has no counter-part to the `command` property in Docker Compose services. It is much more simplified and expects service container images to be configurable with environment variables (this is normally common, I personally do not know Minio too much in this regard so writing the two-line dockerfile was just easier than searching the Minio environment parameters for the "server" subcommand and to specifiy the storage location for this example).
So just take it as what it is, an example that can be validated, for demonstration purposes.
The image `ktomk/pipelines:minio-server` is available on Docker Hub so it can be tested on Bitcuket Pipelines as well, but I have skipped this step so far while writing the answer (edit: it runs flawlessly on Bitbucket Pipelines as well, verfieid it after the initial posting).
Example 2: Minio Docker Compose inside Bitbucket Pipelines
This is the variant mentioned earlier to run `docker compose` in a pipeline. Again a quick command-line example with the pipelines utility. Note that this time you need to have the `docker` service in the pipeline step:
$ <<BBPL pipelines --file -
---
# Example pipelines file with docker compose
pipelines:
default:
- step:
name: docker compose example
image: docker/compose:latest
services:
- docker
caches:
- docker
script:
- |
<<'DC' docker-compose --file - up --abort-on-container-exit
version: "3.8"
services:
minio:
image: minio/minio:edge
container_name: minio
expose:
- "9000"
environment:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: minio123
command: server /data
mc:
image: minio/mc:latest
depends_on:
- minio
entrypoint: >
/bin/sh -c "/usr/bin/mc config host rm local
/usr/bin/mc config host add --quiet --api s3v4 \
local http://minio:9000 minio minio123
test `curl -o /dev/null -sw '%{http_code}' \
http://minio:9000` = 403 || sleep 1 \
&& test `curl -o /dev/null -sw '%{http_code}' \
http://minio:9000` = 403 || sleep 1
/usr/bin/mc rb --force local/ciip-support/
/usr/bin/mc mb --quiet local/ciip-support/"
DC
BBPL
(yes that is a Docker Compose document inside a Bitbucket Pipelines document); output:
+++ step #1
name...........: "docker compose example"
effective-image: docker/compose:latest
container......: pipelines-1.docker-compose-example.default.pipelines
container-id...: 034ac6522b86
+++ copying files into container...
+ <<'DC' docker-compose --file - up --abort-on-container-exit
version: "3.8"
services:
minio:
image: minio/minio:edge
container_name: minio
expose:
- "9000"
environment:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: minio123
command: server /data
mc:
image: minio/mc:latest
depends_on:
- minio
entrypoint: >
/bin/sh -c "/usr/bin/mc config host rm local
/usr/bin/mc config host add --quiet --api s3v4 \
local http://minio:9000 minio minio123
test `curl -o /dev/null -sw '%{http_code}' \
http://minio:9000` = 403 || sleep 1 \
&& test `curl -o /dev/null -sw '%{http_code}' \
http://minio:9000` = 403 || sleep 1
/usr/bin/mc rb --force local/ciip-support/
/usr/bin/mc mb --quiet local/ciip-support/"
DC
Starting minio ... done
Recreating app_mc_1 ... done
Attaching to minio, app_mc_1
mc_1 | Removed `local` successfully.
mc_1 | Added `local` successfully.
minio | Endpoint: http://172.20.0.2:9000 http://127.0.0.1:9000
minio |
minio | Browser Access:
minio | http://172.20.0.2:9000 http://127.0.0.1:9000
minio |
minio | Object API (Amazon S3 compatible):
minio | Go: https://docs.min.io/docs/golang-client-quickstart-guide
minio | Java: https://docs.min.io/docs/java-client-quickstart-guide
minio | Python: https://docs.min.io/docs/python-client-quickstart-guide
minio | Javascript: https://docs.min.io/docs/javascript-client-quickstart-guide
minio | .NET: https://docs.min.io/docs/dotnet-client-quickstart-guide
mc_1 | Removed `local/ciip-support/` successfully.
mc_1 | Bucket created successfully `local/ciip-support/`.
app_mc_1 exited with code 0
Stopping minio ... done
Aborting on container exit...
Example Notes:
- The Docker Compose document has been re-constructed from the code-block in your question
- The "mc" container needs to wait for minio to be healthy. Added a test with curl which was not part of your original question (same might apply to the first example)
This second example shows that it is entirely possible to just create a pipeline and let run docker compose in it. In such a scenario, you would not need to port the Docker Compose YAML to Bitbucket Pipelines YAML.
Some limitations might apply to docker compose in bitbucket pipelines, which can be due to security settings, but those are in the details and this should not stop you using docker compose in Bitbucket Pipelines.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.