Bitbucket pipelines: How can I share a volume between two containers using Docker Compose?

[Eugenio Carocci]

Hi all,

I'd like to understand if there's a way to share a volume, in the Bitbucket pipeline context, between two containers defined in a `docker-compose` cluster at version 3.*.

To let everybody better understand my issue I have a `docker-compose.yml` like this:

version: "3.7"
# --------- #
# Services  #
# --------- #
services:
  ###########
  # PHP-FPM #
  ############
  php-fpm:
    image: custom/php-fpm
    volumes:
      - app-data:/var/www/html
  ######################
  # Apache Application #
  ######################
  www:
    image: custom/apache
    depends_on:
      - php-fpm
    volumes:
      - app-data:/var/www/html
# --------- #
# Volumes   #
# --------- #
volumes:
  app-data:

This is obviously a super simplification of my scenario.

Anyway, the key issue is that I'm not able to share the `app-data` volume since Bitbucket gives the following error:

"ERROR: for app-pipelines_php-fpm_1  Cannot create container for service php-fpm: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories
ERROR: for php-fpm  Cannot create container for service php-fpm: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories"

Thanks for your time :)

Eugenio

[Taras Postument]

Hey, I faced the same problem and ended up with the following hack.

Instead of defining the global volume (like your `app-data:`) I used dynamic volumes definition with the `BITBUCKET_CLONE_DIR` dir variable.

This way, when composing is running in the BB Pipeline volume is mounted int `BITBUCKET_CLONE_DIR` and when I run it locally, the standard volume is created.

- &static_volume ${BITBUCKET_CLONE_DIR:-static}:/usr/src/app/static

Then you can use it like

volumes: 
- *static_volume

In all the container definitions.

Also, I unsuccessfully tried other workarounds like:

volumes:
  static:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: ${BITBUCKET_CLONE_DIR:-/tmp}

or

volumes:
  static:
    driver: local
    driver_opts:
      type: tmpfs
      device: tmpfs

While both worked locally, none worked in the Pipeline.

Hope it helps!

[Stefano Angaran]

Hey Taras, thanks for posting this. I've never seen that syntax with ampersand and asterix used, could you give me some references to investigate a little bit more? Thank you very much!

[Taras Postument]

@Stefano Angaran 
Sure, it is called YAML Anchors and it is a feature of YAML language.

Using them is the only way to avoid terrible repetition int Pipelines definitions.

Here are some references:
1. https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/
2. https://docs.gitlab.com/ee/ci/yaml/#anchors
3. https://medium.com/@kinghuang/docker-compose-anchors-aliases-extensions-a1e4105d70bd

[Tyler Renslow]

Hi Taras, would you mind explaining where you defined the 

- &static_volume ${BITBUCKET_CLONE_DIR:-static}:/usr/src/app/static

part?

I understand that the other part is defined under the individual services, but I can't figure out where to put the first part in the docker compose file.

Cheers 

[Taras Postument]

Hi Tyler, basically, you can define it in one of two places:
1. Define anchor in the custom fields and reuse for services(https://docs.docker.com/compose/compose-file/compose-file-v3/#extension-fields)

like:

x-custom:
   volumes: 
   - &static_volume ${BITBUCKET_CLONE_DIR:-static}:/usr/src/app/static

service1:
   ...
   volumes: 
   - *static_volume 


service2:
   ...
   volumes: 
   - *static_volume

2. or you can define the anchor in the first service and reuse it all the subsequent services:

service1:
   ...
   volumes: 
   - &static_volume ${BITBUCKET_CLONE_DIR:-static}:/usr/src/app/static


service2:
   ...
   volumes: 
   - *static_volume

[Tyler Renslow]

Thanks for taking the time Taras. This is super helpful!

[Marc NALPAS]

hey, I get the same error, but I don't have any volumes in my docker files.

do you have any idea where this error come from?

[Aliaksandr Parfianiuk]

If you are using docker-compose in pipeline, try to install it via such command:

- curl -L --fail "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o ./docker-compose && chmod +x ./docker-compose
- mv ./docker-compose /usr/bin

[Marc NALPAS]

thanks for the answer the problem was that the docker file was generated by visual studio and it adds a docker-compose.override.yml with a shard volume for ssl configuration.

The problem was solved after removing it.