Bitbucket pipelines: How can I share a volume between two containers using Docker Compose?

Eugenio Carocci October 25, 2019

Hi all,

 

I'd like to understand if there's a way to share a volume, in the Bitbucket pipeline context, between two containers defined in a `docker-compose` cluster at version 3.*.

 

To let everybody better understand my issue I have a `docker-compose.yml` like this:

version: "3.7"
# --------- #
# Services #
# --------- #
services:
###########
# PHP-FPM #
############
php-fpm:
image: custom/php-fpm
volumes:
- app-data:/var/www/html
######################
# Apache Application #
######################
www:
image: custom/apache
depends_on:
- php-fpm
volumes:
- app-data:/var/www/html
# --------- #
# Volumes #
# --------- #
volumes:
app-data:

This is obviously a super simplification of my scenario.

 

Anyway, the key issue is that I'm not able to share the `app-data` volume since Bitbucket gives the following error:

"ERROR: for app-pipelines_php-fpm_1  Cannot create container for service php-fpm: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories
ERROR: for php-fpm  Cannot create container for service php-fpm: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories"

 

Thanks for your time :)

 

Eugenio

1 answer

3 votes
Taras Postument July 28, 2020

Hey, I faced the same problem and ended up with the following hack.

Instead of defining the global volume (like your `app-data:`) I used dynamic volumes definition with the `BITBUCKET_CLONE_DIR` dir variable.

This way, when composing is running in the BB Pipeline volume is mounted int `BITBUCKET_CLONE_DIR` and when I run it locally, the standard volume is created.

- &static_volume ${BITBUCKET_CLONE_DIR:-static}:/usr/src/app/static

Then you can use it like

volumes: 
- *static_volume

In all the container definitions.

 


Also, I unsuccessfully tried other workarounds like:

volumes:
static:
driver: local
driver_opts:
type: none
o: bind
device: ${BITBUCKET_CLONE_DIR:-/tmp}

or

volumes:
static:
driver: local
driver_opts:
type: tmpfs
device: tmpfs

 

While both worked locally, none worked in the Pipeline.

Hope it helps!

Stefano Angaran July 28, 2020

Hey Taras, thanks for posting this. I've never seen that syntax with ampersand and asterix used, could you give me some references to investigate a little bit more? Thank you very much!

Taras Postument July 28, 2020

@Stefano Angaran 
Sure, it is called YAML Anchors and it is a feature of YAML language.

Using them is the only way to avoid terrible repetition int Pipelines definitions.

Here are some references:
1. https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/
2. https://docs.gitlab.com/ee/ci/yaml/#anchors
3. https://medium.com/@kinghuang/docker-compose-anchors-aliases-extensions-a1e4105d70bd

Like # people like this
Tyler Renslow February 16, 2021

Hi Taras, would you mind explaining where you defined the 

- &static_volume ${BITBUCKET_CLONE_DIR:-static}:/usr/src/app/static

part?

 

I understand that the other part is defined under the individual services, but I can't figure out where to put the first part in the docker compose file.

Cheers 

Taras Postument February 16, 2021

Hi Tyler, basically, you can define it in one of two places:
1. Define anchor in the custom fields and reuse for services(https://docs.docker.com/compose/compose-file/compose-file-v3/#extension-fields)

like:

x-custom:
   volumes: 
- &static_volume ${BITBUCKET_CLONE_DIR:-static}:/usr/src/app/static

service1
:

   ...
   volumes: 

   - *static_volume


service2:
   ...
   volumes: 

   - *static_volume

 

2. or you can define the anchor in the first service and reuse it all the subsequent services:

service1:
   ...
   volumes: 

   - &static_volume ${BITBUCKET_CLONE_DIR:-static}:/usr/src/app/static


service2:
   ...
   volumes: 

   - *static_volume
Like Tyler Renslow likes this
Tyler Renslow February 16, 2021

Thanks for taking the time Taras. This is super helpful!

Marc NALPAS July 8, 2021

hey, I get the same error, but I don't have any volumes in my docker files.

do you have any idea where this error come from?

 

Like Aliaksandr Parfianiuk likes this
Aliaksandr Parfianiuk
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 19, 2021

If you are using docker-compose in pipeline, try to install it via such command:

- curl -L --fail "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o ./docker-compose && chmod +x ./docker-compose
- mv ./docker-compose /usr/bin
Like Marc NALPAS likes this
Marc NALPAS August 24, 2021

thanks for the answer the problem was that the docker file was generated by visual studio and it adds a docker-compose.override.yml with a shard volume for ssl configuration.

The problem was solved after removing it.

Like Aliaksandr Parfianiuk likes this

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events