I'm working on a CI nextflow pipeline for some bioinformatics projects. The way it works, is it spawns a docker container for each step of the pipeline to complete the job, and then moves onto the next one. Everything happens in the ./work directory. For some reason, it is being denied the ability to create new containers in Bitbucket cloud's workspace: docker: Error response from daemon: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories. But it seems like I satisfy those requirements: echo $PWD /opt/atlassian/pipelines/agent/build echo $BITBUCKET_CLONE_DIR /opt/atlassian/pipelines/agent/build Can someone help me figure out what's going on?

Hi Michael and welcome to the community! The argument -v in the docker run command is used to mount directories into the container that starts with this command: * https://docs.docker.com/engine/reference/commandline/run/#volume For Pipelines builds that run on Atlassian's infrastructure, we don't allow mapping volumes with a source outside $BITBUCKET_CLONE_DIR. You can find a list of all restricted commands here: * https://support.atlassian.com/bitbucket-cloud/docs/run-docker-commands-in-bitbucket-pipelines/#Running-Docker-commands You will need to check the docker run command that starts these containers or the docker-compose.yml in case you use Compose and only mount directories with a source inside $BITBUCKET_CLONE_DIR. We have a feature request for allowing mounting volumes outside $BITBUCKET_CLONE_DIR: https://jira.atlassian.com/browse/BCLOUD-20986 This restriction does not apply to the self-hosted pipeline runners. If you have a Linux machine where you can run your builds, you could also use a Linux Docker Runner: * https://support.atlassian.com/bitbucket-cloud/docs/runners/ Kind regards, Theodora

nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Trouble running nextflow (nf-core) pipeline - docker authorization denied

Michael Beavitt

I'm working on a CI nextflow pipeline for some bioinformatics projects. The way it works, is it spawns a docker container for each step of the pipeline to complete the job, and then moves onto the next one. Everything happens in the ./work directory.

For some reason, it is being denied the ability to create new containers in Bitbucket cloud's workspace:

<span>docker: Error response from daemon: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories.</span>

But it seems like I satisfy those requirements:

echo $PWD

/opt/atlassian/pipelines/agent/build

echo $BITBUCKET_CLONE_DIR

/opt/atlassian/pipelines/agent/build

Can someone help me figure out what's going on?

Accepted answers

Theodora Boudale

Hi Michael and welcome to the community!

The argument -v in the docker run command is used to mount directories into the container that starts with this command:

https://docs.docker.com/engine/reference/commandline/run/#volume

For Pipelines builds that run on Atlassian's infrastructure, we don't allow mapping volumes with a source outside $BITBUCKET_CLONE_DIR. You can find a list of all restricted commands here:

https://support.atlassian.com/bitbucket-cloud/docs/run-docker-commands-in-bitbucket-pipelines/#Running-Docker-commands

You will need to check the docker run command that starts these containers or the docker-compose.yml in case you use Compose and only mount directories with a source inside $BITBUCKET_CLONE_DIR.

We have a feature request for allowing mounting volumes outside $BITBUCKET_CLONE_DIR: https://jira.atlassian.com/browse/BCLOUD-20986

This restriction does not apply to the self-hosted pipeline runners. If you have a Linux machine where you can run your builds, you could also use a Linux Docker Runner:

https://support.atlassian.com/bitbucket-cloud/docs/runners/

Kind regards,
Theodora

All comments

Theodora Boudale

Hi Michael and welcome to the community!

The argument -v in the docker run command is used to mount directories into the container that starts with this command:

https://docs.docker.com/engine/reference/commandline/run/#volume

For Pipelines builds that run on Atlassian's infrastructure, we don't allow mapping volumes with a source outside $BITBUCKET_CLONE_DIR. You can find a list of all restricted commands here:

https://support.atlassian.com/bitbucket-cloud/docs/run-docker-commands-in-bitbucket-pipelines/#Running-Docker-commands

You will need to check the docker run command that starts these containers or the docker-compose.yml in case you use Compose and only mount directories with a source inside $BITBUCKET_CLONE_DIR.

We have a feature request for allowing mounting volumes outside $BITBUCKET_CLONE_DIR: https://jira.atlassian.com/browse/BCLOUD-20986

This restriction does not apply to the self-hosted pipeline runners. If you have a Linux machine where you can run your builds, you could also use a Linux Docker Runner:

https://support.atlassian.com/bitbucket-cloud/docs/runners/

Kind regards,
Theodora

Michael Beavitt

Thanks Theodora,

Unfortunately it's not possible to use any of the company machines as a runner, but I figured out the issue.

Nextflow was trying to use 'user emulation' in docker, which meant mounting /etc/passwd and some other root directories, and I was able to turn off this feature in the config file.

It runs fine now!

They've also updated nf-core after an issue was raised to remove this default behaviour so this shouldn't be an issue in future..!

Theodora Boudale

Thank you for the update Michael, it's good to hear that the issue is resolved now!

Please feel free to reach out if you ever need anything else!

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

cloud

bitbucket

bitbucket-cloud