Different DinD settings in different steps

markus_torstensson0 July 27, 2023

I have a pipeline with three steps. Two of the steps needs a 5GB DinD, the last one a 1GB DinD. I tried something like

```

services:
  dind-1gb:
    type: docker
    image: docker:dind
    memory: 1024
  dind-5gb:
    type: docker
    image: docker:dind
    memory: 5120
```

but that only seems to work for "self-hosted". This is giving me a headache. The first two steps will not work with less memory for the DinD, and the last step to not work without more for the step container.
Sure - I could define "services" for the containers I run - for now. But eventually I will have them brought up by gradle instead. And more than that, caching does not seem to happen for service containers, which makes the build dead slow. But if I could have them cached in some way I could be ok with it for a time I guess.

I could include DinD in my agent images ... But I really really do not want to go down that hole unless I'm forced. The ability to declaratively ask for DinD capability is something of a killer feature for me, and I really want to use it.
I'm not sure it's even possible at that. I think it needs to be run in privileged mode

What to do?

1 answer

1 vote
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 28, 2023

Hi Markus,

While it is not possible to use a custom Docker image for the docker service in Pipelines that run in our own infrastructure, it is possible to define two Docker services with different memory limits.

You can see an example in the following blog post:

Is this something that works for you?

Kind regards,
Theodora

Markus Torstensson July 28, 2023

While I cannot reach the page you posted, it sounds like exactly what I need.

I will review the documentation again.

Thanks a million :)

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 28, 2023

You are very welcome!

It's strange that you cannot reach the page, it is not restricted as far as I can see.

I am posting the example from that page here, please feel free to reach out if you have any questions!

definitions:
services:
docker:
memory: 512
docker-with-more-memory:
memory: 2048
type: docker
docker-with-large-memory:
memory: 5120
type: docker

pipelines:
custom:
pipeline1:
- step:
services: [docker]
script:
- echo "Docker service with 512 MB memory"
pipeline2:
- step:
services: [docker-with-more-memory]
script:
- echo "Docker service with 2048 MB memory"
pipeline3:
- step:
services: [docker-with-large-memory]
size: 2x
script:
- echo "Docker service with 5120 MB memory"
Like # people like this
markus_torstensson0 July 28, 2023

I tried on my phone. Clicked on the link that was part of the email. Either gmail did something strange, or my phone did. It is reproducible at least. I can view it if I click the link while I'm in the forum.

It seems to work! It's interesting to note that is I name the service "docker5G" I get the error

Configuration error

The name of at least one of the service definitions in your bitbucket-pipelines.yml is invalid. Please make sure that the service names contain only lower case alphanumeric characters and hyphens and that the hyphens are not at the start or the end of the name

But I can live with the name "docker-with-large-memory"

Once again, thanks :)

 

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 31, 2023

Hi Markus,

It's good to hear that it works!

Renaming the service to docker5g should also be ok, I believe it's the capital G in the name that causes this error.

You are once again welcome, and please feel free to reach out if you need anything else!

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
AUG Leaders

Atlassian Community Events