Bamboo Elastic Agents and Docker - Starting dockerd on instance launch.

Question

Hey there,

Our team has recently been stuck on an annoying issue with Bamboo Elastic Agents.

When the first build plan runs on a newly launched instances, once it reaches a Docker task, it stops. The build doesn't fail, or stop, or timeout but it halts at the Docker task and generates no more logs. Once you manually stop and restart that build it runs with no issues.

So it seemed to me the issue was something to do with the first time a docker command is run on a new Elastic Instance.

Now, I was able to replicate this via terminal on an Elastic Instance. Running `docker run hello-world` for instance would just do nothing, not even exit.

I tried this with another terminal window open to view the docker logs. Before running anything they are completely empty, but when you run a docker command such as hello-world it does the whole Starting Docker Application Container Engine and finishes up withe API listening on [::]:2375.

This is kinda good, but then it doesn't actually run the container we asked it to. It just does nothing.

I've tried a number of work arounds including:

* Running `dockerd` on the Elastic Image instance start up script. Except this just resulted in the Agent never spinning up on the instance (and also stopped my private key for authenticating.
* Running variations on the above (nohup, &)
* Running `dockerd` with sudo - this resulted `sudo: no tty present and no askpass program specified`
* Adding a script to the start of my build plan to run `dockerd`. This gave a permission error when opening dockers `config.json`.
* Attempting to start docker with `service docker start`, which gave a permission error. 
* Attempting to start docker with `systemctl start docker.service`, which also gave a permission error.

So, has any one encountered this and overcome it with even a hacky solution? Or will we be doomed to manually starting and stopping the first build for every new elastic instance?

If it helps we are using the Ubuntu stock image for our Elastic Image.

Thanks in advance!

Dylan

Jeremy Owen · Answer

Hey folks,

It's a known issue with EBS-only instances and we're tracking it here:

* BAM-18517: Docker does not start successfully on Elastic Agents using EBS-only instances

The workaround is to add `service docker restart` to your instance start-up script at Bamboo Administration >> Image Configurations >> Edit

Hope this helps :)

Sander Brienen [Avisi] · Answer

We have exactly the same behaviour and also no solution yet.