Self hosted runner docker service fails because "docker" is a directory

[Mark G]

We use bitbucket cloud but with self hosted runners. Our jobs use the docker service.

Every week or so our builds start failing with errors like `docker: Permission denied`. After some inspection in the containers, we've found this is because the "docker" command the job is trying to run is actually a directory, not an executable. Following this through the service is mounting the host path `/tmp/<runner ID>/docker` into the containers, and on the host this sometimes becomes a directory instead of an executable file.

Our solution is to stop the runner, delete the directory, and restart the runner. This then recreates `/tmp/<runner ID>/docker` correctly and our builds work fine for about a week or more.

The docker command we were given when creating the runner has a `-v /tmp:/tmp` mount. We assume the host OS does some cleanup of the /tmp directory periodically which is affecting the runner. Do we need to mount the /tmp directory, or can we use a docker volume instead to provide isolation from the host OS?

[Janco Hoekstra]

The name of the hostpath is defined in docker in the config.v2.json file and the  /docker/containers/{id}/hostconfig.json file.
Here you can rename the docker hostpath to another name, not docker. This should solve your problem.

Remember first to stop all docker containers and dockerd before making the changes and then restart them after editing.

I hope this solution fits for you.