Zombie reaping in pipelines

Mikk3lRo May 5, 2018

I have a phpunit test that checks a function which in turn uses `posix_getpgid` to find out if a process exists (given its pid). The tested process is launched (in the background) from the phpunit test, and simply sleeps for a while, then exits.

The test kept failing when run inside a pipeline, though it consistently works in other environments. In the pipeline the function reports that the process is running even after it has finished.

I believe I have tracked down the problem to the pid1 not reaping the process after it exits. Therefore the process does still exist (forever). This seems to be a well-known problem inside docker containers (to which I am fairly new).

I can demonstrate the basic problem with a single-file repo using the default image:

`bitbucket-pipelines.yml`:

pipelines:
  default:
    - step:
        script:
          - ps aux
          - PID=$(ping -c 2 localhost > /dev/null 2>&1 & echo $!)
          - ps -p $PID && sleep 4 && ps -p $PID


This will show what the pid1 is, then launch a background process, show that it is running and that it becomes a zombie after it is done (I would expect the last one to not exist).

Results can be seen here.

For my personal use case I could obviously just detect that I am running inside the pipeline and skip the test, since I know my functions and tests do work in any other environment I need them to... but I feel that this is a problem that should be solved / solvable in the pipeline system.

As far as I understand I have no control over the actual `docker run` command launching the pipeline, so I've only tried to solve it inside the container - by using images that include some sort of init-system that takes care of zombie reaping (s6-overlay, fpco/pid1 and my_init)... but I'm slowly reaching the conclusion that it is impossible, because the pipeline launcher obviously overrides whatever the docker image specifies as `ENTRYPOINT` / `CMD`.

I think the solution could be as simple as adding an `--init` argument to the `docker run` command for a "docker-native" solution - but that is not a change I can make...

Does anyone have a solution for this that will work without bitbucket making any changes? Some ideas?

1 answer

0 votes
Jeroen De Raedt
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 16, 2018

Hey @Mikk3lRo,

I am not entirely sure what the root cause is of this. If you think that you might be able to get it to work by specifying the `docker run` command, you could try this from within your build. As part of the build, just execute `docker run <your-image> <your-params>` and see if this works. 

Alexandru Caracas November 14, 2019

Is it also by now possible to pass the --init option to docker for bitbucket pipelines?

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events