CI Pipeline hangs when using nx with max-parallel higher than 1

[Johannes Fahrenkrug]

Hi,

I'm using Bitbucket Cloud to run the tests of an nx project.

When `max-parallel` is set to a value higher than 1, the pipeline times out. It seems as if Bitbucket doesn't notice that the command has finished. When I set max-parallel to 1, it works.

Here's my yml file:

image: node:16

options:

  max-time: 20 # configure default 20 minute timeout

pipelines:

  pull-requests:

    '**':

     - step:

        name: 'Lint, test, and build'

        caches:

         - node

        script:

         - npm ci

         - npx nx workspace-lint

         - npx nx format:check

         - npx nx affected --target=lint --base=origin/master --parallel --max-parallel=3

         - npx nx affected --target=test --base=origin/master --parallel --max-parallel=1 --ci --code-coverage

         - npx nx affected --target=build --base=origin/master --head=HEAD --parallel --max-parallel=3

When changing `max-parallel` in the second to last line to "3", the pipeline won't finish. When running it locally on my machine, it finishes just fine. Any thoughts?

[Theodora Boudale]

Hi Johannes,

One possibility is that step may need more than 20 minutes to finish.

When `max-parallel` is set to a value higher than 1, is there any indication in the command's output that the command has finished doing what it is doing?

Is there an option to add verbose output to the command (that may show additional output about what is happening)?

Kind regards,
Theodora

[Melvin Williams]

@Johannes Fahrenkrug  did you find a solution for the problem. I'm facing the same behaviour.

[Johannes Fahrenkrug]

@Melvin Williams The only "solution" we found was to set `--max-parallel=1` when running our tests on the BitBucket pipeline. 

[Melvin Williams]

Thanks for the reply. Which Nx version are you using

[Johannes Fahrenkrug]

16.6.0

[aalbericio]

Same problem here with Nx 17.3.0. 

I managed to make it work using --max-parallel=1 but the rest of configurations (2, 4, etc) lead to unexpected results: sometimes it works and sometimes it hangs (always on the first package to be tested).

Is there any news on this issue? Does it have something to do with the runners conf?

@Johannes Fahrenkrug Have you found any other "workaround"? 

Thanks!

[Theodora Boudale]

Hi everyone,

If the command hangs because it needs additional resources (memory or CPU cores), you can look into increasing the step's size:

• https://support.atlassian.com/bitbucket-cloud/docs/step-options/#Size

As the doc mentions, (a) this is applicable to steps running on the Bitbucket Cloud infrastructure or with a Linux Docker self-hosted runner, and (b) bigger size steps consume more build minutes.

If this is not a resources-related issue, I suggest creating an issue at nx's issue tracker here:

• https://github.com/nrwl/nx/issues

Kind regards,
Theodora