Pipelines getting stuck on npm i

[Adam Cooke]

Normally our pipeline takes about 15 minutes to run, currently it seems to be getting stuck at the early stages with npm i.

No recent changes to our package.json or pipeline which should affect this.

Installs and builds fine in vscode.

Its seems to be gettings stuck with a large package (351mb) (https://www.npmjs.com/package/ffprobe-static). Normally this is not a problem.

[Charlie Davies · Accepted solution]

We are seeing the same issues here building inside the node:18-alpine docker image. Codebase built fine yesterday now it is timing out after 2 hours on the npm i step.

[Jakob Wardega]

same for us; ours timeout after 2 hrs

[Adam Cooke]

Reply from support:

Bruno M commented:

Hi there,

Thanks for your report. Our team is aware that yarn/npm install commands are being stuck in Bitbucket Pipelines.

Our engineering team has been notified and is currently looking into the route cause and mitigation efforts.

We appreciate your patience while our team is working to address this issue.

We will be providing regular updates on this ticket.

Best regards,
Bruno M.
Cloud Engineer | Bitbucket Cloud Support

[Charlie Davies]

@Adam Cooke thank you for the update.

[Adam Cooke]

Syahrul commented:

Hello Everyone,

Thanks for reporting the issue to us.

We investigated the issue on our end with help from the Dev team. After a couple of tests, we replicated the issue locally, indicating that the issue was unrelated to the pipeline. Through our testing, we could pinpoint the problem to a single dependency - puppeteer version 21 and lower. In fact, installations solely dependent on puppeteer versions below 22 were observed to freeze during the process.

We have reported this bug to NPM, which you can view and monitor at: https://github.com/npm/cli/issues/7289

The solution is to upgrade your puppeteer version to v22 or greater, as we discovered that the issue only affects versions of puppeteer < v22.

We hope this information is helpful to you. If you have any further questions, please don't hesitate to ask.

-
Mohammad Syahrul
Cloud Engineer | Bitbucket Cloud Support

[Adam Cooke]

Not sure why this has suddenly stopped working, something in the pipeline enviroment must have changed.

[Charlie Davies]

Can confirm that this morning updated puppeteer to latest version and this has fixed the issue - our pipelines are running as they normally would.

Thank you for the swift diagnosis and mitigation of this issue.

[Jakob Wardega]

We were able to fix as well by upgrading a puppeteer version from 20 to 22. 

It's interesting to note that we have another package on puppeteer version 2 that was able to remain unchanged

Thanks to everyone in this thread for raising awareness!

[John Landgrave]

This is, apparently, still an issue. I am very puzzled though because everything was building fine until we merged in some code containing a bunch of new stuff (type-graphql, prisma, apollo, etc.) which is leading me to believe that there is some sort of conflict happening in npm that for some reason only happens in the pipeline environment.

Either way, updating puppeteer seems to have resolved the issue -- even though I never was able to reproduce it locally... 

[Jakob Wardega]

We are having the same issue in some of our pipes, starting today.

Does yours use https://registry.npmjs.org/aws-sdk/-/aws-sdk-2.1577.0.tgz ?

[Adam Cooke]

No, I've spent all day trying to get it to build. Starting to think its a bitbucket issue

[Jakob Wardega]

We have another group in our organization that is facing the same issue. So that makes at least 3 of us. Hopefully they put something out soon. 

[Adam Cooke]

I have removed the package and reverted all changes since the last successfull build and the pipeline still gets stuck

[Prithvi Sathiya]

Im still having this issue for `npm run build` when running with `craco build`: https://community.atlassian.com/t5/Bitbucket-questions/Unable-to-run-npm-run-build-in-docker-using-craco/qaq-p/2704640#M104444

[Lee Alexis Bermejo]

We don't have puppeteer. 
we run a docker build from a Dockerfile which has a multistage the first of which has a yarn install which after logging Completed, gets stuck on th subsequent Fetch Step log

This only happens inside the dockerized (docker build .)  build step.
a separate bitbucket step running the exact same yarn install completes with no problem