Test failing only in pipeline when mocking filesystem

Ezequiel October 2, 2023

We have a few tests that use the Javascript mockFs module to mock out the filesystem in node. 

For the longest time these tests worked fine. Recently however they started failing only when run inside bitbucket-pipelines.

Here's a minimal reproduction https://bitbucket.org/ezequiel-wejugo/mockfs-bb-test/pipelines/results/1

What might be happening?

I imagine it could be to do with some security upgrade or other recent change on Bitbucket's side.

1 answer

1 accepted

0 votes
Answer accepted
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 3, 2023

Hi Eze,

Just to give you some context, Bitbucket Pipelines builds run in Docker containers. For every step of the pipeline, a Docker container starts (the build container) using the image you have specified in your bitbucket-pipelines.yml file. The repo is cloned in that container and then the commands of that step's script are executed. When the step is finished, the container gets destroyed.

We have a guide for debugging Pipelines builds locally. Since you provided a public repo to reproduce, I cloned it and followed this guide.

In Step 2, I used the image node:20-alpine in the Dockerfile (the same one as your repo's). In Step 3, I used --entrypoint=/bin/sh in the command that starts the container (instead of --entrypoint=/bin/bash). The error that you see in the pipeline also occurs when debugging this locally, when I run the npm test command. So, it doesn't look like this is an issue with Bitbucket Pipelines infrastructure.

I would suggest checking the following guide, section "Scenario 1: Builds were successful earlier, but started failing recently" for possible causes and troubleshooting steps:

Please feel free to reach out if you have any questions!

Kind regards,
Theodora

Ezequiel October 4, 2023

Hi Theodora, thanks for the detailed response. 

I run your steps locally and in my case the test passes.

In addition I run the container inside of a GKE kubernetes cluster to eliminate the possibility that running docker on Windows might be changing the result. Here too the test passes.

Assuming npm should detect a version/integrity discrepancy through the lock file, I cannot reconcile your run failing and mine succeeding.

Here's an attempt at custom integrity checking of my node_modules which is stable across installs:

$ find node_modules/ -type f | sort | xargs cat | sha512sum
1ebdcf6f06c9fd01e2213a8da35b75425e94357db18a1cdd7693eb600edf452a59a1d90ebdd72c4a4bfe8b09d5abe00b7df01fee2b8af988867398cdbf23fdad -

 Do you get the same checksum?

Ezequiel October 4, 2023

Actually, I just checked this myself by adding a step to my bitbucket-pipelines.yml file and indeed the node_modules in the pipeline and in my local container are the same, yet the test passes one and fails in the other.

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 4, 2023

Hi Ezequiel,

I got the same checksum as well.

Can you please confirm if you followed the same steps I am sharing below to debug this locally? This is just to confirm that you haven't done anything differently than me.

1. Create a directory for testing

mkdir testdocker

2. Clone the repo in this new directory

cd testdocker
git clone git@bitbucket.org:ezequiel-wejugo/mockfs-bb-test.git localDebugRepo

3. Reset the HEAD of the repository to the same commit as the failed Pipelines build

cd localDebugRepo
git reset --hard 48d74b4

4. Create a Dockerfile outside the clone dir

cd ..
touch my.dockerfile

5. Edit the Dockerfile and add this content:

FROM node@sha256:37750e51d61bef92165b2e29a77da4277ba0777258446b7a9c99511f119db096
WORKDIR /localDebugRepo
COPY ./localDebugRepo /localDebugRepo

I am pinning the Docker image to the same sha that the Pipelines build is using.

6. Build the Docker image

docker build --memory=1g --memory-swap=1g -t myimage/test -f my.dockerfile .

7. Start a container using the image built in the previous step:

docker run -it --memory=4g --memory-swap=4g --memory-swappiness=0 --cpus=4 --entrypoint=/bin/sh myimage/test

8. Inside the container, execute the commands of the repo's yml file

 

If you rerun the last successful build in the private repo, does it still succeed or does it fail?

Is the build image SHA the same or different in the last successful build and the first build that started failing (in your private repo)? You can find the image SHA if you expand the Build setup section in the Pipelines build log.

Kind regards,
Theodora

Like Ezequiel likes this
Ezequiel October 4, 2023

Thanks,

That now reproduces the failing test locally. It's to do with some change in the base image vs the version of the same image tag I have in my local cache.

Like Theodora Boudale likes this
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 6, 2023

You are very welcome, Ezequiel!

If you want, you can pin an image by digest in your yml file to ensure that no changes can be introduced.

You can find an example here:

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
AUG Leaders

Atlassian Community Events