Pipeline fails with No such file or directory

[Shawn Nolan]

After ~13min of running, the process finally fails saying there's "No such file or directory". I'm not really sure what I can do from my end since I've followed the examples of how to use `bitbucket-pipeline.yml` and what it should look like. The creds are setup with the environment variables and i can see some folders in the said bucket.

The folder it's failing on looks very long as well, so not sure if its running a loop and finally quits but here's the dir "fatal error: [Errno 2] No such file or directory: '//proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/proc/1/root/dev/fd/10"

Is there something I'm missing? 

Code below:

image: node:10.15.0
pipelines:
  default:
    - step:
        name: Push to S3
        script:
          - pipe: atlassian/aws-s3-deploy:0.2.4
            variables:
              AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
              AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
              AWS_DEFAULT_REGION: 'us-east-1'
              S3_BUCKET: 'some-bucket'
              LOCAL_PATH: '/' 

thx, shawn 

[Daniel Santos · Accepted solution]

Hi @Shawn Nolan

Thank you for sharing both the error and the bitbucket-pipeline.yml file. It helps us to have better clues on what is causing the issue here. 

If your bitbucket-pipeline.yml was exactly as you shared, the problem is that the pipe is looking for files at "/" (root of the OS filesystem) and it is finding some cyclic paths due to folder links.

The LOCAL_PATH should be pointing to a folder where your artifacts are located. Could you please try again using just a specific folder which contains the files you want to deploy to the S3_BUCKET?

Please let me know how it goes.

[Shawn Nolan]

Thx for the response. Had a feeling it was the `LOCAL_PATH`.

So is LOCAL_PATH required? If so, what would the path be for current directory `./`? And how should I know what the local path is from the pipeline? Basically want to push the code in the repo up.

Thx, Shawn

[Shawn Nolan]

So I put './' for the LOCAL_PATH and that worked. thx for your help!

[Shawn Nolan]

It fails on random files, but some get uploaded... Not sure the reason there, but seems promising now.

[Daniel Santos]

So is LOCAL_PATH required?

yes, you need to declare it.

If so, what would the path be for current directory `./`?

You got this one already =]

And how should I know what the local path is from the pipeline? 

The Pipelines will run on BITBUCKET_CLONE_DIR. You can use this variable for this purpose if you need, but "./"  should work as well. 

It fails on random files, but some get uploaded... 

Do you know what these files have in common?
Are they links?
Are they with a special extension?
Do they have special characters on their names or on their parent folders?

[Shawn Nolan]

Do you know what these files have in common? The only thing I can see atm is they're `.js` files (nodejs proj). Sometimes it fails to upload unit test files, core files, and also fails simple files like `webpack.config.js` at one point.

Are they links? Not that I'm aware of.

Are they with a special extension? Just `.js` files.

Do they have special characters on their names or on their parent folders? Other than `test/unit/**/**` nothing really. The webpack config file failed one time, mostly `index.js` files.

[Daniel Santos]

I'll need to contact one of our developers on this one, please hold on. When I have their response I'll share it here.

[Shawn Nolan]

@Daniel Santos any update on this?

[Daniel Santos]

Not yet. I don't have timezone overlap with the pipeline developers and this makes our communication a bit slower than usual. I do know we have someone checking this and an internal ticket was raised. When I have something from them I'll be sharing it here.

[Shawn Nolan]

@Daniel Santos this is awesome, TY!

[Daniel Santos]

Hello @Shawn Nolan

The developer investigating this wants to test an assumption about unchanged files. The pipe will not deploy files without changes and this may be the cause of this issue.

There is an extra argument you could use to test this assumption:

EXTRA_ARGS: '--exact-timestamps'

Do you mind adding this extra argument in your pipe configuration to check if this will affect the results you see?

Also, when you say "It fails on random files" do you see any error or are you just concluding that the pipe is failing because those files are not deployed? 

[Shawn Nolan]

@Daniel Santos the error i get looks like this ```upload: ./webpack.config.js to s3://pulsekit-bitbucket-pipeline/webpack.config.js
Deployment failed.```

I'll also add this extra arg to my pipe config and get back to you.

[Shawn Nolan]

@Daniel Santos ok i used that and got another failed deploy :(

EXTRA_ARGS: '--exact-timestamps'

upload: test/integration/account-user-life-cycle/queries/admin/adminCreateSubscription.graphql to s3://pulsekit-bitbucket-pipeline/test/integration/account-user-life-cycle/queries/admin/adminCreateSubscription.graphql✖ Deployment failed.

[Daniel Santos]

Hi @Shawn Nolan

Given that this thread is becoming big and we don't have a final word on this one, I decided to open a support case to report this problem and keep the investigation running outside of the community, in our support system. This helps us to escalate the case to other teams if necessary.

You should receive an email soon from our support portal regarding your user creation and next updates from our support engineers when someone is assigned to this case. I believe the next updates will come only on Monday.

When this investigation is finished, if you don't mind sharing the findings here we would be very grateful since it will probably help other users facing the same problem.

Thank you!

[Shawn Nolan]

@Daniel Santos awesome. I did receive the email and thank you. Will do post when something comes from it.

Thank you.