nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Pipe failing to pull from S3 despite provided credentials having full admin permissions

Robert Williams_yy1hgg

When attempting to use https://bitbucket.org/atlassian/aws-cloudformation-deploy/src/0.7.4/ in order to deploy a Cloudformation stack, it consistently fails stating an S3 authentication issue, despite the fact that the AWS credentials provided to use in the repository variables have full admin rights.

I have attempted this with:

credentials being passively present in the environment
credentials explicitly provided to the pip variables
https://s3.amazonaws... path to the template
https://<BUCKET_NAME>.s3-<REGION>.amazonaws.com/... path to the template
s3://... path to the template

Any file path other than the first example fails with a different message, citing errors in the path format provided.

I have also checked the S3 bucket's ACL to confirm that the AWS IAM user relevant to this task is authorised to perform actions on the bucket (it is)

I have also confirmed that the URL in use within the pipeline is the same as the URL supplied within the AWS Console for that file (it is)

I am also uploading files to the same bucket (in fact in this particular case the same files I'm then attempting to read) in a previous step in the pipeline - same bucket, same IAM user, same files.

Error message (edited to remove precise bucket information):

<span>Status: Downloaded newer image for bitbucketpipelines/aws-cloudformation-deploy:0.7.4</span>

<span>INFO: Using stack template from <a href="https://rob-playground.s3-eu-west-1.amazonaws.com/templates/root_stack/add_registry_support.yaml" target="_blank" rel="noopener nofollow noreferrer">https://<BUCKET_NAME>.s3-<REGION>.amazonaws.com/.</a>.. for deploy.</span><span>INFO: Found credentials in environment variables.</span>

<span>INFO: Validating the template.</span>

<span>&#10006; Template validation failed.</span>

<span>For more information check <a href="http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html" target="_blank" rel="noopener nofollow noreferrer">http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html</a></span><span>An error occurred (ValidationError) when calling the ValidateTemplate operation: S3 error: Access Denied</span>

Is there any guidance any would be able to provide for this issue?

Accepted answers

Halyna Berezovska

Hello @Robert Williams !

I see possible root causes here and can propose some troubleshooting to find the issue:

sometimes aws makes user inactive (perhaps when it has actually been inactive for some time), check that this s3 user is active (we have similar errors due to this).
if termplate is url to s3 it shopuld have appropriate format, check you have the right path to your s3 bucket

https://s3.amazonaws.com/{S3_BUCKET}/<path_to_template>/<template>.json

s3 specifically may have complex policy for each bucket. Check bucket's policy

Regards, Galyna

All comments

Halyna Berezovska

Hello @Robert Williams !

I see possible root causes here and can propose some troubleshooting to find the issue:

sometimes aws makes user inactive (perhaps when it has actually been inactive for some time), check that this s3 user is active (we have similar errors due to this).
if termplate is url to s3 it shopuld have appropriate format, check you have the right path to your s3 bucket

https://s3.amazonaws.com/{S3_BUCKET}/<path_to_template>/<template>.json

s3 specifically may have complex policy for each bucket. Check bucket's policy

Regards, Galyna

Halyna Berezovska

I am also uploading files to the same bucket (in fact in this particular case the same files I'm then attempting to read) in a previous step in the pipeline - same bucket, same IAM user, same files.

About the fact that you are uploading same files before: check specifically that file after uploading what permissions it has and do you able to read it or not

Robert Williams_yy1hgg

Hi @Halyna Berezovska, thanks for getting back to me

Unfortunately these initial issues are things that I have already ruled out (responses to your points from my original post):

sometimes aws makes user inactive (perhaps when it has actually been inactive for some time), check that this s3 user is active (we have similar errors due to this).
- I am also uploading files to the same bucket (in fact in this particular case the same files I'm then attempting to read) in a previous step in the pipeline - same bucket, same IAM user, same files.
- if termplate is url to s3 it shopuld have appropriate format, check you have the right path to your s3 bucket
https://s3.amazonaws.com/{S3_BUCKET}/<path_to_template>/<template>.json
- I have also confirmed that the URL in use within the pipeline is the same as the URL supplied within the AWS Console for that file (it is)
s3 specifically may have complex policy for each bucket. Check bucket's policy
- I have also checked the S3 bucket's ACL to confirm that the AWS IAM user relevant to this task is authorised to perform actions on the bucket (it is)

However, I think you may well have solved it with your additional message - the uploaded files (bizarrely) don't seem to have been attributed with access permissions for the host account.

I will attempt to upload these files using the relevant S3 Deploy pipe to see if that resolves the issue, but I can't help but feel as though there's a bug at play somewhere along the lines, as uploading files using AWS CLI from my local machine does not result in this side-effect.

Halyna Berezovska

@Robert Williams yes, perhaps playing with permissions will result in something.

The goal of the pipe is to just use the template provided.

So if this is a bug, I think this is more related either to the pipeline, not pipe or even to AWS CLI (perhaps, they have some edge cases that we need to discover).

Looking forward to get updates from you, and we can define the root cause further to see which side it is on.

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Unanswered
Groups
Help

Find more posts tagged with

cloud

bitbucket-pipelines

aws