Issues with aws-cloudfront-invalidate:0.6.0 NoSuchDistribution error

[Fernando Muller]

Hello,

The error below is occurring when trying to perform the invalidation procedure via Bitbucket pipelines.

Status: Downloaded newer image for bitbucketpipelines/aws-cloudfront-invalidate:0.6.0
INFO: Enabling debug mode.
DEBUG: Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
DEBUG: Changing event name from before-call.apigateway to before-call.api-gateway
DEBUG: Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
DEBUG: Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
DEBUG: Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
DEBUG: Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
DEBUG: Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
DEBUG: Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
DEBUG: Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
DEBUG: Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
DEBUG: Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
DEBUG: IMDS ENDPOINT: http://169.254.169.254/
DEBUG: Looking for credentials via: env
INFO: Found credentials in environment variables.
DEBUG: Loading JSON file: /usr/local/lib/python3.7/site-packages/botocore/data/endpoints.json
DEBUG: Event choose-service-name: calling handler <function handle_service_name_alias at 0x7fc8a773acb0>
DEBUG: Loading JSON file: /usr/local/lib/python3.7/site-packages/botocore/data/cloudfront/2020-05-31/service-2.json
DEBUG: Event creating-client-class.cloudfront: calling handler <function add_generate_presigned_url at 0x7fc8a776b440>
DEBUG: Using partition endpoint for cloudfront, us-east-1: aws-global
DEBUG: Setting cloudfront timeout as (60, 60)
DEBUG: Loading JSON file: /usr/local/lib/python3.7/site-packages/botocore/data/_retry.json
DEBUG: Registering retry handlers for service: cloudfront
INFO: Sending a cloudfront invalidation request
INFO: Sending an invalidation request for the distribution id: E99NMDMBH9B99R
INFO: Distibution URL: https://console.aws.amazon.com/cloudfront/home?region=us-east-1#distribution-settings:E99NMDMBH9B99R
DEBUG: Event before-parameter-build.cloudfront.CreateInvalidation: calling handler <function generate_idempotent_uuid at 0x7fc8a76e7170>
DEBUG: Event before-call.cloudfront.CreateInvalidation: calling handler <function inject_api_version_header_if_needed at 0x7fc8a76e89e0>
DEBUG: Making request for OperationModel(name=CreateInvalidation) with params: {'url_path': '/2020-05-31/distribution/E99NMDMBH9B99R%20/invalidation', 'query_string': {}, 'method': 'POST', 'headers': {'User-Agent': 'Boto3/1.17.12 Python/3.7.11 Linux/5.15.0-1031-aws Botocore/1.20.104'}, 'body': b'<InvalidationBatch xmlns="http://cloudfront.amazonaws.com/doc/2020-05-31/"><Paths><Quantity>1</Quantity><Items><Path>/*</Path></Items></Paths><CallerReference>2023-03-23T22:16:54.253931</CallerReference></InvalidationBatch>', 'url': 'https://cloudfront.amazonaws.com/2020-05-31/distribution/E99NMDMBH9B99R%20/invalidation', 'context': {'client_region': 'aws-global', 'client_config': <botocore.config.Config object at 0x7fc8a6bd9510>, 'has_streaming_input': False, 'auth_type': None}}
DEBUG: Event request-created.cloudfront.CreateInvalidation: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7fc8a6bd95d0>>
DEBUG: Event choose-signer.cloudfront.CreateInvalidation: calling handler <function set_operation_specific_signer at 0x7fc8a76e7050>
DEBUG: Calculating signature using v4 auth.
DEBUG: CanonicalRequest:
POST
/2020-05-31/distribution/E99NMDMBH9B99R%2520/invalidation
host:cloudfront.amazonaws.com
x-amz-date:20230323T221654Z
host;x-amz-date
bf455fa45decfed780fce4443bd2f386844d5530038e290609b08e78feab3344
DEBUG: StringToSign:
AWS4-HMAC-SHA256
20230323T221654Z
20230323/us-east-1/cloudfront/aws4_request
19acdf4127dce3475392a38eaa7314c2d2361d43c5403f38c3f9f79016fe8d68
DEBUG: Signature:
51d199c4e5fc538f8fd8a6ab604874289191d4a61a8212a3c0ff424b2fa61c9f
DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://cloudfront.amazonaws.com/2020-05-31/distribution/E99NMDMBH9B99R%20/invalidation, headers={'User-Agent': b'Boto3/1.17.12 Python/3.7.11 Linux/5.15.0-1031-aws Botocore/1.20.104', 'X-Amz-Date': b'20230323T221654Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIA33LAPKHHUTXF2IIH/20230323/us-east-1/cloudfront/aws4_request, SignedHeaders=host;x-amz-date, Signature=51d199c4e5fc538f8fd8a6ab604874289191d4a61a8212a3c0ff424b2fa61c9f', 'Content-Length': '223'}>
DEBUG: Certificate path: /usr/local/lib/python3.7/site-packages/certifi/cacert.pem
DEBUG: Starting new HTTPS connection (1): cloudfront.amazonaws.com:443
DEBUG: https://cloudfront.amazonaws.com:443 "POST /2020-05-31/distribution/E99NMDMBH9B99R%20/invalidation HTTP/1.1" 404 294
DEBUG: Response headers: {'x-amzn-RequestId': '6143e103-b373-4134-bde2-5a2f7153c486', 'Content-Type': 'text/xml', 'Content-Length': '294', 'Date': 'Thu, 23 Mar 2023 22:16:53 GMT'}

The Distribution exists and the CLOUDFRONT_DISTRIBUTION variable containing the Distribution's id is loaded correctly, as you can see from the DEBUG details in the pipeline.

When changing the docker image in the pipe
from:
atlassian/aws-cloudfront-invalidate:0.6.0
to:
atlassian/aws-cloudfront-invalidate:0.5.0

Pipeline works as expected with no errors. According to DEBUG output:

Status: Downloaded newer image for bitbucketpipelines/aws-cloudfront-invalidate:0.5.0
INFO: Enabling debug mode.
DEBUG: Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
DEBUG: Changing event name from before-call.apigateway to before-call.api-gateway
DEBUG: Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
DEBUG: Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
DEBUG: Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
DEBUG: Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
DEBUG: Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
DEBUG: Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
DEBUG: Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
DEBUG: Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
DEBUG: Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
DEBUG: IMDS ENDPOINT: http://169.254.169.254/
DEBUG: Looking for credentials via: env
INFO: Found credentials in environment variables.
DEBUG: Loading JSON file: /usr/local/lib/python3.7/site-packages/botocore/data/endpoints.json
DEBUG: Event choose-service-name: calling handler <function handle_service_name_alias at 0x7fdd1eb3fdd0>
DEBUG: Loading JSON file: /usr/local/lib/python3.7/site-packages/botocore/data/cloudfront/2020-05-31/service-2.json
DEBUG: Event creating-client-class.cloudfront: calling handler <function add_generate_presigned_url at 0x7fdd1ebf39e0>
DEBUG: Using partition endpoint for cloudfront, us-east-1: aws-global
DEBUG: Setting cloudfront timeout as (60, 60)
DEBUG: Loading JSON file: /usr/local/lib/python3.7/site-packages/botocore/data/_retry.json
DEBUG: Registering retry handlers for service: cloudfront
INFO: Sending a cloudfront invalidation request
INFO: Sending an invalidation request for the distribution id: E99NMDMBH9B99R
INFO: Distibution URL: https://console.aws.amazon.com/cloudfront/home?region=us-east-1#distribution-settings:E99NMDMBH9B99R
DEBUG: Event before-parameter-build.cloudfront.CreateInvalidation: calling handler <function generate_idempotent_uuid at 0x7fdd1eb77320>
DEBUG: Event before-call.cloudfront.CreateInvalidation: calling handler <function inject_api_version_header_if_needed at 0x7fdd1eaf0b90>
DEBUG: Making request for OperationModel(name=CreateInvalidation) with params: {'url_path': '/2020-05-31/distribution/E99NMDMBH9B99R/invalidation', 'query_string': {}, 'method': 'POST', 'headers': {'User-Agent': 'Boto3/1.17.12 Python/3.7.10 Linux/5.15.0-1031-aws Botocore/1.20.29'}, 'body': b'<InvalidationBatch xmlns="http://cloudfront.amazonaws.com/doc/2020-05-31/"><Paths><Quantity>1</Quantity><Items><Path>/*</Path></Items></Paths><CallerReference>2023-03-23T22:34:09.455471</CallerReference></InvalidationBatch>', 'url': 'https://cloudfront.amazonaws.com/2020-05-31/distribution/E99NMDMBH9B99R/invalidation', 'context': {'client_region': 'aws-global', 'client_config': <botocore.config.Config object at 0x7fdd1e1475d0>, 'has_streaming_input': False, 'auth_type': None}}
DEBUG: Event request-created.cloudfront.CreateInvalidation: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7fdd1e147690>>
DEBUG: Event choose-signer.cloudfront.CreateInvalidation: calling handler <function set_operation_specific_signer at 0x7fdd1eb77200>
DEBUG: Calculating signature using v4 auth.
DEBUG: CanonicalRequest:
POST
/2020-05-31/distribution/E99NMDMBH9B99R/invalidation
host:cloudfront.amazonaws.com
x-amz-date:20230323T223409Z
host;x-amz-date
095773807df920724bcbfd720fa0e7d790ec3d8f79a47dd64d11d6cace72695e
DEBUG: StringToSign:
AWS4-HMAC-SHA256
20230323T223409Z
20230323/us-east-1/cloudfront/aws4_request
38ead291147d96196dac3d5698a12c6354e86c4d3c18e5b652ae947a409ace6d
DEBUG: Signature:
680bb259e7ef3e9cfec28daddca4fcf3605984114a0072654de43f84a8db1251
DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://cloudfront.amazonaws.com/2020-05-31/distribution/E99NMDMBH9B99R/invalidation, headers={'User-Agent': b'Boto3/1.17.12 Python/3.7.10 Linux/5.15.0-1031-aws Botocore/1.20.29', 'X-Amz-Date': b'20230323T223409Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIA33LAPKHHUTXF2IIH/20230323/us-east-1/cloudfront/aws4_request, SignedHeaders=host;x-amz-date, Signature=680bb259e7ef3e9cfec28daddca4fcf3605984114a0072654de43f84a8db1251', 'Content-Length': '223'}>
DEBUG: Certificate path: /usr/local/lib/python3.7/site-packages/certifi/cacert.pem
DEBUG: Starting new HTTPS connection (1): cloudfront.amazonaws.com:443
✔ Successfully created a cloudfront invalidation: I5YXWNSPOZW4KIQKQOD6G42T5B
DEBUG: https://cloudfront.amazonaws.com:443 "POST /2020-05-31/distribution/E99NMDMBH9B99R/invalidation HTTP/1.1" 201 385
DEBUG: Response headers: {'x-amzn-RequestId': 'aaa83c35-730a-4496-bc70-f2634ab78000', 'Location': 'https://cloudfront.amazonaws.com/2020-05-31/distribution/E99NMDMBH9B99R/invalidation/I5YXWNSPOZW4KIQKQOD6G42T5B', 'Content-Type': 'text/xml', 'Content-Length': '385', 'Date': 'Thu, 23 Mar 2023 22:34:09 GMT'}
DEBUG: Response body:
b'<?xml version="1.0"?>\n<Invalidation xmlns="http://cloudfront.amazonaws.com/doc/2020-05-31/"><Id>I5YXWNSPOZW4KIQKQOD6G42T5B</Id><Status>InProgress</Status><CreateTime>2023-03-23T22:34:09.565Z</CreateTime><InvalidationBatch><Paths><Quantity>1</Quantity><Items><Path>/*</Path></Items></Paths><CallerReference>2023-03-23T22:34:09.455471</CallerReference></InvalidationBatch></Invalidation>'
DEBUG: Event needs-retry.cloudfront.CreateInvalidation: calling handler <botocore.retryhandler.RetryHandler object at 0x7fdd1e147b10>
DEBUG: No retry needed.

How can I solve the problems when using the image in version 0.6.0?

[Igor Stoyanov]

hi @Fernando Muller . I think the reason for error is that you have a whitespace (ASCII character space %20) at the end of your DistributionID variable name. Compare the logs with links in pipe versions:
===
0.6.0: E99NMDMBH9B99R%20

DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://cloudfront.amazonaws.com/2020-05-31/distribution/E99NMDMBH9B99R%20/invalidation

====
0.5.0: E99NMDMBH9B99R

DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://cloudfront.amazonaws.com/2020-05-31/distribution/E99NMDMBH9B99R/invalidation

===

The reason for different links is that started from 0.6.0 version there is not the same validation for the repository variables (we updated toolkit).

Please, update your DistributionID (remove whitespace), and notify us, if this solved your issue.

Regards, Igor

[Oleksandr Kyrdan]

Hi @Fernando Muller 

Thank you for your question!

The new version of the pipe is released 0.7.0:

script:
  - pipe: atlassian/aws-cloudfront-invalidate:0.7.0
    variables:
      DISTRIBUTION_ID: '123xyz'

with the next changes: 

• Bump version of the base Docker image to python:3.10-slim.
• Bump version of boto3 to version 1.26.* and other dependencies.
• Bump version of the bitbucket-pipe-release.

It would be nice if you could try the new version and share your feedback about your case with the community!

Best regards,
Oleksandr Kyrdan

[Fernando Muller]

Hello,

I tried to use release 0.7.0, but I was unsuccessful, problems occurred.

First error was about the user permission used in the invalidation (Cloudfront: Createinvalidation), although the user has the permissions correctly, the error below occurred:

Status: Downloaded Newer Image for BitbucketPipelines/AWS-Cloudfront-Invalidate: 0.7.0
Info: Found Credentials in Environment Variables.
Info: Sending to Cloudfront Invalidation Request
Info: Sending an invalidation Request for the Distribution ID: E99NMDMBH9B99R
Info: Distribution URL: https://console.ws.amazon.com/cloudfront/home?region=us-east-1#distribution-ottings:e99nmdmbh9b99r
✖ Failed to Create to Cloudfront Invalidation: An error Occred (AccessDenied) When Calling the Createinvalidation Operation: User: AWS: AI :: 994635504079: User/S3-Test-Homolog is Not Authorized to Perform: Cloudfront: CREATEINVALDATION ON RESOURCE: ARN: AWS: Cloudfront :: 994635504079: Distribution/E99nmdmbh9b99r Because No identity-based polycy allows the cloudfront: CREATEINVALIDATION ACTION
Error: Error Creating to Cloudfront Invalidation

Polycy was adjusted, exchanging the specific permission to distribution, for a broader permission, allowing all resources:

From:
"Arn: AWS: Cloudfront :: 994635504079: Distribution/E99NMDMBH9B99R"

TO:
"*"

So the error about permissions did not occur again, but there is a new error, as if the distribution did not exist:

Status: Downloaded Newer Image for BitbucketPipelines/AWS-Cloudfront-Invalidate: 0.7.0
Info: Found Credentials in Environment Variables.
Info: Sending to Cloudfront Invalidation Request
Info: Sending an invalidation Request for the Distribution ID: E99NMDMBH9B99R
Info: Distribution URL: https://console.ws.amazon.com/cloudfront/home?region=us-east-1#distribution-ottings:e99nmdmbh9b99r
Error: Error Creating to Cloudfront Invalidation
✖ FAILED TO CREATE A Cloudfront Invalidation: an error Occred (Nosuchdistribution) When Calling the Createinvalidation Operation: The Specified Distribution Does Not Exist.

I returned to release 0.5.0 and pipeline returned to normal, invalidation occurred as expected.