Waiter Deployment Successful failed: Max attempts exceeded

[Noman Ahmed]

Hi,

we are getting the below error with code-deploy pipeline through Bitbucket pipeline,

How can we resolve this, I;ve checked it successfully run in Code-Deploy AWS but failed at bitbucket, it happens with last 2 deployments with the same scenario.

Waiter DeploymentSuccessful failed: Max attempts exceeded

ERROR: Deployment failed. Fetching deployment information...

bitbucket-pipelines.yml

pipelines:
  branches:
    master:
      - step:
          name: AWS CodeDeploy
          image: python:3.7.2
          script:
            - apt-get update
            - apt-get install -y zip curl wget
            - echo  "'"$UT_AWS_REGION"'"
            - wget -O - https://$UT_S3_BUCKET.s3.amazonaws.com/replace-env-vars.sh | bash -s prod.env .env UT_
            - zip -r s-app.zip .
            - pipe: atlassian/aws-code-deploy:0.5.3
              variables:
                AWS_DEFAULT_REGION: $UT_AWS_REGION
                AWS_ACCESS_KEY_ID: $UT_CODEDEPLOY_ACCESS_KEY_ID
                AWS_SECRET_ACCESS_KEY: $UT_CODEDEPLOY_SECRET_ACCESS_KEY
                APPLICATION_NAME: $AWS_CODEDEPLOY_APP
                S3_BUCKET: $UT_S3_BUCKET
                COMMAND: 'upload'
                ZIP_FILE: 's-app.zip'
                VERSION_LABEL: $BUILD_VERSION
            - pipe: atlassian/aws-code-deploy:0.5.3
              variables:
                AWS_DEFAULT_REGION: $UT_AWS_REGION
                AWS_ACCESS_KEY_ID: $UT_CODEDEPLOY_ACCESS_KEY_ID
                AWS_SECRET_ACCESS_KEY: $UT_CODEDEPLOY_SECRET_ACCESS_KEY
                APPLICATION_NAME: $AWS_CODEDEPLOY_APP
                DEPLOYMENT_GROUP: $AWS_DEPLOYMENT_GROUP
                S3_BUCKET: $UT_S3_BUCKET
                COMMAND: 'deploy'
                WAIT: 'true'
                VERSION_LABEL: $BUILD_VERSION
                IGNORE_APPLICATION_STOP_FAILURES: 'true'
                FILE_EXISTS_BEHAVIOR: 'OVERWRITE'

[Sven]

Hi @Oleksandr Kyrdan ,

Have you found a solution for this? We have had this problem for quite some time too. We are having a bigger number of production servers and deploys take some time, as we are doing either one by one or blue green deployments. A deployment can take more than 30 minutes for example. (As we do not want to affect our customers and need 0% downtime)

As Noman says, as soon as we do half at a time or all at once deploys, it works. The problem is the time of the deploy, it seems Pipelines just has a maximum wait time of 20-30 minutes or something. Is there any workaround for this?

Cheers,

Sven

[Oleksandr Kyrdan]

Hi @Sven @Noman Ahmed 

Have your tried to tune WAIT_INTERVAL ? 

By default, it's setup for 15 sec x 120 attempts = 30 min.

WAIT_INTERVAL: Time to wait between polling for deployment to complete (in seconds). Default: 15.

The aws-code-deploy pipe supports WAIT_INTERVAL and you could tune it according to your case.

To increase waiter's time, provide your value for WAIT_INTERVAL variable:

- step:
    oidc: true
    script:
      - pipe: atlassian/aws-code-deploy:1.1.0
        variables:
          AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
          AWS_OIDC_ROLE_ARN: 'arn:aws:iam::123456789012:role/role_name'
          COMMAND: 'upload'
          APPLICATION_NAME: 'my-application'
          ZIP_FILE: 'myapp.zip'
          WAIT: 'true'
          WAIT_INTERVAL: 60

More details your could find in the aws-code-deploy pipe README.

Cheers,
Oleksandr Kyrdan

[Sven]

Hi Oleksandr,

Thanks a lot for your quick response, it works like a charm! :)

Cheers,

Sven

[Oleksandr Kyrdan]

Hi @Noman Ahmed

Thank you for your question!

We are investigating this issue and notify you.

Cheers,
Oleksandr Kyrdan

[Noman Ahmed]

I found that it happens due to maximum instances like I had 4 instances where deployment should run but it failed after third one. now I had to reduce the instances but it is not a solution. hope it will help you investigate the issues.

P.S i don't want to alter the aws/.config file because the its run through autoscalling group on aws.