AWS "The provided image is invalid." when using an image built from BitBucket Pipelines

Garth Wood July 13, 2023

Hi

I am trying to run a NextJS application in an AWS lambda using a docker image built from BitBucket Pipelines.

I have setup a standard lambda with 1024 memory and default storage and if I build the docker image from my local machine and push it to AWS ECR it works fine, but when I build it from BitBucket Pipelines the lambda dashboard says "Failed to update the function <my function name>: The provided image is invalid." with no additional detail.

My first assumption is BitBucket Pipelines is doing something which is causing it to fail but I'm not sure where to start looking. I also deploy other lambdas from BitBucket Pipelines (for Go etc) and it works fine.

I have tried with my app and the example from the lambda web adaptor repo and get the same result.
https://github.com/awslabs/aws-lambda-web-adapter/tree/main/examples/nextjs

As this is only an issue when building from Pipelines I imagine either Pipelines is doing something different or there is something that I am missing.

Where could I start in trying to find the issue?

2 answers

0 votes
Garth Wood July 15, 2023

After some digging it appears that the issue lies when copying the lambda adapter binary from the remote image.

COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0 /lambda-adapter /opt/extensions/lambda-adapter

If that binary exists locally and is copied across in the Dockerfile using "COPY ./lambda-adapter /opt/extensions/lambda-adapter" everything works. And as mentioned in my original post this only happens when building from Pipelines.

Could it be a docker version issue or something in Pipelines?

Garth Wood July 15, 2023

And after that discovery it appears all that was needed was to copy using a chown. But why this is only needed in Pipelines I'm not sure.

COPY --chown=root:root --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0 /lambda-adapter /opt/extensions/lambda-adapter
0 votes
Igor Stoyanov
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 13, 2023

@Garth Wood  hi. Thanks for question.

Some of the example of how to build (and deploy) docker image inside a pipeline:

step:
script:
# build the image
- docker build -t $IMAGE_NAME:$IMAGE_VERSION -t $IMAGE_NAME:latest .
- docker save --output my-docker-image.tar.gz $IMAGE_NAME
services:
- docker
artifacts:
- my-docker-image.tar.gz
step:
script:
# load previously saved image that will available as an artifact
docker load --input docker-image.tar.gz
# use the pipe to push the image to AWS ECR
- pipe: atlassian/aws-ecr-push-image:2.0.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID_1
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY_1
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION_1
IMAGE_NAME: $IMAGE_NAME
services:
- docker



Also you could try to use Bitbucket Pipe for AWS Lambda deploy:

script:
  - pipe: atlassian/aws-lambda-deploy:1.8.2
    variables:
      FUNCTION_NAME: 'my-lambda-function'
      COMMAND: 'update'
      IMAGE_URI: '${AWS_ACCOUNT}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/my-lambda-ecr-repository:image-tag'

 Regards, Igor

Garth Wood July 13, 2023

Hi Igor

Thanks for the quick response.

I'm not having issue any building the docker or pushing it to ECR, it's only when I try to select it in the lambda dashboard that I get the error.

Igor Stoyanov
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 13, 2023

Yes, and i suggested you to update your lambda with aws-lambda-deploy pipe using image you pushed to ecr.
I don't know the reason why do you have this issue.
I provide you a workaround of how you could update your lambda function. 

script:
  - pipe: atlassian/aws-lambda-deploy:1.8.2
    variables:
      FUNCTION_NAME: 'my-lambda-function'
      COMMAND: 'update'
      IMAGE_URI: 'your ecr docker image uri'

Regards, Igor

Garth Wood July 14, 2023

The root of the issue is that AWS lambda cannot use the docker image when built from Pipelines. It doesn't matter if Pipelines tries to update the function or I try to select the image manually in the lambda dashboard.

It seems as though there is something in the Pipelines docker build process which is affecting it but I'm not sure what. Possibly some permission issue.

Igor Stoyanov
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 17, 2023

@Garth Wood . Working with Lambda container images :

To create a Lambda function from a container image, build your image locally and upload it to an Amazon Elastic Container Registry (Amazon ECR) repository. Then, specify the repository URI when you create the function.

Regards, Igor

Garth Wood July 17, 2023

Hi Igor

I managed to figure out what it was. I posted some followup comments to this thread regarding having to use chown in the Dockerfile during the copy. This is only needed in Pipelines for some reason.

Is this something you or the dev team can investigate?

Igor Stoyanov
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 17, 2023

@Garth Wood . I mentioned you the process above (with pushing to ECR), because i can't see that aws-lambda-deploy pipe supports local Docker images, only ECR URI.

Regards, Igor

Igor Stoyanov
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 17, 2023

@Garth Wood . You provided in your example:

    - pipe: atlassian/aws-ecr-push-image:2.0.0
variables:
...
IMAGE_NAME: $IMAGE_NAME 

but IMAGE_NAME is not presented in list of variables that this pipe supports.

Garth Wood July 17, 2023

I am using ECR. Please read my followup comments. There is no issue in pushing the image or with the pipes.

The issue happens in the Dockerfile when performing a COPY from a remote image, i.e.

COPY -from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0 /lambda-adapter /opt/extensions/lambda-adapter

In Pipelines there is some ownership issue or something else I'm not aware of because it only works if I use chown, i.e

COPY --chown=root:root --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0 /lambda-adapter /opt/extensions/lambda-adapter

 And having to use chown is only needed in Pipelines.

Edit: just to clarify, when I say works I mean it works in AWS lambda. Without chown it still builds correctly but because of the file ownership issues and because that file being copied is an extension AWS lambda fails.

Like Igor Stoyanov likes this
Igor Stoyanov
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 17, 2023

@Garth Wood . Let's summarize.
Your initial issue: Failed to update the function <my function name>: The provided image is invalid.

Your example:

    - pipe: atlassian/aws-ecr-push-image:2.0.0
variables:
...
IMAGE_NAME: $IMAGE_NAME 

What i told you:
IMAGE_NAME is not presented in list of variables that this pipe supports.

What i suggest you:
update your lambda with aws-lambda-deploy pipe using image you pushed to ecr.

script:
  - pipe: atlassian/aws-lambda-deploy:1.8.2
    variables:
      FUNCTION_NAME: 'my-lambda-function'
      COMMAND: 'update'
      IMAGE_URI: 'public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0'

 Is this works?

Answering your comments:
The issue happens in the Dockerfile when performing a COPY from a remote image, i.e.
Why are you trying to copy your ECR Docker image? You could use the URI without this.

Regards, Igor

Garth Wood July 17, 2023

That error message is from the AWS lambda dashboard not from Pipelines. It happens after I have already successfully uploaded the image to ECR. As mentioned previously there is no issue when uploading to ECR, whether I use a docker push or a pipe. There was only an issue when AWS lambda tried to use the image.

I'm not copying my image. The COPY is pulling in an extension from the lambda web adapter image.

https://github.com/awslabs/aws-lambda-web-adapter/tree/main

Igor Stoyanov
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 17, 2023

@Garth Wood . Sorry for misunderstanding. I thought you have problems with a pipe usage, not with pipelines.
Let's wait if someone from pipelines team will provide a response here.

Regards, Igor

Like Garth Wood likes this

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
AUG Leaders

Atlassian Community Events