Hi
I am trying to run a NextJS application in an AWS lambda using a docker image built from BitBucket Pipelines.
I have setup a standard lambda with 1024 memory and default storage and if I build the docker image from my local machine and push it to AWS ECR it works fine, but when I build it from BitBucket Pipelines the lambda dashboard says "Failed to update the function <my function name>: The provided image is invalid." with no additional detail.
My first assumption is BitBucket Pipelines is doing something which is causing it to fail but I'm not sure where to start looking. I also deploy other lambdas from BitBucket Pipelines (for Go etc) and it works fine.
I have tried with my app and the example from the lambda web adaptor repo and get the same result.
https://github.com/awslabs/aws-lambda-web-adapter/tree/main/examples/nextjs
As this is only an issue when building from Pipelines I imagine either Pipelines is doing something different or there is something that I am missing.
Where could I start in trying to find the issue?
After some digging it appears that the issue lies when copying the lambda adapter binary from the remote image.
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0 /lambda-adapter /opt/extensions/lambda-adapter
If that binary exists locally and is copied across in the Dockerfile using "COPY ./lambda-adapter /opt/extensions/lambda-adapter" everything works. And as mentioned in my original post this only happens when building from Pipelines.
Could it be a docker version issue or something in Pipelines?
And after that discovery it appears all that was needed was to copy using a chown. But why this is only needed in Pipelines I'm not sure.
COPY --chown=root:root --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0 /lambda-adapter /opt/extensions/lambda-adapter
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Garth Wood hi. Thanks for question.
Some of the example of how to build (and deploy) docker image inside a pipeline:
step:
script:
# build the image
- docker build -t $IMAGE_NAME:$IMAGE_VERSION -t $IMAGE_NAME:latest .
- docker save --output my-docker-image.tar.gz $IMAGE_NAME
services:
- docker
artifacts:
- my-docker-image.tar.gz
step:
script:
# load previously saved image that will available as an artifact
docker load --input docker-image.tar.gz
# use the pipe to push the image to AWS ECR
- pipe: atlassian/aws-ecr-push-image:2.0.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID_1
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY_1
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION_1
IMAGE_NAME: $IMAGE_NAME
services:
- docker
Also you could try to use Bitbucket Pipe for AWS Lambda deploy:
script: - pipe: atlassian/aws-lambda-deploy:1.8.2 variables: FUNCTION_NAME: 'my-lambda-function' COMMAND: 'update' IMAGE_URI: '${AWS_ACCOUNT}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/my-lambda-ecr-repository:image-tag'
Regards, Igor
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Igor
Thanks for the quick response.
I'm not having issue any building the docker or pushing it to ECR, it's only when I try to select it in the lambda dashboard that I get the error.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Yes, and i suggested you to update your lambda with aws-lambda-deploy pipe using image you pushed to ecr.
I don't know the reason why do you have this issue.
I provide you a workaround of how you could update your lambda function.
script: - pipe: atlassian/aws-lambda-deploy:1.8.2 variables: FUNCTION_NAME: 'my-lambda-function' COMMAND: 'update' IMAGE_URI: 'your ecr docker image uri'
Regards, Igor
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
The root of the issue is that AWS lambda cannot use the docker image when built from Pipelines. It doesn't matter if Pipelines tries to update the function or I try to select the image manually in the lambda dashboard.
It seems as though there is something in the Pipelines docker build process which is affecting it but I'm not sure what. Possibly some permission issue.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Garth Wood . Working with Lambda container images :
To create a Lambda function from a container image, build your image locally and upload it to an Amazon Elastic Container Registry (Amazon ECR) repository. Then, specify the repository URI when you create the function.
Regards, Igor
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Igor
I managed to figure out what it was. I posted some followup comments to this thread regarding having to use chown in the Dockerfile during the copy. This is only needed in Pipelines for some reason.
Is this something you or the dev team can investigate?
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Garth Wood . I mentioned you the process above (with pushing to ECR), because i can't see that aws-lambda-deploy pipe supports local Docker images, only ECR URI.
Regards, Igor
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Garth Wood . You provided in your example:
- pipe: atlassian/aws-ecr-push-image:2.0.0
variables:
...
IMAGE_NAME: $IMAGE_NAME
but IMAGE_NAME is not presented in list of variables that this pipe supports.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I am using ECR. Please read my followup comments. There is no issue in pushing the image or with the pipes.
The issue happens in the Dockerfile when performing a COPY from a remote image, i.e.
COPY -from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0 /lambda-adapter /opt/extensions/lambda-adapter
In Pipelines there is some ownership issue or something else I'm not aware of because it only works if I use chown, i.e
COPY --chown=root:root --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0 /lambda-adapter /opt/extensions/lambda-adapter
And having to use chown is only needed in Pipelines.
Edit: just to clarify, when I say works I mean it works in AWS lambda. Without chown it still builds correctly but because of the file ownership issues and because that file being copied is an extension AWS lambda fails.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Garth Wood . Let's summarize.
Your initial issue: Failed to update the function <my function name>: The provided image is invalid.
Your example:
- pipe: atlassian/aws-ecr-push-image:2.0.0
variables:
...
IMAGE_NAME: $IMAGE_NAME
What i told you:
IMAGE_NAME is not presented in list of variables that this pipe supports.
What i suggest you:
update your lambda with aws-lambda-deploy pipe using image you pushed to ecr.
script: - pipe: atlassian/aws-lambda-deploy:1.8.2 variables: FUNCTION_NAME: 'my-lambda-function' COMMAND: 'update' IMAGE_URI: 'public.ecr.aws/awsguru/aws-lambda-adapter:0.7.0'
Is this works?
Answering your comments:
The issue happens in the Dockerfile when performing a COPY from a remote image, i.e.
Why are you trying to copy your ECR Docker image? You could use the URI without this.
Regards, Igor
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
That error message is from the AWS lambda dashboard not from Pipelines. It happens after I have already successfully uploaded the image to ECR. As mentioned previously there is no issue when uploading to ECR, whether I use a docker push or a pipe. There was only an issue when AWS lambda tried to use the image.
I'm not copying my image. The COPY is pulling in an extension from the lambda web adapter image.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
@Garth Wood . Sorry for misunderstanding. I thought you have problems with a pipe usage, not with pipelines.
Let's wait if someone from pipelines team will provide a response here.
Regards, Igor
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.