Pipeline failing to upload Maven build to Google App Engine

[mvalent23]

Hello!

I'm trying to use pipelines to build a project with Maven and and then deploy using the google-app-engine-deploy pipe. 

Here's my pipelines.yaml file which I created starting with the Maven build template:

pipelines:
    branches:
        dev: #can use {master, integration, dev}: for executing on all 3
            - parallel:
                - step:
                    name: Proxy setup and Build MemberApp
                    script:
                        - echo "Downloading gcloud cli"
                        - curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-437.0.1-linux-x86_64.tar.gz
                        - tar -xf google-cloud-cli-437.0.1-linux-x86_64.tar.gz -C  /tmp
                        - /tmp/google-cloud-sdk/install.sh
                        - source /tmp/google-cloud-sdk/path.bash.inc

                        - echo "Regisering gcloud key"
                        - echo $KEY_FILE | base64 --decode --ignore-garbage > /tmp/gcloud-api-key.json
                        - gcloud auth activate-service-account --key-file /tmp/gcloud-api-key.json
                        - gcloud config set project $GCLOUD_PROJECT

                        - echo "Installing the cloudsql proxy client"
                        - curl -o cloud-sql-proxy https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.4.0/cloud-sql-proxy.linux.amd64
                        - chmod +x cloud-sql-proxy
                        - ./cloud-sql-proxy $CLOUDSQL_CONN_NAME > cloudsql.log 2>&1 &

                        - mvn -B clean install package --file pom.xml

                        - pipe: atlassian/google-app-engine-deploy:1.2.0
                          variables:
                            KEY_FILE: $KEY_FILE
                            PROJECT: $GCLOUD_PROJECT
                            DEPLOYABLES: 'src/main/appengine/app.yaml'
                            STOP_PREVIOUS_VERSION: 'true'

                    after-script:
                        # Collect checkstyle results, if any, and convert to Bitbucket Code Insights.
                        - pipe: atlassian/checkstyle-report:0.3.0
                    artifacts:
                        - cloudsql.log
                - step:
                    name: Security Scan
                    script:
                        # Run a security scan for sensitive data.
                        # See more security tools at https://bitbucket.org/product/features/pipelines/integrations?&category=security
                        - pipe: atlassian/git-secrets-scan:0.5.1

The testing phase uses cloudsql proxy to test against a staged database, so the bulk of that is configuring the cloud cli and sql proxy.

The build results look fine as far as I can tell:

[INFO] The original artifact has been renamed to /opt/atlassian/pipelines/agent/build/target/MemberApp-0.0.1-SNAPSHOT.jar.original

1378

[INFO] ------------------------------------------------------------------------

1379

[INFO] BUILD SUCCESS

1380

[INFO] ------------------------------------------------------------------------

1381

[INFO] Total time: 22.467 s

1382

[INFO] Finished at: 2023-07-08T12:50:37Z

1383

[INFO] ------------------------------------------------------------------------

When it gets to the deploy step, then it fails:

INFO: Setting up environment.
echo "${KEY_FILE}" | base64 -d >> /tmp/key-file.json
gcloud auth activate-service-account --key-file /tmp/key-file.json --quiet --verbosity=warning
Activated service account credentials for: [[redacted]-205917@appspot.gserviceaccount.com]
gcloud config set project [redacted]-205917 --quiet --verbosity=warning
Updated property [core/project].
INFO: Starting deployment to GCP app engine...
gcloud app --quiet deploy src/main/appengine/app.yaml --stop-previous-version --verbosity=warning
Services to deploy:
descriptor: [/opt/atlassian/pipelines/agent/build/src/main/appengine/app.yaml]
source: [/opt/atlassian/pipelines/agent/build/src/main/appengine]
target project: [[redacted]-205917]
target service: [member-web-service]
target version: [20230708t125134]
target url: [https://member-web-service-dot-[redacted]-205917.uc.r.appspot.com]
target service account: [[redacted]-205917@appspot.gserviceaccount.com]
Beginning deployment of service [member-web-service]...
Created .gcloudignore file. See `gcloud topic gcloudignore` for details.
╔════════════════════════════════════════════════════════════╗
╠═ Uploading 0 files to Google Cloud Storage ═╣
╚════════════════════════════════════════════════════════════╝
File upload done.
Updating service [member-web-service]...
...........................................................................................................................failed.
ERROR: (gcloud.app.deploy) Error Response: [9] Cloud build 40538f50-fc9b-4917-b931-30d7a694713f status: FAILURE
did not find any jar files with a Main-Class manifest entry
Full build logs: https://console.cloud.google.com/cloud-build/builds;region=us-central1/40538f50-fc9b-4917-b931-30d7a694713f?project=496993301316
✖ Deployment failed.

Seems really strange as everything is running smooth until it tells me there's 0 files that were uploaded, but I think that's just because my code hasn't changed and the sum of the files should be identical.  I will note that when I run the deploy with Maven locally, it deploys just fine. I only seem to have issues while using the pipeline.

[mvalent23 · Accepted solution]

So, I accidentally stumbled on to the real answer to the problem while troubleshooting some configuration issues I saw when the app was actually deployed through the pipeline vs the local maven build.  The problem lies in here, in the original script:

- pipe: atlassian/google-app-engine-deploy:1.2.0
                          variables:
                            KEY_FILE: $KEY_FILE
                            PROJECT: $GCLOUD_PROJECT
                       ---->DEPLOYABLES: 'src/main/appengine/app.yaml'
                            STOP_PREVIOUS_VERSION: 'true'

the DEPLOYABLES: parameter was pointing at the yaml file in my source tree - and when the pipeline ultimately runs 'gcloud app deploy' it was using the source tree/s appengine directory as the root, and it obviously found no jar files to upload.  So what I ended up doing was mimicking what appengine:deploy maven task is doing and building an appengine-staging area in the build output (target/ in my maven build case), which contains the app.yaml file and the Jar file, and THEN passing in

- pipe: atlassian/google-app-engine-deploy:1.2.0
                          variables:
                            KEY_FILE: $KEY_FILE
                            PROJECT: $GCLOUD_PROJECT
                            DEPLOYABLES: 'target/appengine-staging/app.yaml'
                            STOP_PREVIOUS_VERSION: 'true'

This not only got the pipe to work, but also fixed my environment configuration I was running into!

[mvalent23]

Workaround: I found the pipe file here:
https://bitbucket.org/atlassian/google-app-engine-deploy/src/1.2.0/pipe/pipe.sh

And between that and doing some manual gcloud sdk configuration found here:
https://stackoverflow.com/questions/47989200/cant-deploy-on-google-cloud-app-engine-using-bitbucket-pipeline

I think I was actually able to get something deployed!  It's a shame because using the pipe would have been much easier.  But ultimately we are doing the same thing now (so it's even a bit more of a mystery as to why writing it out the long way works, and the pipe doesn't).

For anyone interested, this is what I ended up with.  Keep in mind the third block is only necessary if running sql proxy to hit a database through a proxy in my test environment.  May not be applicable in your setup:

- step:
    caches:
        - maven
    name: Proxy setup and Build MemberApp
    script:
        - echo "Downloading gcloud cli"
        - curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-437.0.1-linux-x86_64.tar.gz
        - tar -xf google-cloud-cli-437.0.1-linux-x86_64.tar.gz -C  /tmp
        - /tmp/google-cloud-sdk/install.sh
        - source /tmp/google-cloud-sdk/path.bash.inc

        - echo "Regisering gcloud key"
        - echo $KEY_FILE | base64 --decode --ignore-garbage > /tmp/gcloud-api-key.json
        - gcloud auth activate-service-account --key-file /tmp/gcloud-api-key.json
        - gcloud config set project $GCLOUD_PROJECT

        - echo "Installing the cloudsql proxy client"
        - curl -o cloud-sql-proxy https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.4.0/cloud-sql-proxy.linux.amd64
        - chmod +x cloud-sql-proxy
        - ./cloud-sql-proxy $CLOUDSQL_CONN_NAME > cloudsql.log 2>&1 &
        
        - gcloud components install app-engine-java
        - mvn -B clean install -Pdev --file pom.xml
        - gcloud app deploy --stop-previous-version

[Igor Stoyanov]

@mvalent23  hi. Check this article, maybe it can help you.

Regards, Igor

[mvalent23]

I had found this post as well.  Originally when running Maven locally, the accepted answer was the solution.  I made sure that the plugin was present in the build script and was able to deploy.

However, once I tried using the pipeline, I'm still having the same issue.  I'm not sure if this is due tot he maven docker image I'm using to execute my pipeline script, or if this is just a deployment issue on the G Cloud side

[mvalent23]

@Igor Stoyanov I came back to revisit this.  I did read through the article you linked again, and I think the disconnect here is that I cannot seem to figure out the difference between when I manually run maven deploy in my environment, and when the app engine pipeline executes.  There's a difference there, but I'm not yet finding out what it is.