Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

Bitwise DEVS
Contributor
October 29, 2023

We noticed that Gradle Daemon always get killed recently, not sure if the issue lies with the Gradle itself or Docker.

 

First attempt, since many thread says that disabling daemon will fix it.

 

 - step: # Non-parallel step to build Android debug APK and upload it to AppSweep
          name: Build APK (Debug) & Upload to AppSweep
          caches: # Caching speed up subsequent execution https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/
            - gradle
          script:
            - git submodule update --init --recursive # Clone the missing submodules, fetches any new remote commits, and updates the directory tree. Adding the --init flag to the command eliminates the need to run git submodule init . The --recursive option tells Git to check the submodules for nested submodules and update them as well.
            - chmod +x gradlew # Change permission of this file to make it executable
            - "./gradlew
                         -PANDROID_BUILD_COMPILED_SDK_VERSION=$ANDROID_BUILD_COMPILED_SDK_VERSION
                         -PANDROID_BUILD_MIN_SDK_VERSION=$ANDROID_BUILD_MIN_SDK_VERSION
                         -PANDROID_BUILD_TARGET_SDK_VERSION=$ANDROID_BUILD_TARGET_SDK_VERSION
                         -PDEBUG_KEYSTORE_FILE=../keys/$DEBUG_KEYSTORE_FILE
                         -PDEBUG_KEYSTORE_PASSWORD=$DEBUG_PASSWORD
                         -PDEBUG_KEYSTORE_KEY_ALIAS=$DEBUG_KEY_ALIAS
                         -PDEBUG_KEYSTORE_KEY_PASSWORD=$DEBUG_KEY_PASSWORD
                         -PCRYPTONIAN_KEYSTORE_FILE=../keys/$KEYSTORE_FILE
                         -PCRYPTONIAN_KEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
                         -PCRYPTONIAN_KEYSTORE_KEY_ALIAS=$KEYSTORE_KEY_ALIAS
                         -PCRYPTONIAN_KEYSTORE_KEY_PASSWORD=$KEY_PASSWORD
                         -PAPPSWEEP_API_KEY=$APPSWEEP_API_KEY
                         -PPLAY_API_KEY_FILE=../keys/$GOOGLE_PLAY_API_KEY_FILE
                          clean :app:assembleDebug :app:uploadToAppSweepDebug --no-daemon" # Fix issue for 'Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)'
          artifacts: # Keep these artifacts for subsequent pipeline steps
            - app/build/outputs/** # Artifacts are files that are produced by a step https://support.atlassian.com/bitbucket-cloud/docs/use-artifacts-in-steps/

 

Second attempt, adding Gradle options that is constantly being suggested by community to adjust JVM.

 

      - step: # Non-parallel step to build Android debug APK and upload it to AppSweep
          name: Build APK (Debug) & Upload to AppSweep
          caches: # Caching speed up subsequent execution https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/
            - gradle
          script:
            - git submodule update --init --recursive # Clone the missing submodules, fetches any new remote commits, and updates the directory tree. Adding the --init flag to the command eliminates the need to run git submodule init . The --recursive option tells Git to check the submodules for nested submodules and update them as well.
            - chmod +x gradlew # Change permission of this file to make it executable
            - export GRADLE_OPTS='-Dorg.gradle.jvmargs="-Xmx2048m -XX:+UseParallelGC -XX:MaxPermSize=1024m -XX:ReservedCodeCacheSize=440m -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8" -Dorg.gradle.parallel=true -Dorg.gradle.daemon=false -Dorg.gradle.configureondemand=true' # Fix issue for 'Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)'
            - chmod +x gradlew # Change permission of this file to make it executable.
            - "./gradlew
                         -PANDROID_BUILD_COMPILED_SDK_VERSION=$ANDROID_BUILD_COMPILED_SDK_VERSION
                         -PANDROID_BUILD_MIN_SDK_VERSION=$ANDROID_BUILD_MIN_SDK_VERSION
                         -PANDROID_BUILD_TARGET_SDK_VERSION=$ANDROID_BUILD_TARGET_SDK_VERSION
                         -PDEBUG_KEYSTORE_FILE=../keys/$DEBUG_KEYSTORE_FILE
                         -PDEBUG_KEYSTORE_PASSWORD=$DEBUG_PASSWORD
                         -PDEBUG_KEYSTORE_KEY_ALIAS=$DEBUG_KEY_ALIAS
                         -PDEBUG_KEYSTORE_KEY_PASSWORD=$DEBUG_KEY_PASSWORD
                         -PCRYPTONIAN_KEYSTORE_FILE=../keys/$KEYSTORE_FILE
                         -PCRYPTONIAN_KEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
                         -PCRYPTONIAN_KEYSTORE_KEY_ALIAS=$KEYSTORE_KEY_ALIAS
                         -PCRYPTONIAN_KEYSTORE_KEY_PASSWORD=$KEY_PASSWORD
                         -PAPPSWEEP_API_KEY=$APPSWEEP_API_KEY
                         -PPLAY_API_KEY_FILE=../keys/$GOOGLE_PLAY_API_KEY_FILE
                          clean :app:assembleDebug :app:uploadToAppSweepDebug"
          artifacts: # Keep these artifacts for subsequent pipeline steps
            - app/build/outputs/** # Artifacts are files that are produced by a step https://support.atlassian.com/bitbucket-cloud/docs/use-artifacts-in-steps/

 
Third attempt, just a different Gradle options since the above configuration still sometimes failing.

 

- step: # Non-parallel step to build Android debug APK and upload it to AppSweep
          name: Build APK (Debug) & Upload to AppSweep
          caches: # Caching speed up subsequent execution https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/
            - gradle
          script:
            - git submodule update --init --recursive # Clone the missing submodules, fetches any new remote commits, and updates the directory tree. Adding the --init flag to the command eliminates the need to run git submodule init . The --recursive option tells Git to check the submodules for nested submodules and update them as well.
            - chmod +x gradlew # Change permission of this file to make it executable
            - export GRADLE_OPTS='-Dorg.gradle.jvmargs="-Xmx2048m -XX:+UseParallelGC -XX:ReservedCodeCacheSize=440m -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8" -Dorg.gradle.parallel=true -Dorg.gradle.daemon=false -Dorg.gradle.configureondemand=true' # Fix issue for 'Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)'
            - chmod +x gradlew # Change permission of this file to make it executable.
            - "./gradlew
                         -PANDROID_BUILD_COMPILED_SDK_VERSION=$ANDROID_BUILD_COMPILED_SDK_VERSION
                         -PANDROID_BUILD_MIN_SDK_VERSION=$ANDROID_BUILD_MIN_SDK_VERSION
                         -PANDROID_BUILD_TARGET_SDK_VERSION=$ANDROID_BUILD_TARGET_SDK_VERSION
                         -PDEBUG_KEYSTORE_FILE=../keys/$DEBUG_KEYSTORE_FILE
                         -PDEBUG_KEYSTORE_PASSWORD=$DEBUG_PASSWORD
                         -PDEBUG_KEYSTORE_KEY_ALIAS=$DEBUG_KEY_ALIAS
                         -PDEBUG_KEYSTORE_KEY_PASSWORD=$DEBUG_KEY_PASSWORD
                         -PCRYPTONIAN_KEYSTORE_FILE=../keys/$KEYSTORE_FILE
                         -PCRYPTONIAN_KEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
                         -PCRYPTONIAN_KEYSTORE_KEY_ALIAS=$KEYSTORE_KEY_ALIAS
                         -PCRYPTONIAN_KEYSTORE_KEY_PASSWORD=$KEY_PASSWORD
                         -PAPPSWEEP_API_KEY=$APPSWEEP_API_KEY
                         -PPLAY_API_KEY_FILE=../keys/$GOOGLE_PLAY_API_KEY_FILE
                          clean :app:assembleDebug :app:uploadToAppSweepDebug"
          artifacts: # Keep these artifacts for subsequent pipeline steps
            - app/build/outputs/** # Artifacts are files that are produced by a step https

 

Fourth attempt, again just a different Gradle options since the above configuration still sometimes failing.

 

- step: # Non-parallel step to build Android debug APK and upload it to AppSweep
          name: Build APK (Debug) & Upload to AppSweep
          caches: # Caching speed up subsequent execution https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/
            - gradle
          script:
            - git submodule update --init --recursive # Clone the missing submodules, fetches any new remote commits, and updates the directory tree. Adding the --init flag to the command eliminates the need to run git submodule init . The --recursive option tells Git to check the submodules for nested submodules and update them as well.
            - chmod +x gradlew # Change permission of this file to make it executable
            - export GRADLE_OPTS='-XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -Dorg.gradle.jvmargs="-Xmx2048m -XX:MaxPermSize=1024m -XX:ReservedCodeCacheSize=440m -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8" -Dorg.gradle.parallel=false -Dorg.gradle.daemon=false -Dorg.gradle.configureondemand=true' # Fix issue for 'Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)'
            - chmod +x gradlew # Change permission of this file to make it executable.
            - "./gradlew
                         -PANDROID_BUILD_COMPILED_SDK_VERSION=$ANDROID_BUILD_COMPILED_SDK_VERSION
                         -PANDROID_BUILD_MIN_SDK_VERSION=$ANDROID_BUILD_MIN_SDK_VERSION
                         -PANDROID_BUILD_TARGET_SDK_VERSION=$ANDROID_BUILD_TARGET_SDK_VERSION
                         -PDEBUG_KEYSTORE_FILE=../keys/$DEBUG_KEYSTORE_FILE
                         -PDEBUG_KEYSTORE_PASSWORD=$DEBUG_PASSWORD
                         -PDEBUG_KEYSTORE_KEY_ALIAS=$DEBUG_KEY_ALIAS
                         -PDEBUG_KEYSTORE_KEY_PASSWORD=$DEBUG_KEY_PASSWORD
                         -PCRYPTONIAN_KEYSTORE_FILE=../keys/$KEYSTORE_FILE
                         -PCRYPTONIAN_KEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
                         -PCRYPTONIAN_KEYSTORE_KEY_ALIAS=$KEYSTORE_KEY_ALIAS
                         -PCRYPTONIAN_KEYSTORE_KEY_PASSWORD=$KEY_PASSWORD
                         -PAPPSWEEP_API_KEY=$APPSWEEP_API_KEY
                         -PPLAY_API_KEY_FILE=../keys/$GOOGLE_PLAY_API_KEY_FILE
                          clean :app:assembleDebug :app:uploadToAppSweepDebug"
          artifacts: # Keep these artifacts for subsequent pipeline steps
            - app/build/outputs/** # Artifacts are files that are produced by a step https

 

Last attempt, separating build task to upload task so that Gradle is not running too long.

 

- step: # Non-parallel step to build Android debug APK
name: Build APK (Debug)
caches: # Caching speed up subsequent execution https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/
- gradle
script:
- git submodule update --init --recursive # Clone the missing submodules, fetches any new remote commits, and updates the directory tree. Adding the --init flag to the command eliminates the need to run git submodule init . The --recursive option tells Git to check the submodules for nested submodules and update them as well.
- chmod +x gradlew # Change permission of this file to make it executable.
- "./gradlew
-PANDROID_BUILD_COMPILED_SDK_VERSION=$ANDROID_BUILD_COMPILED_SDK_VERSION
-PANDROID_BUILD_MIN_SDK_VERSION=$ANDROID_BUILD_MIN_SDK_VERSION
-PANDROID_BUILD_TARGET_SDK_VERSION=$ANDROID_BUILD_TARGET_SDK_VERSION
-PDEBUG_KEYSTORE_FILE=../keys/$DEBUG_KEYSTORE_FILE
-PDEBUG_KEYSTORE_PASSWORD=$DEBUG_PASSWORD
-PDEBUG_KEYSTORE_KEY_ALIAS=$DEBUG_KEY_ALIAS
-PDEBUG_KEYSTORE_KEY_PASSWORD=$DEBUG_KEY_PASSWORD
-PCRYPTONIAN_KEYSTORE_FILE=../keys/$KEYSTORE_FILE
-PCRYPTONIAN_KEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
-PCRYPTONIAN_KEYSTORE_KEY_ALIAS=$KEYSTORE_KEY_ALIAS
-PCRYPTONIAN_KEYSTORE_KEY_PASSWORD=$KEY_PASSWORD
-PAPPSWEEP_API_KEY=$APPSWEEP_API_KEY
-PPLAY_API_KEY_FILE=../keys/$GOOGLE_PLAY_API_KEY_FILE
clean :app:assembleDebug"
artifacts: # Keep these artifacts for subsequent pipeline steps
- app/build/outputs/** # Artifacts are files that are produced by a step https://support.atlassian.com/bitbucket-cloud/docs/use-artifacts-in-steps/
- step: # Non-parallel step uploading debug APK to AppSweep
name: Upload Debug APK to AppSweep
caches: # Caching speed up subsequent execution https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/
- gradle
script:
- git submodule update --init --recursive # Clone the missing submodules, fetches any new remote commits, and updates the directory tree. Adding the --init flag to the command eliminates the need to run git submodule init . The --recursive option tells Git to check the submodules for nested submodules and update them as well.
- chmod +x gradlew # Change permission of this file to make it executable.
- "./gradlew
-PANDROID_BUILD_COMPILED_SDK_VERSION=$ANDROID_BUILD_COMPILED_SDK_VERSION
-PANDROID_BUILD_MIN_SDK_VERSION=$ANDROID_BUILD_MIN_SDK_VERSION
-PANDROID_BUILD_TARGET_SDK_VERSION=$ANDROID_BUILD_TARGET_SDK_VERSION
-PDEBUG_KEYSTORE_FILE=../keys/$DEBUG_KEYSTORE_FILE
-PDEBUG_KEYSTORE_PASSWORD=$DEBUG_PASSWORD
-PDEBUG_KEYSTORE_KEY_ALIAS=$DEBUG_KEY_ALIAS
-PDEBUG_KEYSTORE_KEY_PASSWORD=$DEBUG_KEY_PASSWORD
-PCRYPTONIAN_KEYSTORE_FILE=../keys/$KEYSTORE_FILE
-PCRYPTONIAN_KEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
-PCRYPTONIAN_KEYSTORE_KEY_ALIAS=$KEYSTORE_KEY_ALIAS
-PCRYPTONIAN_KEYSTORE_KEY_PASSWORD=$KEY_PASSWORD
-PAPPSWEEP_API_KEY=$APPSWEEP_API_KEY
-PPLAY_API_KEY_FILE=../keys/$GOOGLE_PLAY_API_KEY_FILE
:app:uploadToAppSweepDebug"
artifacts: # Keep these artifacts for subsequent pipeline steps
- app/build/outputs/** # Artifacts are files that are produced by a step https://support.atlassian.com/bitbucket-cloud/docs/use-artifacts-in-steps/

 

Screenshot 2023-10-30 at 11.12.25 AM.png

2 answers

1 accepted

1 vote
Answer accepted
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 31, 2023

Hi @Bitwise DEVS,

The error message you posted here indicates that the build needed more memory than the memory that was available and it failed.

Every regular step in a Pipelines build gets 4 GB of memory. If it has services, each service gets 1 GB of memory (unless you specify otherwise in your yml) and the rest is left for the build container.

You can add the following commands at the beginning of your step's script in your yml file if you want to see memory usage per process in order to figure out which processes consume a lot of memory:

- while true; do date && ps -aux && sleep 5 && echo ""; done &
- while true; do date && echo "Memory usage in megabytes:" && echo $((`cat /sys/fs/cgroup/memory/memory.memsw.usage_in_bytes | awk '{print $1}'`/1048576)) && echo "" && sleep 5; done &

You can also debug the build locally with Docker as per this guide, and then run the following command in a separate window to get memory usage:

while true; do date >> mem.txt && docker stats -a --no-stream >> mem.txt && sleep 2; done &

If the build cannot be configured to use less memory, you can configure up to 8 GB of memory for a step with the option size: 2x

Please keep in mind that 2x steps use twice the number of build minutes of a 1x step.

8 GB is the maximum memory that can be configured for steps running on our infrastructure. If you use a self-hosted runner for your builds, you can configure up to 32 GB of memory for a step (provided that the host machine has that memory available):

Kind regards,
Theodora

Bitwise DEVS
Contributor
October 31, 2023

Thanks a lot, for now I am setting it to 2x since it is the only solution that is working. Not sure if this is an issue with new Gradle since this is not frequently happening before.

Like Theodora Boudale likes this
2 votes
Imam Robani
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
February 25, 2024

I have the same issue, I already set `size: 2x` but still not working

can anyone help?

Screenshot 2024-02-26 at 11.59.36.png

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
February 27, 2024

Hi Imam,

You can check which processes consume memory in your build (I provided in my previous answer the commands you can include in your yml file to troubleshoot this). You can then check if your build can be configured to use less memory. If you need help with that specifically, it may be best to reach out to a forum specific to the tools you are using in your build.

If the step cannot be configured to use less memory, you can look into using one of our self-hosted runners.

I provided more info in my previous answer.

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
AUG Leaders

Atlassian Community Events