Pipelines poor maven performance

[Mehlstaub]

Hi there,

we currently try to migrate from our Jenkins server to Bitbucket Pipelines. 

We have a multi module Maven / Spring Project which is painfully slow in build times and takes around 60 minutes in pipelines, while only taking around 10 minutes in our Jenkins server. 

I therefore went ahead and pulled the docker container locally and started it exactly as described here https://support.atlassian.com/bitbucket-cloud/docs/debug-pipelines-locally-with-docker/ and issued the mvn command. 

On my machine the build is just fine and fast as always, it only seems to be slow within the Bitbucket Pipelines cloud. 

bitbucket-pipelines.yml (we first setup the build environment with a few npm commands. These take time as well, but they are not the problem here)

image:
name: privateregistry/privateimage
username: $_USER
password: $_PASS
email: someone@example

pipelines:
  default:
    - step:
        name: Build
        size: 2x
        caches:
          - maven
          - node
          - docker
        script:
          - bash create-settings.sh
          - npm install --prefix web
          - npm install grunt-cli@1.2.0 --save --prefix web
          - npm install --prefix web2
          - npm install grunt-cli@1.2.0 --save --prefix web2
          - npm install --prefix web3
          - npm install grunt-cli@1.2.0 --save --prefix web3
          - npm install --prefix web4
          - npm install grunt-cli@1.2.0 --save --prefix web4
          - npm install --prefix web5
          - npm install grunt-cli@1.2.0 --save --prefix web5
          - npm install --prefix web6
          - npm install grunt-cli@1.2.0 --save --prefix web6
          - npm install --prefix web7
          - npm install grunt-cli@1.2.0 --save --prefix web7
          - npm install --prefix web8
          - npm install grunt-cli@1.2.0 --save --prefix web8
          - npm install --prefix web9
          - npm install grunt-cli@1.2.0 --save --prefix web9
          - MAVEN_OPTS="-Dorg.slf4j.simpleLogger.showDateTime=true -Dorg.slf4j.simpleLogger.dateTimeFormat=HH:mm:ss" mvn -B -s settings.xml verify deploy --file pom.xml -P someprofiles 
        after-script:
          # Collect checkstyle results, if any, and convert to Bitbucket Code Insights.
          - pipe: atlassian/checkstyle-report:0.2.0

Looking through the logs it occurs to me that some of the dependencies might not be stored in the cache and are downloaded on every build. 

Furthermore, I am wondering why our private registry docker image is pulled every single time entirely. This seems like a waste of time and bandwith. 

Is there something that I miss? Like a "faster" switch?

I already went through a lot of documentation.

[Mehlstaub · Accepted solution]

Turns out we got 2 problems here. 

As Theodora said the caches. We did put our final artifacts in the maven repo and therefore the cache size was too big at the end. This is because our WAR file artifacts are pretty huge. 

We simply added an rm -rf for our own created artifacts to remove them before cache creation. Now a maven cache is created and the following buils go fast. 

Additionally we see that upload to our repository server is super slow due to US <> EU datacenter locations. To solve this problem we now use a runner instance in our datacenter so that the build and upload is executed "nearby" instead of on the other side of the world. 

[Theodora Boudale · Accepted solution]

Hi @Mehlstaub ,

Regarding the private registry docker image being pulled every single time, I'm afraid that at the moment we don't cache private images for the build container. We cache public image from Dockerhub, but not private ones. We have a feature request for what you are asking in our issue tracker:

• https://jira.atlassian.com/browse/BCLOUD-19918

I would suggest that you add your vote there (by selecting the Vote for this issue link) as the number of votes helps the development team and product managers better understand the demand for new features. You are more than welcome to leave any feedback, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

You additionally mentioned you believe some of the dependencies may not be stored in the cache but downloaded on every build.

A few things to check:

• Do you see any saved caches if you navigate to the Pipelines page of the repo, and then select the link "Caches"?
• If so, do you see a message about the caches being downloaded and extracted in the "Build setup" of your builds?

My suggestion would be to open a support ticket for this issue, and provide also a link for a build where you think there's an issue with cache and mention which dependencies seem to get downloaded every time. If there is a support ticket open, the engineer working on your case will be able to access the builds and logs from our side, which will allow for faster and better troubleshooting for this issue.

You can open a support ticket via https://support.atlassian.com/contact/#/, in "What can we help you with" select "Technical issues and bugs" and then Bitbucket Cloud as product. The support ticket will be visible only to you and Atlassian staff.

If you have any questions, please feel free to let me know.

Kind regards,
Theodora