Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,293,855
Community Members
 
Community Events
165
Community Groups

Pipelines poor maven performance

Edited

Hi there,

we currently try to migrate from our Jenkins server to Bitbucket Pipelines. 

We have a multi module Maven / Spring Project which is painfully slow in build times and takes around 60 minutes in pipelines, while only taking around 10 minutes in our Jenkins server. 

I therefore went ahead and pulled the docker container locally and started it exactly as described here https://support.atlassian.com/bitbucket-cloud/docs/debug-pipelines-locally-with-docker/ and issued the mvn command. 

On my machine the build is just fine and fast as always, it only seems to be slow within the Bitbucket Pipelines cloud. 

 

bitbucket-pipelines.yml (we first setup the build environment with a few npm commands. These take time as well, but they are not the problem here)

image:
name: privateregistry/privateimage
username: $_USER
password: $_PASS
email: someone@example

pipelines
:
default:
- step:
name: Build
size: 2x
caches:
- maven
- node
- docker
script:
- bash create-settings.sh
- npm install --prefix web
- npm install grunt-cli@1.2.0 --save --prefix web
- npm install --prefix web2
- npm install grunt-cli@1.2.0 --save --prefix web2
- npm install --prefix web3
- npm install grunt-cli@1.2.0 --save --prefix web3
- npm install --prefix web4
- npm install grunt-cli@1.2.0 --save --prefix web4
- npm install --prefix web5
- npm install grunt-cli@1.2.0 --save --prefix web5
- npm install --prefix web6
- npm install grunt-cli@1.2.0 --save --prefix web6
- npm install --prefix web7
- npm install grunt-cli@1.2.0 --save --prefix web7
- npm install --prefix web8
- npm install grunt-cli@1.2.0 --save --prefix web8
- npm install --prefix web9
- npm install grunt-cli@1.2.0 --save --prefix web9
- MAVEN_OPTS="-Dorg.slf4j.simpleLogger.showDateTime=true -Dorg.slf4j.simpleLogger.dateTimeFormat=HH:mm:ss" mvn -B -s settings.xml verify deploy --file pom.xml -P someprofiles
after-script:
# Collect checkstyle results, if any, and convert to Bitbucket Code Insights.
- pipe: atlassian/checkstyle-report:0.2.0

 

Looking through the logs it occurs to me that some of the dependencies might not be stored in the cache and are downloaded on every build. 

Furthermore, I am wondering why our private registry docker image is pulled every single time entirely. This seems like a waste of time and bandwith. 

Is there something that I miss? Like a "faster" switch?

 

I already went through a lot of documentation.

2 answers

2 accepted

0 votes
Answer accepted

Turns out we got 2 problems here. 

As Theodora said the caches. We did put our final artifacts in the maven repo and therefore the cache size was too big at the end. This is because our WAR file artifacts are pretty huge. 

We simply added an rm -rf for our own created artifacts to remove them before cache creation. Now a maven cache is created and the following buils go fast. 

 

Additionally we see that upload to our repository server is super slow due to US <> EU datacenter locations. To solve this problem we now use a runner instance in our datacenter so that the build and upload is executed "nearby" instead of on the other side of the world. 

0 votes
Answer accepted

Hi @Mehlstaub ,

Regarding the private registry docker image being pulled every single time, I'm afraid that at the moment we don't cache private images for the build container. We cache public image from Dockerhub, but not private ones. We have a feature request for what you are asking in our issue tracker:

I would suggest that you add your vote there (by selecting the Vote for this issue link) as the number of votes helps the development team and product managers better understand the demand for new features. You are more than welcome to leave any feedback, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

You additionally mentioned you believe some of the dependencies may not be stored in the cache but downloaded on every build.

A few things to check:

  • Do you see any saved caches if you navigate to the Pipelines page of the repo, and then select the link "Caches"?
  • If so, do you see a message about the caches being downloaded and extracted in the "Build setup" of your builds?

My suggestion would be to open a support ticket for this issue, and provide also a link for a build where you think there's an issue with cache and mention which dependencies seem to get downloaded every time. If there is a support ticket open, the engineer working on your case will be able to access the builds and logs from our side, which will allow for faster and better troubleshooting for this issue.

You can open a support ticket via https://support.atlassian.com/contact/#/, in "What can we help you with" select "Technical issues and bugs" and then Bitbucket Cloud as product. The support ticket will be visible only to you and Atlassian staff.

If you have any questions, please feel free to let me know.

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
Community showcase
Published in Bitbucket

Git push size limits are coming to Bitbucket Cloud starting April 4th, 2022

Beginning on April 4th, we will be implementing push limits. This means that your push cannot be completed if it is over 3.5 GB. If you do attempt to complete a push that is over 3.5 GB, it will fail...

2,130 views 2 9
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you