we currently try to migrate from our Jenkins server to Bitbucket Pipelines.
We have a multi module Maven / Spring Project which is painfully slow in build times and takes around 60 minutes in pipelines, while only taking around 10 minutes in our Jenkins server.
I therefore went ahead and pulled the docker container locally and started it exactly as described here https://support.atlassian.com/bitbucket-cloud/docs/debug-pipelines-locally-with-docker/ and issued the mvn command.
On my machine the build is just fine and fast as always, it only seems to be slow within the Bitbucket Pipelines cloud.
bitbucket-pipelines.yml (we first setup the build environment with a few npm commands. These take time as well, but they are not the problem here)
- bash create-settings.sh
- npm install --prefix web
- npm install firstname.lastname@example.org --save --prefix web
- npm install --prefix web2
- npm install email@example.com --save --prefix web2
- npm install --prefix web3
- npm install firstname.lastname@example.org --save --prefix web3
- npm install --prefix web4
- npm install email@example.com --save --prefix web4
- npm install --prefix web5
- npm install firstname.lastname@example.org --save --prefix web5
- npm install --prefix web6
- npm install email@example.com --save --prefix web6
- npm install --prefix web7
- npm install firstname.lastname@example.org --save --prefix web7
- npm install --prefix web8
- npm install email@example.com --save --prefix web8
- npm install --prefix web9
- npm install firstname.lastname@example.org --save --prefix web9
- MAVEN_OPTS="-Dorg.slf4j.simpleLogger.showDateTime=true -Dorg.slf4j.simpleLogger.dateTimeFormat=HH:mm:ss" mvn -B -s settings.xml verify deploy --file pom.xml -P someprofiles
# Collect checkstyle results, if any, and convert to Bitbucket Code Insights.
- pipe: atlassian/checkstyle-report:0.2.0
Looking through the logs it occurs to me that some of the dependencies might not be stored in the cache and are downloaded on every build.
Furthermore, I am wondering why our private registry docker image is pulled every single time entirely. This seems like a waste of time and bandwith.
Is there something that I miss? Like a "faster" switch?
I already went through a lot of documentation.
Hi @Mehlstaub ,
Regarding the private registry docker image being pulled every single time, I'm afraid that at the moment we don't cache private images for the build container. We cache public image from Dockerhub, but not private ones. We have a feature request for what you are asking in our issue tracker:
I would suggest that you add your vote there (by selecting the Vote for this issue link) as the number of votes helps the development team and product managers better understand the demand for new features. You are more than welcome to leave any feedback, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.
You additionally mentioned you believe some of the dependencies may not be stored in the cache but downloaded on every build.
A few things to check:
My suggestion would be to open a support ticket for this issue, and provide also a link for a build where you think there's an issue with cache and mention which dependencies seem to get downloaded every time. If there is a support ticket open, the engineer working on your case will be able to access the builds and logs from our side, which will allow for faster and better troubleshooting for this issue.
You can open a support ticket via https://support.atlassian.com/contact/#/, in "What can we help you with" select "Technical issues and bugs" and then Bitbucket Cloud as product. The support ticket will be visible only to you and Atlassian staff.
If you have any questions, please feel free to let me know.
Turns out we got 2 problems here.
As Theodora said the caches. We did put our final artifacts in the maven repo and therefore the cache size was too big at the end. This is because our WAR file artifacts are pretty huge.
We simply added an rm -rf for our own created artifacts to remove them before cache creation. Now a maven cache is created and the following buils go fast.
Additionally we see that upload to our repository server is super slow due to US <> EU datacenter locations. To solve this problem we now use a runner instance in our datacenter so that the build and upload is executed "nearby" instead of on the other side of the world.
Hey everyone, over the past year, we've added several new features to how you can use Bitbucket and Jira together. We'd love to know how you'll are using the integration and how it's helping your tea...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events