Pipelines memory limit exceeded Edited

we use pipelines.

we sometimes have jobs that fail with the message 'Build memory limit exceeded.'

we also run the same docker image internally as we run in pipelines. (we run the same image locally as we declare in the image: of bitbucket-pipelines.yml)

we want a way to run our docker image locally with the same limitations that pipelines enforces (4gb) : https://confluence.atlassian.com/bitbucket/limitations-of-bitbucket-pipelines-827106051.html#LimitationsofBitbucketPipelines-Buildlimits

we want to do this to make sure we are staying within 90% of what the limit is of pipelines.  (so 3.6GB)

Is this a way to achieve that?

docker run --rm --memory=3600M --memory-swap=3600M docker-image-same-as-we-run-in-pipelines

 

 

5 answers

This widget could not be displayed.

We have the same issue with our Java project; during the building process we get a "Build exceeded memory limit" error, but if we run the build locally (as reported here: https://confluence.atlassian.com/bitbucket/debug-your-pipelines-locally-with-docker-838273569.html) all works fine without any problem.

I also tried to limit the memory allocated by maven with the MAVEN_OPTS variable, but anyway the build is stopped (randomly).

Is there a fix for the memory limit issue?

This widget could not be displayed.

We have the same issue with our maven build.

Anybody has found a way to hard limit maven to less than 4G? 

This widget could not be displayed.

Are you using any service containers or the Docker build and push functionality? This will *also* use part of your 4GB of memory for the build. (1GB per service container)

(See here: https://confluence.atlassian.com/bitbucket/use-services-and-databases-in-bitbucket-pipelines-874786688.html )

EDIT: This was a bug. It's currently being worked on. Follow it here: https://bitbucket.org/site/master/issues/14666/enabling-docker-for-pipelines-wrongfully

Nope, it's just the maven build with tests

Same here.  Just maven with tests.  Please help us debug this.

Sorry for the delayed response.

Can you try a slight variation fo your command for your local debugging.

docker run -it --volume=/Users/myUserName/code/localDebugRepo:/localDebugRepo --workdir="/localDebugRepo" --memory=4g --memory-swap=4097m --entrypoint=/bin/bash python:2.7

Akin to these docs: https://confluence.atlassian.com/bitbucket/debug-your-pipelines-locally-with-docker-838273569.html

The difference is I've changed memory-swap from 4g to 4097m (purposefully 1 MB larger than 4GB).

I need to do a little more investigation, but if the values are the same then docker may revert to default swap behaviour, giving you 8GB of swap instead of 0GB. 

I'll update the docs once I confirm this. :) See if that change helps with reproducing the issue.

This widget could not be displayed.

Same issue for us. We're doing a Scala (sbt plus ScalaTest) build using a PostgreSQL database as a service. When I run this locally, it works fine using about 1GB of memory. If I understand the documentation correctly, Posgtres will use 1GB of the 4GB for a pipeline. So it should not run out of memory?

I also ran the build in Docker as suggested in the Pipelines documentation for debugging locally and the build is fine there as well with the container using about 1.6GB of memory. For some branches the memory exceeding happens only occasionally (immediate rerun mostly works), while one other branch does not work at all atm. This suggests to me that the build is very close to some kind of memory threshhold, but I do not know what is the limiting factor here? Any tips how to debug this?

This widget could not be displayed.

Hi! actually, we're still having the same issue - a simple maven build with tests is failing unless we use `size: 2x`.. We've tried limiting memory with MAVEN_OPTS, as well as setting

-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap`

parameters (we're using Java 8, so technically without these flags, JVM is not aware it's running in a container), but still no luck..

Did anyone happen to find a solution for that?

Suggest an answer

Log in or Sign up to answer
Community showcase
Published Aug 21, 2018 in Bitbucket

Branch Management with Bitbucket

As a project manager, I have discovered that different developers want to bring their previous branching method with them when they join the team. Some developers are used to performing individual wo...

1,326 views 8 11
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you