Is it possible to increase the memory limit assigned for the build environment?

Question

According to the documentation:

* The build container is given 1024 MB of the total memory, which covers your build process and some Pipelines overheads (agent container, logging, etc).

However, for the build I'm running, I seem to be hitting this limit:

The step should have a total of 4096MB of memory available for build + services. Is it possible to increase the memory limit for the "Build" container to e.g. 2096MB?

I know that the "size: 2x" option exists, but it seems that if the extra 3GB is available for the step anyway, why shouldn't I be able to use it for the build container?

Theodora Boudale · Answer

Hi,

You can add the following commands at the very beginning of the script in the bitbucket-pipelines.yml file:

- while true; do date && ps -aux && sleep 15 && echo ""; done &
- while true; do echo "Memory usage in megabytes:" && echo $((`cat /sys/fs/cgroup/memory/memory.memsw.usage_in_bytes | awk '{print $1}'`/1048576)); sleep 60; done &

These commands will show memory usage (also per process) in the Build log when you run a build, this way you can see which processes need most of the memory.

Kind regards,
Theodora

basdado · Answer

Is there any way to see how much memory the build container used?

I had no other services active, so the build container should have had at least the 4GB. I worked around it by enabling the 2x option for now, but I am wondering how much memory the build was actually using.

Theodora Boudale · Answer

Hello,

If you don't run any services and don't enable the 2x option, the build container has 4 GB of memory available.
If you have services running, then the build container gets 4 GB minus the memory used by the services.

I believe that the sentence you quoted from the documentation may be referring to the minimum amount of memory required for the build container. I have reached out to the documentation team to clarify this specific part of the doc.

If you run into the "Container 'Build' exceeded memory limit" error, then I would suggest:
* in case you use services, limit the memory used by services (there are examples in the doc that you linked)
* enforce the max size of memory used in your build to be under the available memory
* use the 2x option

Please feel free to reach out if you have any further questions!

Kind regards,
Theodora