It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

pipeline memory limit - elasticsearch exceeded memory limit.

Vincent Jensen Jan 12, 2018

 

Hello,

I am trying to run my functionnal tests in our bitbucket pipeline with elasticsearch an mysql as services.

But everytime, the pipeline launches, I get this error : elasticsearch exceeded memory limit.

Here is the content of my bitbucket-pipelines.yml : 

 

image:  php:7.1.1
options:
  size: 2x
pipelines:
  default:
    - step:
        caches:
          - composer
        script:
          - apt-get update && apt-get install -y unzip
          - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
          - composer install
          - bin/bootstrap
          - ./vendor/bin/simple-phpunit
        services:
          - elasticsearch
          - database
definitions:
  services:
    elasticsearch:
      image: elasticsearch:alpine
    database:
      image: mysql
      environment:
        MYSQL_ROOT_PASSWORD: root

 

I would be very gratefull of some help or some pointers.

Thx in advance,

Vincent.

 

 

2 answers

2 votes
Igor Chepurnoy Jun 27, 2018

Hi,

The following configuration works for me

definitions:
redis:
image: redis:4.0
memory: 512
elasticsearch:
image: elasticsearch:5.6
environment:
ES_JAVA_OPTS: '-Xms512m -Xmx512m'
Jimmy Lien I'm New Here Feb 26, 2019

Agree with this. The elasticsearch docker config takes up most of the 1024m of memory. Specifying the ES_JAVA_OPTS and reducing the redis memory requirements are key. 

ES_JAVA_OPTS: '-Xms512m -Xmx512m'

In my config I also have chrome webdriver to run browser tests so ended up with a services definition like so:

services:
redis:
image: redis
memory: 512
postgres:
image: postgres
chromedriver:
image: selenium/standalone-chrome
memory: 512
ports:
- "4444:4444"
- "9515:9515"
elasticsearch:
image: elasticsearch:6.5.4
environment:
ES_JAVA_OPTS: -Xms512m -Xmx512m

This 4096 MB limitation isn't great for applications that get anymore complex with dependencies.

0 votes
Motoki Wu Jan 24, 2018

I'm having memory issues as well. A pipeline that worked last Friday hits the memory limit. I even added the `2x` option but it still hits the memory limit.

The build should be pretty similar to each other but all of a sudden a 2x memory is not enough?

Vincent Jensen Jan 25, 2018

For me, it feels like the pipeline can not even build the elasticsearch docker image !!

Matthew Hillier Jan 26, 2018

I'm having a similar issue with an instance of the "microsoft/mssql-server-linux" SQL Server service.

It looks like the additional memory options don't apply to services, they're only shared with the pipelines, so when I start my pipeline, the mouseover on the log tabs in pipelines shows:

  • Build 7GB
  • mssql - 1GB

I can't see any obvious way to re-balance this for my needs.

Vincent Jensen Feb 08, 2018

Hi,

You guys found a way to solve this or will I have to install my own CI tool ?

John Stites Feb 24, 2018 • edited

I got around it for now using a jvm.options file for elastic to limit the memory. I created a new dockerfile that was based on the elastic image we need, added in a jvm.options file with:

-Xms512m
-Xmx512m

This keeps it under the 1GB service size.

They've recently added service size limitations that presumably you could just bump your elastic container to 1.5 or so (https://confluence.atlassian.com/bitbucket/use-services-and-databases-in-bitbucket-pipelines-874786688.html#UseservicesanddatabasesinBitbucketPipelines-Definingaservicewithcustomresourcelimits) but I can't get it to work.

If I follow their example with a "memory: 1536m" the online validator tool says it's expecting an integer, but if I put an integer it ignores it. Perhaps it's so new that their documentation is ahead of their implementation so maybe you'll have luck in the next days/weeks.

Suggest an answer

Log in or Sign up to answer
This widget could not be displayed.
This widget could not be displayed.
Community showcase
Published in Bitbucket Pipelines

Building a Bitbucket Pipe as a casual coder

...ipe.sh :  #!/bin/bash source "$(dirname "$0")/common.sh" enable_debug extra_args="" if [[ "${DEBUG}" == "true" ]]; then extra_args="--verbose" fi # mandatory variables R...

1,964 views 1 19
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you