Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,456,673
Community Members
 
Community Events
176
Community Groups

Using elasticsearch docker image as service, requires periods in environment variable names

Hi,

I'm trying to run elasticsearch docker image as a service alongside my pipelines CI, however some of the configuration options we would like to specify as environment vars for elasticsearch have periods in the name eg. discovery.type, and pipelines does not seem to allow variable names with a period in them.
I'm wondering what options there are to work around this, is it possible to bind the es config file to one in our repository, or some other way to get such named options through? Or would creating a custom docker image with our desired config be the only way to do this for now?

Thanks,

Lyndon

5 answers

This also works:

script:
- docker run -td --name elasticsearch -p 9200:9200 -e discovery.type="single-node" -e ES_JAVA_OPTS="-Xms512m -Xmx512m" docker.elastic.co/elasticsearch/elasticsearch:7.9.2
- docker ps
- sleep 15s
- docker logs elasticsearch
- curl -is 127.0.0.1:9200

That's a shame and really confusing that environment format is not documented.

As an example this yml file passes in the validator https://bitbucket-pipelines.prod.public.atl-paas.net/validator

image: php:7.3-alpine

pipelines:
default:
- step:
caches:
- composer
services:
- elasticsearch
script:
- apk update && apk add git openssh
- sed -i 's/<env name="LOCATION_SEARCH_ELASTICSEARCH_SERVER" value="localhost:9999"\/>/<env name="LOCATION_SEARCH_ELASTICSEARCH_SERVER" value="localhost:9200"\/>/' phpunit.xml
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- composer install
- cp .env.example .env
- php artisan key:generate
- vendor/bin/phpunit

definitions:
services:
elasticsearch:
image: elasticsearch:7.3.0
environment:
- discovery.type=single-node
- "ES_JAVA_OPTS=-Xmx512m -Xms512m"

But pipelines keep saying to use a different format.

 Screenshot from 2019-08-21 15-54-31.png

However nothing is mentioned about it here

https://confluence.atlassian.com/bitbucket/configure-bitbucket-pipelines-yml-792298910.html#Configurebitbucket-pipelines.yml-ci_services

Or

https://confluence.atlassian.com/bitbucket/use-services-and-databases-in-bitbucket-pipelines-874786688.html

 

So basically if docker image uses not underscore but period in env variables there is no way around it in bitbucket pipelines:)

You find a way around this?

@Nathan_Rzepecki well, solution is to create your own image, e.g.:

FROM elasticsearch:7.3.0

RUN chown -R :elasticsearch /usr/share/elasticsearch/data
ENV discovery.type single-node


EXPOSE 9200

And use it in pipeline yml as: 

# This is a sample build configuration for PHP.
# Check our guides at https://confluence.atlassian.com/x/e8YWN for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: php:7.3-cli-alpine

pipelines:
default:
- step:
services:
- elasticsearch
script:
- composer install
- vendor/bin/phpunit

definitions:
services:
elasticsearch:
image: user/package-name:7.3
environment:
ES_JAVA_OPTS: '-Xms512m -Xmx512m'

I intentionally don't use cache for composer as it starts everything simultaneously. So for elastisearch it takes around 10-15 sec and composer install ~20 sec. Not sure what would be the best approach here. Other potential option is to use bash script with while.

Like # people like this

Thats what I ended up doing https://cloud.docker.com/u/lionslair/repository/docker/lionslair/elasticsearch-bitbucket-pipelines

 

I have the vars set in the docker image so no need to pass any

You can use my docker image: https://hub.docker.com/r/xiting/elasticsearch-bitbucket-pipeline

It is based on fresh Elasticsearch version: 7.3.2

Add service to your pipeline as below:

definitions:
  steps:
    - step: &run-tests
        name: Run tests
        script:
          - sleep 30 # Waiting elasticsearch. In your real pipeline you can not use it.
          - curl -XGET localhost:9250/_cat/health
        services:
          - elasticsearch
  services:
    elasticsearch:
      image: xiting/elasticsearch-bitbucket-pipeline
      variables:
        ES_JAVA_OPTS: '-Xms512m -Xmx512m'
    docker:
      memory: 2048

pipelines:
  pull-requests:
    '**':
      - step: *run-tests
Like # people like this

@Andrii Soldatenko I managed to but I had to build a custom docker image with my variables in the docker compose, like discovery.type

 

https://stackoverflow.com/questions/47286989/possible-solution-for-bitbucket-pipeline-docker-run-limitation/47304667#47304667

any updates?

Hello @Lyndon Howie did you managed to do this? I need same thing :D

Hey,

Sorry for the extra late reply, the email notification went to my junk mail and I just saw it!

I had to build a custom docker image with the variables i needed based on the public ES docker image. I see you've done something similar below.

Cheers

Suggest an answer

Log in or Sign up to answer
TAGS

Atlassian Community Events