Docker-compose failing on running a pipeline test

[Kuldeep Pisda]

I have a `bitbucket-pipeline.yml` as below

image: python:3.8

options:
  docker: true

pipelines:
  branches:
    ev:
      - step:
          name: Test
          deployment: staging
          services:
            - docker
          script:
            - cp devops/test/.env.test .env
            - export DOCKER_COMPOSE_VERSION=2.0.0
            - export DOCKER_COMPOSE_URL=https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-$(uname -s)-$(uname -m)
            - curl -L $DOCKER_COMPOSE_URL > docker-compose
            - chmod +x docker-compose
            - mv docker-compose /usr/local/bin
            - apt-get install unzip
            - docker-compose -f docker-compose-test.yml up --build --force-recreate --abort-on-container-exit
            - zip application.zip .
            - pipe: atlassian/aws-elasticbeanstalk-deploy:0.5.0
              variables:
                AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
                AWS_DEFAULT_REGION: $REGION
                APPLICATION_NAME: "staging-server"
                ENVIRONMENT_NAME: 'staging'
                ZIP_FILE: "application.zip"

My `docker-compose-test.yml` file

version: "3.7"
services:
  test-database:
    container_name: test-database
    image: postgres:12
    ports:
      - "5432:5432"
    env_file:
      - .env

  test-redis:
    container_name: test-redis
    hostname: redis
    image: redis:6.0.3
    restart: always
    command: ["redis-server", "--bind", "redis", "--port", "6379"]

  test-api:
    container_name: test-api
    build:
      context: .
      dockerfile: Dockerfile
    command: "pytest"
    env_file:
      - .env
    environment:
      - POSTGRES_HOST=test-database
    depends_on:
      - test-database
      - test-redis

 My `devops/test/env.test` file

POSTGRES_DB=test
POSTGRES_USER=test
POSTGRES_PASSWORD=test
POSTGRES_PORT=5432
POSTGRES_HOST=test-database
DEBUG=true
SECRET=SOME_SECRET_KEY
DJANGO_SETTINGS_MODULE=app.config.local
ALLOWED_HOSTS=*
CORS_ALLOWED_ORIGINS=http://localhost
CORS_ALLOW_ALL_ORIGINS=True
APP_URL=https://example.com
BROKER_URL=redis://redis:6379/0

 My `devops/test/Dockerfile` file

FROM python:3.9-slim-buster

ARG APP_USER=app
RUN groupadd -r ${APP_USER} && useradd --no-log-init -r -g ${APP_USER} ${APP_USER}

ENV POETRY_VERSION 1.1.6

# Install packages needed to run your application (not build deps):
#   mime-support -- for mime types when serving static files
#   postgresql-client -- for running database commands
# We need to recreate the /usr/share/man/man{1..8} directories first because
# they were clobbered by a parent image.
RUN set -ex \
    && RUN_DEPS=" \
    libpcre3 \
    mime-support \
    postgresql-client \
    libcurl4-openssl-dev \
    libssl-dev \
    " \
    && seq 1 8 | xargs -I{} mkdir -p /usr/share/man/man{} \
    && apt-get update && apt-get install -y --no-install-recommends $RUN_DEPS \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt /tmp/requirements.txt

RUN set -ex \
    && BUILD_DEPS=" \
    build-essential \
    libpcre3-dev \
    libpq-dev \
    " \
    && apt-get update && apt-get install -y --no-install-recommends $BUILD_DEPS \
    \
    && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false $BUILD_DEPS \
    && rm -rf /var/lib/apt/lists/*

RUN pip install --upgrade pip
RUN pip install "poetry==$POETRY_VERSION"

RUN mkdir /api/
WORKDIR /api/

COPY pyproject.toml poetry.lock ./
RUN poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi

COPY . /api/

# Add any static environment variables needed by Django or your settings file here:
ENV DJANGO_SETTINGS_MODULE=app.config.local

# Call collectstatic (customize the following line with the minimal environment variables needed for manage.py to run):
# RUN DATABASE_URL='' python manage.py collectstatic --noinput

# Tell uWSGI where to find your wsgi file (change this):
ENV UWSGI_WSGI_FILE=app/wsgi.py

# Base uWSGI configuration (you shouldn't need to change these):
ENV UWSGI_HTTP=:5000 UWSGI_MASTER=1 UWSGI_HTTP_AUTO_CHUNKED=1 UWSGI_HTTP_KEEPALIVE=1 UWSGI_LAZY_APPS=1 UWSGI_WSGI_ENV_BEHAVIOR=holy

# Number of uWSGI workers and threads per worker (customize as needed):
ENV UWSGI_WORKERS=2 UWSGI_THREADS=4

# uWSGI static file serving configuration (customize or comment out if not needed):
ENV UWSGI_STATIC_MAP="/static/=/api/static/" UWSGI_STATIC_EXPIRES_URI="/static/.*\.[a-f0-9]{12,}\.(css|js|png|jpg|jpeg|gif|ico|woff|ttf|otf|svg|scss|map|txt) 315360000"

# Deny invalid hosts before they get to Django (uncomment and change to your hostname(s)):
# ENV UWSGI_ROUTE_HOST="^(?!localhost:8000$) break:400"

# Change to a non-root user
USER ${APP_USER}:${APP_USER}

ENTRYPOINT ["/bin/bash", "devops/test/entrypoint.sh"]

My `devops/test/ntrypoint.sh` file

#!/bin/ash

# exit on any failure with non-zero code
set -e

pytest
flake8
exit 0

So it is a Django project. I want to run pytest if something is pushed/merged on the `dev` branch. The same conf is working on local. But in the pipeline, it fails saying:

django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "test-database" (X.X.X.X) and accepting
TCP/IP connections on port 5432?

 If I run this command

 docker-compose -f docker-compose-test.yml up --build --force-recreate --abort-on-container-exit

It runs everything as expected. But on the pipeline, it is falling. Is it something I am doing wrong?

[Mark C]

Hi @Kuldeep Pisda

Welcome to the community.

I can see that you're using the .env file to pass your environments in your services.

Would it be possible for you to try to add the variables with the below sample format for us to see if it makes a difference?

environment:
      POSTGRES_DB: ${DATABASE_NAME}
      POSTGRES_USER: ${DATABASE_USER}
      POSTGRES_PASSWORD: ${DATABASE_PASSWORD}

Also, when you tested your Pipelines build locally, may I kindly ask if you've tested it in a Docker container?
If not yet, would you be able to try to run your Pipelines build locally in a Docker container?

Ideally, if it runs locally in a Docker container, it should also run on Bitbucket Pipelines.

Let me know how it goes.

Regards,
Mark C