Pipelines Build Setup taking too long

Paul Cable

I've been playing around with pipelines and have managed to get my pipeline working exactly the way I want. However it takes 5 minutes to run and looking back through the logs most of this is due to the Build Setup portion of each step.

My pipeline has 3 steps and the Build Setup takes between 30s and 90s on each step. 

I was using a private docker image but have changed this to be a public image as I understand these are cached however this hasn't affected performance.  The image is large (650MB). The repo is about 25mb.

Any ideas of what I can change to improve / diagnose the performance issues?

 

image:
  name: pjcable/temp:meteor-base-v3
#username: pjcable
  #email: paul@highspecsolutions.com
  #password: $DOCKER_HUB_PASSWORD

options:
  max-time: 5

pipelines:
  branches:
    master:

      - step:
          name: Testing
max-time: 3
caches:
            - composer
script:
            - cp .env.example .env
            - composer install
            - php artisan key:generate
            - sleep 10
            - vendor/bin/phpunit
            - php artisan serve &
            - php artisan dusk
            - kill $(jobs -p)
services:
            - mysql
            - dusk-runner

      - step:
          name: Generate production assets
caches:
            - node
script:
            - echo "<?php return [ 'number' => ${BITBUCKET_BUILD_NUMBER}];" > config/version.php
            - npm install
            - npm run production
            - git config --global user.name "Meteor Bot"
            - git config --global user.email "paul+meteor-bot@highspecsolutions.com"
            - git commit -am "Committing V${BITBUCKET_BUILD_NUMBER} production assets"
            - git tag "V${BITBUCKET_BUILD_NUMBER}"
            - git remote remove origin
            - git remote add origin https://${BB_AUTH_STRING}@bitbucket.org/meteorsg/web.git
            - git push origin "V${BITBUCKET_BUILD_NUMBER}"
      - step:
          name: Deploy to staging
deployment: staging
#trigger: manual
          script:
            - echo "Push to staging server"
            - ssh -v bitbucket@meteor.highspecsolutions.com "echo hello"
            - envoy run deploy-staging --verbose --release_tag=V${BITBUCKET_BUILD_NUMBER}

definitions:
  services:
    mysql:
      image: mysql:5.7
environment:
        MYSQL_DATABASE: 'homestead'
        MYSQL_USER: 'homestead'
        MYSQL_PASSWORD: 'secret'
        MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
    dusk-runner:
       image: pjcable/dusk-runner:latest

 

 

Danyal Iqbal

Conventional wisdom would provide resources to the environment. However, docker builds are conventionally slow. (Docker is not the rockstar, everyone thinks it is:)

You can optimize the build time using the --cache-from feature, see an example:

pipelines:
 default:
 - step:
 services:
 - docker
 script:
 - export IMAGE_NAME=your-docker-hub/your-repo:$BITBUCKET_BRANCH
 - docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
 - docker pull $IMAGE_NAME || true
 - docker build --cache-from $IMAGE_NAME -t $IMAGE_NAME .
 - docker push $IMAGE_NAME

Avoid using the pipeline docker cache, only wastes time...

check https://confluence.atlassian.com/bitbucket/caching-dependencies-895552876.html for more infos.