Trigger pipeline by tag, while pushing tags only

[Janne Costiander]

Hi,

We have set up a pipeline which is triggered by tags, and it works nicely. However, it only recognizes a tag if the tag was pushed with some commit. If you push only tags (git push --tags) with no other changes, the pipeline does not get triggered (the tag appears nicely to the latest commit)

In comparison, in Docker Cloud the automated builds are activated also with "tags only" push.

Is there a way to make this happen? I want to create the release tag when other developers have finished their work, and just release the latest commit by tagging it.

[Philip Hodder · Accepted solution]

Hi Janne,

Can you paste in an example of your configuration?

I got one behaving the way you want with the following configuration:

pipelines:
  branches:
    '*':
      - step:
          script:
            - echo "A branch push build"
  tags:
    release-*:
      - step:
          script:
            - echo "A tag triggered build"

The tags section will build on any pushed tags that start with "release". And does not require a new commit.

Let me know how you go.

Thanks,

Phil

[Janne Costiander]

Hi,

My tags-section looks like this:

tags:
  '*':
    - step:
        name: Build and test angular app
        caches:
          - node
        script:
          - npm install -g @angular/cli
          - npm install
          - ng build --target=production --no-progress
          - ng test --watch=false --no-progress
   - step:
       name: Build docker image and push to Docker hub
       script:
         - docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD
         - docker build -t project:$BITBUCKET_TAG .
         - docker push project:$BITBUCKET_TAG
         - docker tag project:$BITBUCKET_TAG project:latest
         - docker push project:latest

But gets only triggered if there is commit also.

[Janne Costiander]

Hi,

My tags-section looks like this:

tags:
 '*':
 - step:
 name: Build and test angular app
 caches:
 - node
 script:
 - npm install -g @angular/cli
 - npm install
 - ng build --target=production --no-progress
 - ng test --watch=false --no-progress
 - step:
 name: Build docker image and push to Docker hub
 script:
 - docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD
 - docker build -t project:$BITBUCKET_TAG .
 - docker push project:$BITBUCKET_TAG
 - docker tag project:$BITBUCKET_TAG project:latest
 - docker push project:latest

[Philip Hodder]

Hmmm...

I've set up a repo here testing this. Does this look like the correct behaviour to you? I've created multiple tags against a commit that already exists and pushed them with "git push --tags". Is that the same thing you're trying to do?

Repo: https://bitbucket.org/phodder/push-tag-test/addon/pipelines/home#!/

If there's no obvious differences between our repos, can you have a look at your repository settings for webhooks. Settings -> Workflows -> Webhooks -> Pipelines (https://bitbucket-pipelines.prod.public.atl-paas.net/rest/bitbucket/event/connect/onpush)

Do you see any failed webhooks?

Otherwise, can you try to add the following to your configuration?

pipelines:
  # keep the tag stuff
  default:
    - step:
        script:
          - echo "Hi"

Then push a new commit, and after see if tag pushes work afterwards?

[Janne Costiander]

Hmm, funny, but now it seems to work. Maybe there was some delay or error before, but now it seems to be working like it should.

Thanks for the help!

[Philip Hodder]

Awesome! :D

[Elias Serneels]

Just chipping in on this as I've had similar issues and was able to resolve them.

For me pipelines wouldn't trigger if I pushed multiple (more than 3) tags at the same time.

I was able to get the pipelines running by removing the tags and pushing them one by one instead of all at the same time. Maybe this is somehow related? 

[Philip Hodder]

Hi Elias,

Yes, that is expected functionality. We have a limitation on the number of branches or tags that can be pushed at once. This is to prevent users from triggering more builds than they may intend. Although, the limit is 5 tags.

Do you have a use-case where you're expected all the tags to be built? Or are you only expecting the latest one to be triggered?

Thanks,

Phil

[Elias Serneels]

Hi Philip,

Thanks for the reply. It's good to know that this is intended behaviour. I do wonder if this is actually documented somewhere? Before I landed here I did not find much information regarding this limitation which is also why I initially saw this as a bug rather than a feature.

To answer your question; our use case is that we have multiple versions of the same resource that we need to deploy to multiple instances. We don't have the luxury to just push updates out to every resource so to allow for greater flexibility about which resource we actually update we have them set up so each resource is linked to one tag. This allows us to update each resource individually by setting the corresponding tag on a commit. This might not be an ideal situation but it works.

When I previously encountered the tag limit we had to update some core feature which applied to a number of (but not all) of the different versions, which is why I tagged the same commit multiple times, once for each applicable version.

I'm aware that this is a very specific use case and that there are probably better ways of managing it but it has worked for us so far.

Hope this provides some insight,

Elias

[Philip Hodder]

Thanks for letting me know. I checked and I couldn't find any documentation either. We implemented the limit a few years ago, so it's possible it got lost in a rewrite of a page. I'll chase up getting it re-added to the docs.

Interesting use case. I haven't encounter anyone else with a similar one, but I'll pass this onto our PMs just in case they have. Are there any patterns with your deployments? Or is the deployment order of the versions fairly arbitrary?

I'm not sure if there's anything more native that Pipelines can provide for you, that you haven't already considered. However, we have a parameterised build feature request which might be of interest to you. It's for manually triggered builds, so it might not fit your needs for automated pipelines. But it might be of interest to you. :)

https://bitbucket.org/site/master/issues/13737/parameterized-builds

[Philip Hodder]

Hi @Elias Serneels,

I had a chat with a few team members:

• We have the reference (tag or branches) limitations documented here in the 'Push Limits' section. It's not too clear (or Google-able) though, so we'll get that updated later.
• I chat with one of the PMs and they had seen a similar use case! The other user was also using Git tags to mark which resources they wanted to deploy to. However, they didn't use an individual tag for each resource, but instead listed them all in a single tag and then extract the values out from the BITBUCKET_TAG variable. So instead of using three tags 'resource1-<postfix>', 'resource2-<postfix>' and 'resource3-<postfix>' you could use 'resource1-resource-3-<postfix>' to deploy to to Resource 1 and 3. So that you're pushing one tag instead of a bunch at once. She also recommended that you follow the Parameterised Builds feature request.

Thanks,

Phil

[Elias Serneels]

Hi Philip,

I did come across that page once but I don't think I associated "tags" with "references" in my mind, that one's on me!

Your proposal about extracting the BITBUCKET_TAG variable and using that to determine which resources we should target is a good one. We've done something similar in another project so it might be a good idea to revisit that knowing about the limits of tags.

The parameterised builds feature sounds like it could have a lot of potential and would solve some DRY issues that pipelines has had up until now, looking forward to it.

Thanks for the help and feedback Philip, appreciate it.

Elias

[Gabriel Labelle]

Oh this explains everything lol.

We use lerna to manage our monorepo that contains about 20 packages and it pushes all tags at once when bumping package versions.

Only 2 of these packages are apps that should gets deployed when their version gets updated.

We wanted to use something like this

tags:

  '@workjam/web-next@*':

    - step: *run-preparation-and-build

    - parallel:

      - step: *run-tests

    - step: *build-and-push-to-gcp-registry

So basically that pipeline can only match a single tag at the time since there is only one package begining with @workjam/web-next@*.

The limit of 5 should be applied to the tags that actually match and would trigger a pipeline. What do you think?

Or maybe there is a way to increase that limit @Philip Hodder ?