Scheduled pipeline

[Richard Tremblay]

Hi,

I'm looking for a way to execute pipeline at a specific time each day but only if there's something new in a specific branch, if there's nothing new, don't run. And at the same time, to only run at that time, do not run when something is push in that branch.

The idea is to have a way to automate code pushed to a specific branch to one of our server automatically but only at a specific time, i.e. only deploy in our server at 1am.

Any idea are welcome.

[ktomk]

You can create a custom pipeline and time it (schedule it).

A custom pipeline is immune to pushes and can be scheduled to be executed on a specific branch.

You would need to deal in the pipeline to do the change detection.

E.g. if you have a deployment package and store it on an object store like AWS S3, and the object name contains the revision, you can easily check whether or not the revision has been build already. Only if not, build it, deploy it and store it on S3.

S3 is only exemplary here. Bitbucket has downloads as well, it might work as well (e.g. you don't have or want an(other) AWS account for only having S3. The REST API is fairly straight forward for it and behind the scenes it is S3 as well, you just don't need to manage it your own.

[Richard Tremblay]

Thanks for your answer.

I have figured everything except how can I do the changes detection part, this is a git repository, and I want to look for changes in the current branch the pipeline is being executed.

Is there a way to do something like

If git diff current branch vs deployment branch

- Merge branch

Else

- Do nothing

[ktomk]

@Richard Tremblay Yes, that is possible with the git command line utility.

Given the pipeline script runs on the current branch:

• when HEAD is already merged into the deployment branch, exit the pipeline script (already deployed)
• when HEAD does not merge with the deployment branch, exit the pipeline script with error (does not merge and can not be deployed)
• when HEAD does merge, merge it fast-forward into the deployment branch and push it upstream (origin) to trigger the deployment pipeline

Would this scenario describe your use-case requirements?

If so you might be looking for the git commands. I have no concrete example at hand but did this already so I know it's perfectly possible.

Some hints on git commands, I'm pretty sure there are others, so this is merely informative:

• `git fetch origin <deployment-branch>` will ensure the merge target from origin is available remote reference `origin/<deployment-branch>` in the local clone (within the pipeline)
• `git merge-base origin/<deployment-branch> HEAD` will either
  1. exit non-zero if there is no merge-base (this *should* not happen, so it's fine as it paints the pipeline red then)
  2. gives the git revision hash otherwise (if it's the same as HEAD it means that HEAD is merged already)
• `git rev-parse --verify HEAD` gives you the actual commit from the pipeline, so this is HEAD as the revision head.
• `git merge ...`  and `git push ...` I assume you know already and can handle them.

Some caveat:

In Bitbucket Pipelines the HEAD revision is available as well as a Bitbucket Pipelines variable (`$BITBUCKET_COMMIT`) which kicked of the pipeline. I have found it unstable in the meaning that it was sometimes in the short form and sometimes in the long form.

This is cumbersome for comparing, e.g. the short one might reference the same revision but does not work in a string comparison when it's against the long one.

The `git rev-parse --verify <ref>` command does help here to turn it into the full (long) revision hash. That command will also fail if `<ref>` can not be verified.

Summing it up:

Finding about differences is making use of the version control system, here git. This requires understanding git a bit more and also how to obtain all needed revisions to make the decisions you need to make. 

A Bitbucket pipelines script is effectively a shell script, so it's easy to call the diverse git commands and also do decisions based on string comparison and similar tests.

One example for such a test:

test ! -z "$(git merge-base deploy HEAD)" -a ! "$(git merge-base deploy HEAD)" = "$(git rev-parse --verify HEAD)"

test for 1.) there is a merge base AND 2.) the merge base is not at HEAD

See `man test`, `man git-merge` and `man git-rev-parse` for detailed information.

If you've got a shell at hand, you can write a shell script for a first test as long as the git utility is available on the command-line.

To test further with pipeline integration you can consider to simulate pipelines with more scripts and docker or a local Bitbucket Pipelines runner which does the heavy lifting (intended as an out-of-the-box experience, disclosure: I'm the author of it, your mileage may vary).

Hope this helps you to further improve your pipeline and enables you to reach your goals.

[Richard Tremblay]

That is exactly what I want, thank you again!

I'll do a few test and let you know how it turns out.

[Richard Tremblay]

Hello there.

I can't get the 'git fetch origin Server-Live' to work, I can only see and access the Branch Server-Approved (which is the branch pipeline is currently executing).

I searched the web, tried many variation, but can't find why it's not working, any idea?

[ktomk]

Check the remote is correct and fetch from it.

/E: @Richard Tremblay Are you limiting the clone depth by chance? (Or Bitbucket might do that with the default 500). The git clone command is shown in the "Build" section at the very beginning of the Pipeline log. Can you post that section from a Pipeline log where you run into that problem?

[Richard Tremblay]

+ umask 000

+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="Server-Approved" --depth 50 https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...

+ git reset --hard 61dfaa82326140e58fd5c545f705b940613335b7
HEAD is now at 61dfaa8 D46

+ git config user.name bitbucket-pipelines

+ git config user.email commits-noreply@bitbucket.org

+ git config push.default current

+ git config http.${BITBUCKET_GIT_HTTP_ORIGIN}.proxy http://localhost:29418/

+ git remote set-url origin http://bitbucket.org/$BITBUCKET_REPO_FULL_NAME

+ git reflog expire --expire=all --all

+ echo ".bitbucket/pipelines/generated" >> .git/info/exclude

+ chmod 777 $BUILD_DIR

Images used:
build : docker.io/atlassian/default-image@sha256:689e2c63e20a48e0a4d31156adcf32b4474dc32b50ab05abe3682b39fb9767a8
+ echo "Compare branches"
Compare branches

+ git remote
origin

+ git fetch origin Server-Live
From http://bitbucket.org/vgcweb/git.visiongroupcanada
* branch Server-Live -> FETCH_HEAD

+ git branch
* Server-Approved

+ git branch -r
origin/Server-Approved

+ git merge-base Server-Live HEAD
fatal: Not a valid object name Server-Live
Searching for test report files in directories named [test-results, failsafe-reports, test-reports, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

[ktomk]

Yes, that's much better.

Server-Live is not a local branch, the remote prefix is needed:

git merge base origin/Server-Live HEAD

So instead of "Server-Live" it is "origin/Server-Live" (or even "remotes/origin/Server-Live").

Alternatively the fetch can be done to the same local name:

git fetch origin Server-Live:Server-Live

[Richard Tremblay]

After doing

git fetch origin Server-Live:Server-Live

I can now see both branch:

+ git branch

* Server-Approved

Server-Live

Now my understanding of git is very basic, so I tried to run the 'test' command and add the && git checkout/merge at the end of the line and ran in all kind of error, so I changed the code to this:

 - if [ ! -z "$(git merge-base Server-Live HEAD)" -a ! "$(git merge-base Server-Live HEAD)" = "$(git rev-parse --verify HEAD)" ]; 
then 
echo 'Yes it is different'
git checkout Server-Live && git merge Server-Approved && git push origin Server-Live:Server-Live;
else
echo 'No it is identical'
fi

And for some reason now it always goes in the 'else', if I put the checkout/merge/push in the else I get the message that everything is up-to-date, but it is not.
I must be missing one step here, any idea?

[ktomk]

Well the else part is if it is identical.

Are you one or multiple commits ahead?

[Richard Tremblay]

I have such a hard time to post an answer here... 9 time out of 10 the page reload but my answer is not posted...

[Richard Tremblay]

I added the 'else' to trace what was going on.

In the Server-Approved I have many commit, all my pipeline.yml test.

In the Server-Live I have 6 commit, different test I did over the last few months.

And if I simply do these command:

- git checkout Server-Live
- git merge Server-Approved

I get this error

+ git merge Server-ApprovedAuto-merging bitbucket-pipelines.ymlCONFLICT (add/add): Merge conflict in bitbucket-pipelines.ymlAutomatic merge failed; fix conflicts and then commit the result.

So should I completely delete the repo and start fresh?

[Richard Tremblay]

ok there was a conflict somewhere, I merged the Server-Approved into the Server-Live locally in my computer and pushed the changes and now the script was able to run.

When the merge/push was completed, the pipeline started on the Server-Live and deployed all the changes!

Thank you so much for all the great help!!

[Richard Tremblay]

Actually I have one last issue, maybe I misunderstood something.

At the begining you mentioned that a custom pipeline is immune to pushes, my understanding was that once a pipeline step is set to be launched by the scheduler it should not auto-execute when something is pushed.

I just did a test and it still run on pushes.

What I need is to not run on pushes, but only when scheduled, did I miss something?

[ktomk]

Hi Richard, good to see things are moving towards your favour.

I also did see the issue with posting answers recently that the post is dropped. Using the clipboard to backup before pressing the button (select all, cut, undo and then click) is how I work around it - after loosing lots of posts.

Good to see you have not deleted the repository. This can be tempting but leaves one only puzzled afterwards. Especially in your case this was merely a merge conflict (only). Which is good to see it breaks the pipeline as it should. Solve it the way you did already, by resolving the merge conflicts in the branch to merge first. Looks good to me, good choice.

About your most recent question:

my understanding was that once a pipeline step is set to be launched by the scheduler it should not auto-execute when something is pushed

I just did a test and it still run on pushes.

What I need is to not run on pushes, but only when scheduled, did I miss something?

This is merely in the details and perhaps an impreciseness of mine at that level. If you schedule a Custom pipeline, it can only be execute either manually or with the scheduler and not by pushes (not the pipeline type "custom").

(Default/) Branch/Tag/Bookmark pipelines are triggered by pushes.

Pull-Request pipelines are triggered by creating a pull-request (therefore require a push, but aren't auto-triggered unless the pull-request already exists).

So if you want to have a schedule only pipeline, make it a custom pipeline.

As it's possible to schedule any kind of pipeline I assume you have been scheduling a pipeline of the types that are triggered on push (default, branch/tag/bookmark or even pull-request).

Changing it into a custom type pipeline should do it for your needs.

If you face duplication and you don't want it, you can consider to make use of YAML anchor / aliases in the bitbucket-pipelines.yml file. This is totally optional, I just mention it in case you face it and you're looking for some options when authoring the YAML.

References:

• YAML anchors (Atlassian Bitbucket Pipelines documentation)
• 6.9.2. Node Anchors (YAML 1.2 spec)
• 7.1. Alias Nodes (YAML 1.2 spec)

[Richard Tremblay]

I did miss that important detail!

Looks good now, everything is working as expected.

Thanks again.

[cyberitch]

Just when I thought it was all perfect...

When integrating this process in our real repo I realize that we have a permissions issue.

The branch Server-Live is restricted to the Administrator group, to avoid all Developers from pushing in that branch, so it also prevent the custom pipeline from merging into that branch.

I don't see an option to allow pipeline in the permission CI of BitBucket, is there a way to give pipeline permissions?

Or any other solution to prevent everyone to push in the branch but allow pipeline to do it?

[ktomk]

Is there a way to give pipeline permissions? [...] to prevent everyone to push in the branch but allow pipeline to do it?

Good question. I never needed to find that out honestly, but even if awkward, I assume this should technically be possible. E.g. with an API key or Git user that has the rights and comes into use within that pipeline. Effectively it will be that everyone with write access to the project repository is able to obtain these credentials. Just saying.

Which brings me to some perhaps crazy idea, I never did it: Is it possible to make it a deployment step in the custom pipeline that is being scheduled? Benefit of it would be a more fine-grained control of the variables in use and those would be actually limited to that custom pipeline if you still have a deployment environment to use (there were three last time I looked). Anyways, it's just a very little thought about idea.

I'm currently running only cheap on Atlassian Bitbucket so have to look into it and if it's possible to suggest more than half-baked ideas.

It's perhaps good enough for a start in your case to poc it, e.g.

• Merge via the REST API (instead of local git utility) after successful local checks
• Dedicated git user account in the custom pipeline

Leaves security/policy out of the context, this is purely technically. Might be still already fitting for your scenario, e.g. if the Bitbucket branch permissions should first of all and only prevent git pushes in error.

[Richard Tremblay]

I started to look into creating a Pipeline user and give it the permission, but then I found this:

https://support.atlassian.com/bitbucket-cloud/docs/push-back-to-your-repository/

And it seems the OAuth should be the way to go, but can't get it to work, I always get this error:

fatal: unable to access 'https://x-token-auth:@bitbucket.org/vgcweb/git.visiongroupcanada/': gnutls_handshake() failed: Handshake failed

I'll do a few more test, and I'll try to message Atlassian support, to see what they suggest.

[ktomk]

@Richard Tremblay Please see here: https://community.atlassian.com/t5/Bitbucket-questions/fatal-unable-to-access-https-bitbucket-org-gnutls-handshake/qaq-p/1468075 - there is more if you search for gnutls_handshake() failed.

Are you using the default image (from Atlassian)? If so I remember someone solved this by switching to the second version of it.

[Richard Tremblay]

You were right, switching to the second version has fixed the handshake issue, but now Im running in another issue:

+ git fetch origin Server-Live:Server-Live

remote: Invalid username or password

fatal: Authentication failed for 'https://x-token-auth:@bitbucket.org/vgcweb/git.visiongroupcanada/'

Looks like the access_token is not being set, I emailed support, I have the feeling that I missing a step, or lacking some understanding of the OAuth concept.