Building Pull Requests (from Stash) on Bamboo

[Daniel Sobral]

I want to build pull requests (that is, the branch which is being requested to be merged) when they are created on Bamboo. The repository is Stash, so the pull requests would come from personal forks to a known project/repository. I can't find any way to configure such a build from Bamboo, nor trigger it from Stash.

[Marek Parfianowicz · Accepted solution]

Bamboo 6.0 brings the pull request support for Bitbucket Server, see the release notes:

https://confluence.atlassian.com/bamboo/bamboo-6-0-release-notes-894743804.html 

[I]

Its a good news that Bamboo 6.0 supports the PR builds now. 

Wonder there is a example guide to setup the plan with all configurations needed in Bamboo? For me its a nightmare setting up a CI build on PR for bamboo plans.

If you could show us all the settings/configurations required to setup exact steps that would be really appreciated. 

Thanks.

[Daniel Sobral · Accepted solution]

There's presently no way of doing that without writing all the pieces yourself.

On Bamboo, you'll need the following configurations:

* Adding a shell script step to fetch the remote pull request references; they are considered internal use only, but that's the only way right now. More information in this question. I use the following:

git remote | grep -q origin || git remote add origin "https://${bamboo.user}:${bamboo.password}@${bamboo.repo.url}"
git fetch origin '+refs/pull-requests/*/merge-clean:refs/remotes/origin/pr/*'
git fetch origin '+refs/pull-requests/*/from:refs/remotes/origin/pr-from/*'

* Checkout the desired pull request; that means adding a plan variable that can be used on the build tasks. For example, a variable "prnumber" would be used like this:

git checkout pr/${bamboo.prnumber}

* Update build status on Stash using the REST API. This article has everything that's needed, but note that "merge-clean" is what is going to be merged (and, thus, what one needs to build), but the build status on the pull request status on Stash only shows up for the source commit -- that's why I bring the two references above.

* Since there's no variable in Bamboo to indicate whether a build has succeeded or not, I use three steps: mark a build in progress after checkout, mark a build as succeeded as the last task, and then, on a final task (which is always run, whether the previous tasks succeed or not), I mark the build as failed if I don't find a file left behind indicating the build has succeeded (I use a file I create to store the json I send to Stash).

On the Stash side, things are more difficult because you have to listen to pull request events -- the existing hooks don't cover it. This question covers most of what's needed. Because you are not using the hook mechanism, you won't have access to the easy per-repository configuration it provides, so you'll have to write it yourself.

One last note: you use the "queue" REST interface on Bamboo to trigger the build from Stash. Look at your Bamboo's version API documentation here to see the details (a generic REST description found elsewhere doesn't go into that).

[Ahn Kiwook]

I dont' know how I can add the script you uploaded... Can you help me?

[antoine1fr]

@Daniel Sobral Your answer is almost 2 years old. Do you know if the situation has changed since?

[Robert Dailey]

An easier solution would be to write a hook that creates a branch under `refs/heads` for every ref it finds under the PR ref.

[shaulbehr]

@Daniel Sobral Any updates on this?

[Andrea]

Any news on this?

[antoine1fr]

@Andrea I found a JIRA issue on this.¹ I've just upvoted it and I suggest you do the same. 

1. https://jira.atlassian.com/browse/BSERV-5112

[Andrea]

@Antoine Luciani thanks! done

[martinda14]

Any update? Can it be done without writing all the pieces by hand?

[Felix Köhler]

Is there any porgess on this topic ?

[WillemA]

This plug-in will help you with it: https://marketplace.atlassian.com/plugins/se.bjurr.prnfs.pull-request-notifier-for-stash/server/overview

[Dobes Vandermeer]

Could you provide more information on how that plugin helps? Do you use it to trigger a Bamboo manual build or something?

[WillemA]

Yes, we use it to start a build of a build-plan that is able to build for every repository and branch we have. So this plugin call the rest api: <bamboo server>/rest/api/latest/queue/PR-BLD With the plugin parameters: bamboo.PULL_REQUEST_ID=${PULL_REQUEST_ID}&bamboo.PULL_REQUEST_TO_SSH_CLONE_URL=${PULL_REQUEST_TO_SSH_CLONE_URL}&bamboo.PULL_REQUEST_TO_REPO_NAME=${PULL_REQUEST_TO_REPO_NAME}&bamboo.PULL_REQUEST_TO_BRANCH=${PULL_REQUEST_TO_BRANCH}&bamboo.PULL_REQUEST_TO_REPO_PROJECT_KEY=${PULL_REQUEST_TO_REPO_PROJECT_KEY}&bamboo.PULL_REQUEST_URL=${PULL_REQUEST_URL}&bamboo.PULL_REQUEST_VERSION=${PULL_REQUEST_VERSION}&bamboo.REPOSITORYSLUG=${PULL_REQUEST_AUTHOR_SLUG}&os_authType=basic&executeAllStages=true

[Timothy_Harris]

But how do you get at the variables.e.g branch, which are passed as part of the hook?

[Laszlo Szabo]

Maybe copy the way TeamCity implement this feature (available since 2013):

https://blog.jetbrains.com/teamcity/2013/02/automatically-building-pull-requests-from-github-with-teamcity/

[Martin Faust]

I believe this to be a rather standard feature. Are there any news?

[Daniel Sobral]

There's presently no way of doing that without writing all the pieces yourself.

On Bamboo, you'll need the following configurations:

* Adding a shell script step to fetch the remote pull request references; they are considered internal use only, but that's the only way right now. More information in this question. I use the following:

git remote | grep -q origin || git remote add origin "https://${bamboo.user}:${bamboo.password}@${bambo.repo.url}"
git fetch origin '+refs/pull-requests/*/merge-clean:refs/remotes/origin/pr/*'
git fetch origin '+refs/pull-requests/*/from:refs/remotes/origin/pr-from/*'

* Checkout the desired pull request; that means adding a plan variable that can be used on the build tasks. For example, a variable "prnumber" would be used like this:

git checkout pr/${bamboo.prnumber}

* Update build status on Stash using the REST API. This article has everything that's needed, but note that "merge-clean" is what is going to be merged (and, thus, what one needs to build), but the build status on the pull request status on Stash only shows up for the source commit -- that's why I bring the two references above.

* Since there's no variable in Bamboo to indicate whether a build has succeeded or not, I use three steps: mark a build in progress after checkout, mark a build as succeeded as the last task, and then, on a final task (which is always run, whether the previous tasks succeed or not), I mark the build as failed if I don't find a file left behind indicating the build has succeeded (I use a file I create to store the json I send to Stash).

On the Stash side, things are more difficult because you have to listen to pull request events -- the existing hooks don't cover it. This question covers most of what's needed. Because you are not using the hook mechanism, you won't have access to the easy per-repository configuration it provides, so you'll have to write it yourself.

One last note: you use the "queue" REST interface on Bamboo to trigger the build from Stash. Look at your Bamboo's version API documentation here to see the details (a generic REST description found elsewhere doesn't go into that).

[ThomasA]

Hi Daniel,

Have you seen Integrating Bamboo with Stash and this blog post for branch builds?

I think they should cover setting up both sides of PR builds.

Regards,

Tom.

[Kris Nuttycombe]

We've been looking through the docs fairly extensively; the problem is that we use a fork-branch-and-PR model, so the branch from which the merge is being request is not actually a branch of the repository that Bamboo is building; or at least its ref is not in refs/heads, which is what I suspect the Bamboo git plugin is working from to get the names of branches that it might need to build. So, using plan branches does not actually work.

And, while I thought I'd found a workaround, it didn't pan out. What we essentially need is for the bamboo plan branches integration to be able to read from refs/pull-requests/\d+/from as well as refs/heads

[Kris Nuttycombe]

To additionally clarify, here's the workflow we need to support:

(1) Joe creates a new fork of the upstream repository using Stash

(2) Joe creates a local clone of his fork

(3) Joe pushes a feature branch from his local clone to his fork

(4) Joe uses stash to create a pull request for his feature branch, targeting the upstream/master branch

(5) Bamboo notices and checks out the "from" code of the pull request

(6) Bamboo attempts to merge upstream/master to the PR'ed code. If the merge is conflicted, it puts a nasty comment into the PR, rejects it, whatever.

(7) If the merge succeeds, Bamboo attempts to build and run tests. If the build fails, it puts in it's cruel commentary and rejects the PR.

(8) If the build succeeds, Bamboo tells Stash that it did so. Celebration! The PR is one step closer to being merged.

[cofarrell]

Hi Kris,

I've been talking to Bryan about this offline, and in the very short-term suggested using ls-remote in a script to create real branches for the private PR merge refs. He mentioned that you might be looking at the Bamboo plugin, which is definitely a much better idea. Did you have any luck with your investigation?

Our eventual goal is to support pull requests as first class citizens, and the following issue might be of interest (you may want to follow/vote for it):

https://jira.atlassian.com/browse/BAM-11205

I've been following up with the Bamboo team to see what we can do to help support this common workflow.

Cheers,

Charles

[Daniel Sobral]

I have gone a long way, mostly with lots of duct tape and help from other questions. On Bamboo, I have resorted to shell script steps to fetch the desired branches (the internal use-only refs/pull-requests/* stuff) and update build status through Stash REST API. On Stash, a plugin listening to pull request events is doing the job. There's still configuration to set up for the Stash plugin (per-repository build plans), however, which doesn't look simple.