How to use JIRA/STASH and Team City for automated Pull/Merge Requests?

[Phill Fox]

We are looking at using Stash as our Git server alongside JIRA and Team City.

One of our requirements we would like to fill is the automatic building of Pull/Merge Requests. Ideally, we would like to be able to:

1. Fork a repository
2. Update the forked repository
3. Issue a PR from the fork
4. Have the CI server be notified of the PR and build the merged result of the PR & run tests etc
5. Have the CI server notify Stash of the success of the build
6. If any updates are made on the PR, they are rebuilt and the status updated

At present we are using TeamCity as the CI server. I’ve not seen anything that suggests that Stash/JIRA has this functionality with TeamCity out-of-the-box. Is that correct?

Looking for info on adapting Stash/JIRA to work this way this implies that we would need to go beyond just having webhooks and develop some plug-ins on both side to notify TeamCity that a PR has been issued and has passed Stash’s merge, and then one to notify Stash about the build result from TeamCity. This suggested solution makes use of the internal refs that Stash uses for merging PRs and have them monitored by TeamCity. Is this an acceptable band-aid solution for the problem?

[Balázs Szakmáry]

First of all, instead of building pull requests, you should probably build feature branches instead (with or without pull requests). The refs for pull requests have been promised to be deprecated for a while.

If you do this, for point 4, you will simply need to define the appropriate VCS root in TeamCity. (And I would recommend switching on the "Requires a minimum of N successful builds" pull request setting, to prevent bad code from being merged.)

For point 5, there is a plugin to TeamCity. We use this one, works very nicely. (It adds a new item to the build features list that you can add to individual builds as desired.) Recommendation: Switch on the "Mark cancelled builds as failed" and the "Only show latest build status for each commit" options.

[Mark Turner]

Thanks for the answer Balazs. The description of the flow was simplified to highlight the flow desired. All of the branches will be tracked and built as is common. This obviously shows that the branches are ok etc etc However, the flow I've found very useful (and used on Github projects) is to also build the PRs when they come in as a gate above the normal clearance to merge that Github or Stash gives you. As you likely know, the ability to merge cleanly does not tell me the code can build or pass tests. Now, we could argue that if we keep our features small and frequently pull in any changes in the master, then this shouldn't be something we see. From experience, this is usually the case, however I've seen/worked in places where it was quite common to miss something like this and the result was a nicely merged but broken master (be it build or tests). If you were then following a model that encouraged lots of branching off master, you'd suddenly be in trouble. I'm very keen to avoid this possibility. What I've seen when looking for answers on this is that some people have been using the ref pointing to the merge resulting from the PR but this seems to be unreliable and undocumented. An alternative answer was suggesting that a plug in could wait for the async operation of creating the merge to happen and then fire off a build. The former sounds a bit risky and the latter a lot of heavy lifting (and I'm not sure what it would fully involve). I'm surprised that this isn't something that is more often attempted as it's (at least for me) common workflow with GitHub. As a test I've checked out how to set this up using GitHub and AppVeyor and had an answer pretty quickly. Perhaps this isn't as common as I thought?

[Balázs Szakmáry]

If all the builds (including unit tests and other checks) are green in the feature branch, the chance that it will break the target branch after the PR is merged is very slim (but non-zero). We have more than 50 devs working on a large codebase and it happens very rarely that someone manages to break the master branch by merging a pull request that Stash allowed to merge (ie. all builds were green in the feature branch before the merge). All in all, I think this is a good trade-off of minimising risk and cost. If you want to implement something stricter than this, you could have a development branch where the feature branches are merged (and then built by TeamCity again) and a release branch (e.g. master) where nobody has the right to push directly. Every time when the develop branch is green (or more rarely), you merge it into the production branch. You can do this either by hand (having a gatekeeper or release engineer) or by writing a little plug-in for it. (recommended reading: https://confluence.atlassian.com/display/STASH/Using+branches+in+Stash and https://confluence.atlassian.com/display/STASH/Automatic+branch+merging )

[Mark Turner]

Aye, I've thought about that kind of integration approach but it isn't as slick. I can work around not having this feature using either integration branches or looking for a feature branch being updated etc, it's just being used to having builds on PRs from working onGithub projects, it is a very nice to have. The last teams I worked on that had problems was only 15 or so spread over a couple of teams and it did become a pain when someone managed to break the master and cause issues for everyone. I'd totally agree that PR builds is not a fix to this kind of 'attention to process' but it is a very useful warning. I could get to a similar place by manually pulling down and merging the feature to master and check locally but load soon falls into the 'stuff that gets forgotten' category. I think, overall, it's sounding like what I'd like isn't something that we will get to with Stash. Thanks for the links. I'll give them another read :)

[Balázs Szakmáry]

About breaking master: It takes 2 minutes to revert the merge commit that broke master and them master is back to green and the offending feature can be corrected without affecting everybody else. (I know that this is a reactive and thus not very elegant solution, but it makes the above, relatively low-overhead workflow much more usable.)

[Mark Turner]

Aye, it is quick to revert but then we can apply the same process to anything and assume we can just fix later it if there are issues. That I could see that there is an issue ahead of time and be proactive to fix it I think would be preferable. I've been playing with Stash to find a solution. Building the PR is ok but the issue of getting this back to Stash is now the problem. I've been looking at if it's possible to update a task or add TC as a reviewer and approve the PR on a successful build. Atm the task route seems to have an issue that the task needs to be attached to a comment on the PR. I could create a dummy on but that seems annoying, The reviewer route looked more promising but I've not found a way to update the approved status using the REST API. I did think the PUT operation to the request resource would work but its not updating it.

[Balázs Szakmáry]

Theoretically, this: https://developer.atlassian.com/static/rest/stash/3.5.0/stash-rest.html#idp1085360 (/rest/api/1.0/projects/{projectKey}/repos/{repositorySlug}/pull-requests/{pullRequestId}/approve; POST) should approve the PR

[Mark Turner]

I came across that and read it as approving the PR as merging it. Looks like I got that wrong. I think that will do what I'm thinking of. I think this way you could at least see that the build system has run it and approved it or not. I'll give it a test. I think there will be some more work to highlight this in the UI but it's a good start :)

[Balázs Szakmáry]

Semi-random thought: If your build happens to be fast (<1min), you can also implement this as a merge check https://developer.atlassian.com/stash/docs/latest/tutorials-and-examples/controlling-when-pull-requests-can-be-merged.html

[Mark Turner]

Aye, I liked the look of that. I think that it should try in with verifying a known user (TeamCity in this case) had approved the merge. I've gotten a manual experiment working to demo the flow from Stash to TC and back so my next task is to build a TC plug in to send the approval status. Once that is good I think I'll take another look at the Stash side

[Balázs Szakmáry]

Check out the code of Mendhak's plugin here: https://github.com/mendhak/teamcity-stash