How can I implement a policy in Stash to prevent check-in without an issue linked in the commit?

[Ray Sanchez]

I dont want to allow checkins without a linked issue

[Bryan Turner · Accepted solution]

Ray,

This is a feature that we haven't fully fleshed out internally yet; it's still under a lot of discussion. We'd certainly be keen to hear your thoughts on the issue.

With an SVCS like Subversion, this sort of feature is easy to implement because every commit goes to the server. However, in git, commits are made against your local repository. Stash cannot control what those commits do and do not do. The only option Stash has is to reject the push if all the new commits don't have an issue key in their message. We've built a pre-receive hook that does this.

The primary drawback there is that, when such a push is rejected, the user then has to amend their commits and rewrite history to add those issue keys. Depending on how many commits the user had made, that can be pretty painful. A better solution may be to have a local pre-commit hook which prevents commits from ever being made without an issue key but, unfortunately, Stash cannot set this up for users. There is no mechanism built into git to allow the server to setup hooks on clients.

One idea that has come up from our discussions internally has been that, instead of blocking the push, Stash allow some form of retrospective linking approach. That way, even after a commit has been pushed, it is possible to associate it with one or more issues. That would also allow for "fixing" commits where the commit message referenced the wrong issue key (something we see happen a lot in our own development on Stash itself, since it has 2 JIRA projects).

How would you like to see this handled? Does forcing your developers to rewrite their history when they forget their issue keys work for you?

Best regards,
Bryan Turner
Atlassian Stash

[Colin Goudie]

I like that retrospetive idea. At least the reporting/audit capability is there then. Although the 'truth' isn't all in your SCM then.

A GUI in Stash to download pre-commit scripts that work in conjunction may be another idea to block commits in the first place. That's something we talked about internally

[John Burns]

+1 for the retrospective linking approach idea.

[Deleted user]

I like the Retrospective Linking (RL) approach as well. (Has any progress been made along that line?) The only caveot to that is that it has no enforcment teeth to it. Consequently, in my mind, it is only a first step. So I also like Colin's idea of providing a place for pre-commit scripts.

To extend that idea, perhaps you could allow the repository creator to mark a repository with an "Enforce JIRA Integration" property. This feature would do a few things to that repository:

1. Attach the pre-receive hook.
2. Send the administrator to the RL page to clean up existing unlinked commits.
3. Insert a Stash directory into the repository with the following files:
   • An executable commit-msg hook script.
   • A README file instructing people to copy that script into their .git/hooks directory.
4. Include links in Stash pointing to documention on JIRA Integration and the setup necessary for contributing. Some places to include them:
   • A "JIRA Integrated" column in the Stash project's repository list.
   • The repository's main page.
   • Instructions on how to clone that repository.
   • A troubleshooting guide for the repository.
5. Include a control for the administrator only to temporarily suspend the pre-receive hook to help the non-compliant programmer clean up things. They could then walk the programmer through the process of using the RL tool (to clean up the existing mess) AND activating the commig-msg script (to prevent more messes) before getting off the phone.

I think these measures would give the repository administrator more confidence in creating a strictly enforced Jira Integrated project. It's not quite as clean as being able to push hook scripts to remote repositories, but it would hopefully be clean, nevertheless. It would move the burden of execution to the repository cloners, while giving them a nice way to clean up after themselves if they miss a step.

[Karan Shah]

The only option Stash has is to reject the push if all the new commits don't have an issue key in their message. We've built a pre-receive hook that does this.

Hi Bryan, this is actually all I would need. I would usually tell everyone using my git repo to ensure Jira tickets exist in all their commits and if not it's their problem to deal with anyways, as long as the stash git repo remains 'pure' (has issue ID in all commit messages).

Do you know how I can use this as you said you guys have already built it?

[David Greene]

This sort of thing is exactly what git notes are for. IMHO BitBucket/JIRA should be reworked to recognize issue numbers in notes (probably in an "issue" namespace or something similar). Issue numbers are metadata, relevant to the commit but not strictly part of it. That's what git notes implement. git notes can be retroactively updated at any time. Originally we had planned to put issue metadata in notes until we discovered that BitBucket doesn't support notes. Adding notes support would solve a number of issues.

[Colin Goudie]

I would say that would be best achieved using pre-commit hooks. I'm sure there will be some for that (if not already available)

http://blogs.atlassian.com/2013/03/stash-git-hooks-api/

[Bryan Turner]

Colin,

Because "commits" never happen, on the Stash server itself, there is no way to hook in a pre-commit hook on the server side. It would have to be manually setup in developers' clones, where the commits actually happen. The only hooks Stash can provide are pre-receive and post-receive, the two hooks provided by its current API. These are invoked when new commits (emphasis there on the plurality) are pushed to the server.

Hope this clarifies the implementation a little!
Bryan Turner
Atlassian Stash

[Ellen Feaheny [AppFusions]]

Yeah totally. Makes sense

[Ellen Feaheny [AppFusions]]

I voted UP in Colin's post above, but reflecting down and no undo. Sigh! Sorry Colin!

[Midori]

Commit Policy Plugin is a relatively new JIRA(!) add-on to verify issue keys in commit messages, among others.

It integrates nicely with Stash, although that requires a little bit of manual work on the hook script, for which we provide a step-by-step guide. We also have a public feature request to offer a smoother integration, possibly in the form of a Stash hook plugin (make sure you vote for this).

For the problem mentioned by Bryan Turner, our approach is that it's better to prevent problems than retrospectively fix them. The idea here is to offer a mechanism which would make it super easy to integrate the checks not only to Stash managed blessed repositories and forks, but also to clones (working copies located at the developer machines). That means your developers wouldn't even be able to commit to their working copies if their commit is rejected by the team's policy. If this feature created value for you, vote to this!

Important: unlike most other solutions suggested here, it does not only verify if there is an issue key formally present in the commit message (like "FOO-123"), but even matches that or those against a JIRA JQL query!
For instance, this allows to verify if the mentioned issue(s) are:

• in-progress user stories in the current sprint
• unresolved bugs or tasks targeting the next product version
• high-priority tasks assigned to team-leads (in a code freeze period)

Other than checking the mentioned issues, it can also verify:

• the committer's identity (is he in JIRA? is he in a JIRA group?) 
• the changed files (only images are submitted? is there any *.class,*.obj,*.tmp file? all *.PNG are in the /images directory?)
• the commit message (is this 10+ characters long without whitespace? does it start with a JIRA key?)

commit-policies.png

[Karl Ostmo]

Hi,

I've created an open-source Stash plugin to enforce the presence of issue references and cross-check them against Jira: https://bitbucket.org/teslamotors/jira-issue-enforcement-stash-plugin

It's also published on the Atlassian Marketplace: https://marketplace.atlassian.com/plugins/com.teslamotors.stash.hook.jira-issue-enforcer

Karl

[Ron Chan]

You know what, I just installed this and it seems to pose the same issue as described up top...the hook prevents the "push" but not "commits". Which means commits would need to be amended which we do not want.

Am I missing something?

By the way, I meant to down vote but didn't have permissions.

[Bryan Turner]

Ron, No, you're not missing anything. That's how it will work. As I noted in my answer to the original question, unlike in SVCSs like Subversion in a DVCS like Git the Stash server is not contacted when commits are made. Users do not commit directly to Stash. Their commits are made in their local repositories and then pushed to Stash later. That means no hook you install on the Stash server has any ability to control what users can or cannot commit; they can only control what users can or cannot _push_. Because of that, yes; such a hook will require that commits be amended in users' local repositories before they can be pushed. That is why some form of retrospective linking support, allowing commits that are pushed to be linked to JIRA issues after the fact without updating their commit messages, may be a better approach than trying to use hooks. Best regards, Bryan Turner Atlassian Stash

[Ron Chan]

Thanks for the clarifying response Bryan.

We'll have to figure out other ways around this dilemma.

[pyus_13]

Recently I faced issue in pushing the code not because my commits are not having the issue reference number but the other one's code have a wrong reference id which is already merged with the develop branch of us. From two days I am wondering how that branch got merged with develop, how it gor successfully pushed in repo ? still no Answer. But the main problem is whoever taking a pull from develop branch and trying to push the code stash is rejecting the push saying that "wrong issue reference number". Is there any solution for the same without disabling the plugin. Seems like a bug in plugin.

[Garret Pick]

Hi Charles,

I'd definitely be interested in trying that out. Did you upload the source as well or is the jar file the only thing in the repo?

Thank you very much,

Garret

[cofarrell]

The source is definitely uploaded (it is Bitbucket after all). Please feel free to tweak as you fit, and don't hesitate to ask any questions if you get stuck.

[Garret Pick]

Hi Charles,

The reason I ask is because at

https://bitbucket.org/cofarrell/stash-enforce-message-hook-plugin/commits/all

it says

"I have no commits, rendering me useless. Now I am sad."

and at

https://bitbucket.org/cofarrell/stash-enforce-message-hook-plugin/src

it says

"The source directory is empty."

Am I not looking in the right place?

thanks again,

Garret

[Garret Pick]

I can see it now. Thank you.

[cofarrell]

*stares dumbly at Bitbucket*

You're quite right. Doh, I didn't actually commit/push because of a bad vim plugin. My apologies - try now.

[Garret Pick]

I have a perl script that handles this:

#! /usr/bin/perl

# From http://stackoverflow.com/questions/2524101/git-how-do-we-verify-commit-messages-for-a-push
# Only enforces the check for the master branch

my $errors = 0;
while (<>) {
  chomp;
  next unless my($old,$new) =
    m[ ^ ([0-9a-f]+) \s+   # old SHA-1
         ([0-9a-f]+) \s+   # new SHA-1
         refs/heads/master # ref
       \s* $ ]x;

  chomp(my @commits = `git rev-list $old..$new`);
  if ($?) {
    warn "git rev-list $old..$new failed\n";
    ++$errors, next;
  }

  foreach my $sha1 (@commits) {
    my $msg = `git cat-file commit $sha1`;
    if ($?) {
      warn "git cat-file commit $sha1 failed";
      ++$errors, next;
    }

    $msg =~ s/\A.+? ^$ \s+//smx;
    unless ($msg =~ /^[A-Z]+-[0-9]/) {
      warn "The git commit message for $sha1\n";
      warn "must be prepended with a valid JIRA key.\n";
      warn "Use CAPS for the project key.\n";
      warn "For example:\n";
      warn "    KEY-123: $msg\n";
      warn "Use \"git commit --amend -m\" to fix the message or\n";
      warn "\"git rebase\" if you are pushing multiple commits.\n";

      ++$errors, next;
    }
  }
}

exit $errors == 0 ? 0 : 1;

Unfortunately, it can't be nicely managed with the latest features in stash. I like the plugin architecture and the ability to enable/disable a particular hook for each repository and also create a unique configuration for each repo.

However, it's not a trivial process to port existing shell/perl scripts like this to Java and the plugin architecture.

I'd really like to see a feature where a script could be uploaded to stash and managed through the dashboard without having to manage scripts like this directly on the disk.

We've been forced to create our own management tools/automation for this and that's where stash should really be adding value IMHO.

thanks,

Garret

[cofarrell]

Hi Garret,

I know this isn't going to much of a consolation, but if you're interested I've recreated that script for you as a Stash plugin:

https://bitbucket.org/cofarrell/stash-enforce-message-hook-plugin

I've also uploaded the jar to make things easier for you to try out.

Charles

[Benjamin Mueller]

Hey Mark,

the JIRA Hooks for Stash Plugin will be available for Bitbucket soon. Then you are able to cover your usecase. 

Cheers

Benni

[mark_gillespie]

Still stunned this isn't an included fearture of git essentials.  JIRA references in commit messages are the essential part that welds code and original issue, and we need this to be enforced.  The Tickets Please! Plugin has been working well, but I can't seem to find anything for Bitbucket Server 4

[Benjamin Mueller]

Hello erberybody,

please see also the plugin JIRA Hooks for Stash (https://marketplace.atlassian.com/1211959)

Best regards

benni

[RémiS]

I find the option to retrospectively edit commit messages very interesting (RL). In the corresponding UI, it would be helpful to be able apprend/prepend all messages with an issue ID in a single operation. This way, if you have 30 messages to ammend, you still could do it in a reasonable time.

Another thing that might help is to consider the branch name. If it was created using Stash/Jira then the the branch already has a valid issue Id in its name in most cases. This could be checked when the branch is pushed/merged and then if it actually contains an issue ID the whole commit batch could be accepted as is (maybe a configurable option) even without checking commit messages. I'm not sure what would be the impact on other integration plugins relying on the issue ID being in each message, though.

Also, I know that local hooks cannot be bundled (at least working hooks) in the git repo, but they could be provided in a standardized manner with every repo created (assuming it was created with the "jira-integrated" option) with simple instructions/scripts for each developer to install them. Instead of a complex hook connecting to Jira, I think a simple regexp checker would be enough at this level. Stash could even generate the hook script dynamically so that it explicitly checks for the specific issue pattern.

Rémi

[Dana Frost]

I have worked on a client hook and cgi script that checks for the presence of a proper (and correct) Jira issue if/when the user is online using a cgi script.

- We provide a git clone "clone -template..." wrapper so that all developers get the client hooks. Since many projects are made up of multiple repos, most use the wrapper without question and get the client hook. Tthe commit-msg hook uses curl to call a remote cgi script on the JIRA server which will return "true/false" if the commit contains a valid JIRA issue. If the commit includes a correct JIRA issue (that actualy exists and is in the correct fixVersion) its ok. Otherwise, it fails. We use cgi instead of just a client hook so that the logic can change but the client hook will not need to be updated because the logic of what is a correct JIRA issue is in the cgi code, not the git hook. The git hook just get back a true/false from the cgi if the commit looks like is has a correct JIRA issue listed.

- If the developer is working offline, the client hook cannot check for a correct JIRA issue. For now, we just let these go (most people work online or VPN).

I wish git would embrace some form of "client hook" copy/update option (other than templates) during a clone that was not "optional" but dictated by the remote/source repo. I know this is kind of against the culture but as long as it was an optional thing set up in the remote side, it would make client hooks much more predictable.

[cofarrell]

Hi Dana,

That's quite a neat idea.

I agree it's unfortunate, but part of the nature of Git. Hopefully Stash and other tools, like Sourcetree, can get smarter at enforcing some of these restrictions on behalf of Git.

Charles

[Benni]

I have installed the version 1.1 together with the Stash version 2.8.2.

The problem I have is, that each commit and each push is allowed. But I would like that the plugin checks, whether a valid issue is entered in the commit message..

I enabled the checkboxes for the usage of the Application Link and the searching of an Issue Key. The rest is default..

But it doesnt work. Any Idea?

Is there any documentaion?

[cofarrell]

Hi Benni,

Unfortunately that is a third-party plugin and Atlassian doesn't provide any support. Have you tried to contact the authors on Marketplace? Are there any errors in your Stash logs?

Charles

[Eclipse Trading]

I had the same problem. Benni, were you able to fix your problem?

[Ishan]

Hi Charles,

I am trying to understand what Brian said initially here:

"The primary drawback there is that, when such a push is rejected, the user then has to amend their commits and rewrite history to add those issue keys. Depending on how many commits the user had made, that can be pretty painful. "

So, I have a stash instance running and I am implementing git hooks.

Scenario: I have 3 commits in my local repo.. Now I want to push all of it to my master, so I do "git push origin master"..

* This fails, which I fixed by updating the git commit message using git commit --amend -m "blah". And then I did the git push origin master again.

* This submitted all my three commits, which means if the git push fails because of a commit message, then they can just fix the message and do the push again.

* This I think will keep the history as it is as well.

Please correct me if I am wrong...

Also I am trying to understand what is "retrospective linking approach?"

Thanks for your patience..

--Ishan

[cofarrell]

Hi Ishan,

Just on your scenario, so you have three commits - your commit ammend will only work on the most recent commit. How do you modify the previous commits? What happens if there was 10 commits without JIRA keys? Or 30? Would you change them all?

I know Git pretty well, and I would be very uncomfortable explaining to most users how to update all those commits without them getting into trouble. And if there are merges things get pretty messy pretty fast.

So I agree with Bryan that the better/safer solution is to have some form of UI in Stash that allows you to select a commit or commits and link to a JIRA issue (and vice-versa) as if you had pushed with the correct message. Maybe if you create a PR with a JIRA issue it links all those commits automatically so we don't need a UI. I don't know, we haven't decided on anything yet. In a DVCS world something _like_ this is the only real solution, unless all your users are Git experts, or you only ever do one commit/push at a time.

Does that make sense?

Cheers,

Charles

[GPReleaseEngineering]

Yes.. I want to try out precommit hooks at the git level, to see that goes.. Also, I saw in one of the posts that you can do precommits at repository level but not at the global level. Is that possible with stash ??

[cofarrell]

Hi Ishan,

You can certainly add hooks to the raw Git repositories, although Stash doesn't make any effort to manage that.

Actually are we talking about pre-commit or pre-receive? Pre-commit are only on the client, for people using Git locally as they commit. Pre-receive is on the Stash side and can potentialy reject a push containing multiple commits/branches.

Regarding your previous question, the arguments are described here:

https://www.kernel.org/pub/software/scm/git/docs/githooks.html

For pre-receive you would have to manaully look up the commit message using the appropriate git command.

Again, assuming you're talking about pre-receive, Stash only supports configurable hooks at the repository level. We also have a global pre-receive hook, but currently don't provide any global administration API. If you plan on using Python scripts you would have to manage the templates yourself - and one option might be to put the scripts in the global Git templates directory, which is copied when Stash creates a new repository. See my answer here for more details.

There are few different concerns here - I hope that helps somewhat?

Charles

[GPReleaseEngineering]

Hey Charles,

This seems to be working fine for me.. Essentially I want to try and do this in python... So is it possible to get the commit message, for the current commit in question ??

Thanks for all your assistance..

--Ishan

[cofarrell]

Hi Ishan,

Apologies, I'm not sure what you mean by 'in python'. Do you mean adding a manual pre-commit hook in Git? Or something else?

Charles

[GPReleaseEngineering]

Hey Charles,

Yeah it seems to be working now. Here is what I tried:

I enabled the precommit check, then I tried to do a submit with my commit message =

git commit -a -m "Testing it out". This failed because of the precommit check.

So ,then I tried to fix the commit message using:

git commit -ammend -m "Testing it out [ISSUE: FEA-12345]"

and then I did git push origin master

but this does not seem to be updating the message and so it was failing for me!!

Aplogies for the inconvinience.

--Ishan

[cofarrell]

but this does not seem to be updating the message and so it was failing for me!!

I'm not sure what you mean? The commit ammend didn't work? Or the push?

Glad it's working though.

Charles

[GPReleaseEngineering]

Hi All,

I installed : stash-enforce-message-hook-plugin-1.0.0-SNAPSHOT.jar on my stash server and was testing it out for a bit, but somehow it does not seems to work..

Here is what I have:

In the hook:

Pattern: [Issue:[ \t]*(.*)] & Message: Needs an issue for this checkin

My commit message: git commit -a -m "Testing it out"

This just works. However it should not!!

Also,

when I try the below it does not work:

Pattern: \[Issue:[ \t]*(.*)\] & Message: Needs an issue for this checkin !!

My commit message: git commit -a -m "Testing it out [ISSUE: FEA-12345]"

This never works..

Please let me know if I am doing something wrong!!

[cofarrell]

Hi,

The second example is correct, assuming you want to have 'Some message [Issue: FEA-1234]'. However, your commit message in the example above is in uppercase, but the regex expects 'Issue'.

I have tested this locally and it works fine.

Charles

[Ivo Kronenberg]

Using the pre-receive hook is not an ideal solution because Git does (currently) not allow changing the log message on a easy way.

@Bryan: I'm wondering is there in the near future a new feature in Stash for the "retrospective linking approach"? I could imaging this as part of the review process adding beside of some commens also JIRA Issues. But how would you handle it if the log message has already a JIRA issue, but it's the wrong one? Do you think about mechanism of overriding; which could make intransparent.

[cofarrell]

Hi Ivo,

You may be interested in the following feature request:

https://jira.atlassian.com/browse/STASH-3628

Cheers,

Charles