Always waiting on lock on subrepository

[g_reimers]

Inside my Mercurial repo I have another subrepo.

Whenever I change the subrepo, commit and push it, I can no longer push the main repo. When trying to push the main repo to its server SourceTree will always report "waiting for lock on repository <path of subrepo> held by '<MyMac>.local:47475'"

I can always resolve this my manually deleting the lock file within .hg/store/lock but that is very inconvenient.

Is there anything I should be aware of, that might cause this, or is this a bug in SourceTree or Mercurial?

Does anyone experience the same problems?

[stevestreeting · Accepted solution]

I haven't seen this before, but out of curiosity how are you committing / pushing the subrepo, from outside or within SourceTree?

There's 2 ways you can deal with subrepo changes in SourceTree, you can commit / push via the parent repo when you commit the parent (you're prompted to create a commit on the subrepo if uncommitted, prompted to push it before the parent etc). Alternatively you can work entirely within the subrepo first in its own repo window before dealing with the parent. In both cases I haven't encountered the lock problem before.

It would also help to know what version of Mercurial you're using for both sets of actions, wherever you're invoking them from. If within SourceTree, please let me know what version is displayed in Preferences > Mercurial.

[g_reimers]

Thanks for the quick reply, Steve.

I have used both ways to work with my subrepo, as you described. The problem occurs with both. I even have two cloned copies of this same subrepo, which I use in two sparate main repos (i.e. two projects which share the code in the subrepo). In both versions this issue appears, so I assume it is not that the repos are damaged.

I only used Source Tree to do any commits and pushes. However I did the initial 'hg init' from terminal and then added the created repo as a subrepo.

I use:
Mercurial Version: 2.4+20121101 on Mac OS 10.8.2 (English) and Source Tree 1.5.6 (also occurred on 1.5.1).

I have also filed a bug on https://support.atlassian.com/browse/STSP-335

[stevestreeting]

You're using a newer version of Mercurial than I've tested with yet, but I can't reproduce this on hg 2.2.2 or 2.3.1. I don't know if you can try with the embedded 2.2.2 (depends if hg made any breaking changes in 2.4, I hope not but occasionally it happens), if you can it might be a useful test. As you say it's probably not just a damaged repo or an odd case of other lock files lying around if it's reproduceable in two different clones.

If that doesn't help, I'm curious whether the lock files ever go away on their own, say if you do each step one at a time, checking the content of the lock folder, and even perhaps close ST to see if that frees them. It might also help to enable 'Always show full output' in Preferences, to make sure there isn't some warning in the hg output which you're not seeing because the overall result is success and ST doesn't show you the output by default in that case.

[g_reimers]

Ok, I did some further testing with disappointing results:

- I could perfectly reproduce this with a fresh repository + sub repo on a different Mac (also with the new Mercurial version installed on OS 10.8.2). I switched in the ST preferences between built-in and system Mercurial version, but this had no effect. The problem occurred with 2.2 (built in) and 2.4 (system)

- I then moved the repos to a fresh virtual machine running OS 10.7 with no Mercurial installed. Same problem. There I also re-created the repos from scratch only using ST (no Terminal) but the effect was the same.

- As you suggested I did the commit and push actions step by step and checked when the lock is created. The lock is only created when I do a push in the parent repo. (Commiting works fine.) The push progress will then just spin forever and the logging says "Waiting for lock". I enabled the full output but that does not provide any further details.

I am surprised that you can not reproduce this. Maybe I'm doing something wrong, but I only use the built in actions in SourceTree and only do commit and push. I have tried different variations in the order how I commit the repo and subrepo, but no matter how I do it, I always end up with this lock...

[stevestreeting]

I was testing with one of a number of existing subrepo test setups I already have, and it didn't have a problem when pushing from the parent. I'll do some tests from a new blank repo tomorrow to see if I can recreate it that way.

[stevestreeting]

A fresh repo doesn't reproduce this for me either. I did the following (all in ST)

1. Create a new parent repo on Bitbucket
2. Clone this parent to a local repo
3. Add a single text file & commit / push
4. Add a subrepo from the sidebar right-click menu, clone another repo as a subrepo of this one
5. Commit the parent to add the subrepo
6. Change a file in the subrepo
7. Commit the change in the subrepo (didn't push)
8. Commit the parent change (since subrepo tracked commit has changed), with the 'Push immediately' option checked
9. During commit/push of the parent, SourceTree tells me I should push the subrepo, which I do by clicking the Push button against the subrepo in the list which comes up
10. The parent push continues and completes successfully

I've tried other combinations of not committing the subrepo separately but doing it when prompted when I commit the parent, and pushing the subrepo separately before going to the parent repo. None of these caused any locking issues.

This is on OS X 10.7.5 using SourceTree 1.5.6 and the embedded Mercurial (2.2.2). It's a MacBook Pro from 2010 with a regular non-SSD drive, so not top of the line or anything. I wondered if it could be a timing issue, but if the lock never goes away without you manually deleting it, it can't really be that.

[g_reimers]

I guess I see what makes the difference: When I created the subrepos, I just copied a complete Mercurial repository-folder into my parent repo's folder. Then I performed "Add subrepo" and selected this folder as the source and as the relative path.

When I add the subrepos like you did (by selecting a repo outside of my parent repo, and have SourceTree clone it to my parent repo), everything works fine.

Is there anyting done differently, when manually copying the subrepo in contrast to ST cloning it?

[stevestreeting]

Ah yes, when you add a subrepo the parent needs to know 2 things: the local path, but also where the subrepo would be cloned from, which should be a remote source so that anyone who clones this parent can automatically also clone the subrepo. This information is stored in the parent, not the subrepo, because you can essentially have a different remote set up in the subrepo itself than where it would usually be cloned from originally (think forks).

So by saying that the source of the subrepo is the same as the local path, you've set up a case where the parent thinks it will clone from inside itself. This is almost certainly what's causing the deadlock.

[Edit]FWIW the reason 'browse' is allowed in the Add Submodule window for the source is in case you have a setup where you clone from a network share or something. It's not designed to be the same as the relative path. I should probably add some validation to that effect to stop this in future.

[g_reimers]

OK, so this was really a misunderstanding on my side. I assumed that I need to have the sub-repo inside the parent repo first, before I could add it to the parent. I was not aware that SourceTree would/could clone the subrepo.

Some kind of validation and a short notification would be great in future versions of ST.

[stevestreeting]

Great, I've added an improvement request for this here: https://jira.atlassian.com/browse/SRCTREE-1291