Bitbucket reserved refs when cleaning up with BFG tool

[Alexander Plotnikov]

I'm cleaning up repositories that has a lot of jar files in it, and i'm using BFG repo-cleaner. 

Also i'm using bitbucket server

- I cloned repo with mirror flag

- Made backup of repo

- Run cleanup

- run git reflog expire --expire=now --all && git gc --prune=now --aggressive as suggested

- run git push

and then git push fails and i get the following error:

Counting objects: 6057, done.

Delta compression using up to 4 threads.

Compressing objects: 100% (2267/2267), done.

Writing objects: 100% (6057/6057), 7.94 MiB | 1.45 MiB/s, done.

Total 6057 (delta 2608), reused 6057 (delta 2608)

remote: Resolving deltas: 100% (2608/2608), done.

remote: Checking connectivity: 6057, done.

remote: You are attempting to update refs that are reserved for Bitbucket's pull request functionality. Bitbucket manages these refs automatically, and they may not be updated by users.

remote: Rejected refs:

remote: refs/pull-requests/34/from

remote: 

 ! [remote rejected] develop -> develop (pre-receive hook declined)

 ! [remote rejected] feature/DEV-1599-date-formatting -> feature/DEV-1599-date-formatting (pre-receive hook declined)

 ! [remote rejected] feature/DEV-1608 -> feature/DEV-1608 (pre-receive hook declined)

 ! [remote rejected] feature/DEV-1785 -> feature/DEV-1785 (pre-receive hook declined)

 ! [remote rejected] feature/DEV-1818 -> feature/DEV-1818 (pre-receive hook declined)

 ! [remote rejected] feature/DEV-1836 -> feature/DEV-1836 (pre-receive hook declined)

 ! [remote rejected] feature/DEV-1839 -> feature/DEV-1839 (pre-receive hook declined)

 ! [remote rejected] feature/DEV-1845 -> feature/DEV-1845 (pre-receive hook declined)

 ! [remote rejected] feature/DEV-1936 -> feature/DEV-1936 (pre-receive hook declined)

 ! [remote rejected] hotfix/DEV-1872 -> hotfix/DEV-1872 (pre-receive hook declined)

 ! [remote rejected] jira6 -> jira6 (pre-receive hook declined)

 ! [remote rejected] jira_6_3_15 -> jira_6_3_15 (pre-receive hook declined)

 ! [remote rejected] master -> master (pre-receive hook declined)

 ! [remote rejected] refs/pull-requests/34/from -> refs/pull-requests/34/from (pre-receive hook declined)

 ! [remote rejected] 1.6.2 -> 1.6.2 (pre-receive hook declined)

 ! [remote rejected] v1.5.0 -> v1.5.0 (pre-receive hook declined)

 ! [remote rejected] v1.6.0 -> v1.6.0 (pre-receive hook declined)

 ! [remote rejected] v1.6.1 -> v1.6.1 (pre-receive hook declined)

error: failed to push some refs to

According to this two comments my problem is in opened pull-requests, and from error it seems like it's pull-request id is 34, but i have no open PR's and trying to find one with id 34 fails, there is no such PR. 

So is there a way to health check PR refs or delete them or do something about thath? Or mb my problem is not even in PR

[Ana Retamal · Accepted solution]

Hi Alexander! The issue you are getting is usually due to the fact that you mirror cloned the remote repository and that copied the Pull Request references as well. Now that you are trying to push back to the repo you are also pushing these references and Bitbucket does not permit that. 

!! Before proceeding with the instructions ahead be sure to perform a backup of your production environment, including the database. If possible test the procedure in a test environment.

Keep in mind that this operation can be risky in the sense that there is a possibility to corrupt you repository while trying to perform it. rewriting history and commits might have unpredictable effect on the other objects of that repo
comments, commits, PRs etc… !!

Our suggestion would be to follow this strategy:

1. Create a new empty repo in Bitbucket server. This is going to be the new repository to use for development, you can keep the old repository as a backup in case something goes wrong and to maintain the old history.
2. Either:
   • Close all the PR in the old repo. Get rid of the pull request(by either getting them merged or declined) and then do the mirror clone again
   • Remove the Pull Requests references from your repository by issuing this command:

 git show-ref | cut -d' ' -f2 | grep 'pull-request' | xargs -r -L1 git update-ref -d

3.Rewrite history in the clone without the PR references.

4. Push the modified repository to the newly created repo.

5. Check if everything went well.

6. Move your developers to the new repository.

Lastly I'd like to mention that if you have large files, such as binaries for example, you might consider enabling LFS (Large File Storage) for this repo. You can learn more about it at Git Large File Storage.

 Let us know how it goes!

Best regards,

Ana

[Alexander Plotnikov]

Thanks @Ana Retamal for answer, can you tell me what does -r flag in xargs command do? I'm using OS X El Captain v10.11.6 and it doesn't have -r flag (illegal option)?

[Alexander Plotnikov]

Also if i ommit -r flag it runs, but seems like it does nothing, because the problem persists

[Alexander Plotnikov]

No, i was wrong it did remove pull-request ref. If i run git show-ref there is no pull-request refs, but it doesn't let me push with the same error 

[Alexander Plotnikov]

@Ana Retamal can you please help? 
We don't want to use LFS since we already have nexus for storing binaries, we just need to remove large binaries to clean up repos

[dandgerson]

maybe `-r` is for `recursive`

[dandgerson]

@Ana Retamal @Alexander Plotnikov  Hello 
I have the same case, but want to stay on old repo. Is the migrate on new repo is mandatory? (I afraid to lost Jira-tasks links)

[Krysta Salera]

@Ana Retamal Following up on the above, do we need to migrate on the old repo? Is there a solution if we need to stay on existing repo?

@dandgerson and @Alexander Plotnikov were you able to find a solution to your problem?

[David Karr]

The answer to the "xargs -r" question is easily solved by googling.  The "-r" parameter makes it not run a command if the resulting input is blank.

[Alexander Plotnikov]

up