Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

Next challenges

Recent achievements

  • Global
  • Personal

Recognition

  • Give kudos
  • Received
  • Given

Leaderboard

  • Global

Trophy case

Kudos (beta program)

Kudos logo

You've been invited into the Kudos (beta program) private group. Chat with others in the program, or give feedback to Atlassian.

View group

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Pre-receive hook for rejecting large files

Hello,

I would like a simple pre-receive hook that rejects commits containing files larger than a certain size (to force users to use LFS). However, I'm blocked at every turn.

 

I found this plugin: https://marketplace.atlassian.com/apps/1212934/file-hooks-plugin/version-history

 

But the "installation instructions" lead to a broken page: https://confluence.atlassian.com/display/UPM/Installing+add-ons?_ga=2.175288618.941315121.1568314065-535602028.1568062230#Installingadd-ons-Installingbyfileupload

 

I am unable to find that plugin in the "marketplace" like it advises and I can't find anything similar in the marketplace. Note this is for my company's local "BitBucket Server" instance which is really hard to find search results for because all results keep taking me to BitBucket Cloud.

 

Thanks for any help.

 

6 answers

1 accepted

2 votes
Answer accepted
Mikael Sandberg Community Leader Sep 12, 2019

Hi @bryan.gillespie

The app you found is no longer supported by the developer/vendor so that is why you could not find it in the marketplace. 

There are a couple of apps that you could use, Scriptrunner has a pre hook that allows you to prevent large files being pushed, External Hooks and Javascript Hooks allows you to write your own hooks.

Mikael Sandberg Community Leader Sep 12, 2019

Another option that would not require an app would be to create your own client side hook and store it in a repository, see Including Hooks in a Git Repository. The drawback is that you have to make sure each client is configured to use the hooks.

The reason the "installation instructions" lead to a broken page is probably because that particular app is not compatible with Bitbucket 6.x.  According to the marketplace page you linked to, its most recent compatibility range was:

Bitbucket Server 5.2.0 - 5.16.10

 

p.s. On October 8th, 2019, we (bit-booster) released a new version of our free Control Freak plugin that now includes the ability to block large files from being pushed into Bitbucket Server or Data Center.  See our separate answer on this thread below for more information.

Like # people like this

Ah, I see. Thanks for the advice. I will look into those options. Maybe I'll create a new app for the marketplace that does this.

Like Mikael Sandberg likes this

Thanks @Sylvie Davies _bit-booster_com_ ! Control Freak is just what I was looking for.

Bryan:

I think we were in a similar situation. I believe we were using the same plugin, too. We had to find alternatives in order to stay current with Bitbucket Server. 

I installed Adaptavist Script Runner for Bitbucket Server. The initial install was for a different function, which it does very well. I thought I lucked out with the "restrict file size" pre-hook that it has, but it turns out that, if the commit contains a file of the type we want to limit (PDF), the restriction will apply to _any_ file in that commit, not just the PDFs. This can be worked around, but I'd rather my users not have to think about this.

I tried to play around with Script Runner more, but I have no Groovy experience and got lost quickly, so now I'm back to considering writing my own script and putting it on the server. This isn't too terrible, but I can't help thinking that I'm reinventing the wheel.

If you have any suggestions or even dead ends, I'd love to hear them.

Thanks,
Pete

 

Pete,

I could add an additional filter to my free Control-Freak plugin.  Right now it does this:

 

[x] - Block large files. Maximum allowed file size (bytes): [     ]

[x] - Encourage Git LFS in push rejection message.

 

I *could* add the following control:

[x] - Only block large files that end in the following extensions: [ <comma-separated> ] (e.g., *.zip, *.exe, *.pdf, *.jar)

 

Is that what you need?  Note: files that fall below (or equal) to the threshold would never be blocked, even if they had the bad file extension. Set the threshold to zero to completely ban that file type.

Like bryan.gillespie likes this

Yes, I believe that it would. 

 

Okay!  Done!  Look for v2019.10.20 of Control Freak in marketplace.atlassian.com.

Screenshot:

a-large.png

 

Consequence for a push with large files:

remote: Control Freak
remote: -----
remote: Push rejected.
remote: All files larger than 10 bytes are not permitted.
remote: Specific files [*.exe, *.pdf, *.zip] larger than 5 bytes are also not permitted.
remote: The following files are too large:
remote:
remote: binary/f.exe (7 bytes)
remote: binary/g.zip (7 bytes)
remote:
remote: You can push large files to this repository using 'git lfs':
remote: git lfs track <path>
remote:

 

Like Jean-Pol_Landrain likes this

Very nice. Should issues with my hand-written pre-receive hook pop up, I'll be sure to give Control Freak a look. 

It is very nice, but if I can suggest an improvement: it is possible to disable Git LFS in the Bitbucket server settings. Can you also take the value of this option into account in order to disable the part of the message that suggests to use the "git lfs" command ? The option should be kept in Control Freak, but it shouldn't be taken into account if Git LFS is disabled in the server settings.

Our free Control-Freak plugin now offers this functionality (compatible with Bitbucket Server and Data Center 5.8.0 - 6.x).

Here's a screenshot:

large-files.png

Here's what the rejection message looks like:

$ git push
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), 7.46 MiB | 4.47 MiB/s, done.
Total 5 (delta 1), reused 0 (delta 0)
remote: Control Freak: Push rejected. Files larger than 8000000 bytes are not permitted
remote:
remote: Control Freak
remote: -----
remote: Push rejected. Files larger than 8000000 bytes are not permitted
remote: in this repository. The following file is too large:
remote:
remote: path/to/bigFile.jar (8722752 bytes)
remote:
remote: You can push large files to this repository using 'git lfs':
remote: git lfs track <path>
remote:
To ssh://vm.bit-booster.com:7999/bb/rep_1.git
! [remote rejected] master -> master (pre-receive hook declined)

I have a question about this: If I'm trying to push a large file, will the file be uploaded and THEN processed, or is the git client telling the server "hey, I have a large file to upload, may I?" and the server will stop the file from going through the network if the check fails?

Thank you!

Another way you can go about solving this is with Better Commit Policy for Bitbucket.  (Please note that relies on Better Commit Policy for Jira!) You can also be more effective with the local hooks, provided by the app. By using local hooks, you can catch non-compliant commits right on the developer's machine at commit time, so it's guaranteed to pass when pushed to the server.

Any plugin that can substitute File hooks plugin for bitbucket 6.x Datacenter?

A control freak plugin can block for the entire server. But we are interested in project-specific blocking. Is that possible with a control freak plugin. Because every project has it's own limitations and policies.

Anyone please suggest an alternative plugin to File hooks plugin which can reject the commits of files of a certain size and file name of certain pattern.?

 

Thanks in advance.

Anoop

Yes, you can use Control Freak. It is very well done: you have configuration options for the whole Bitbucket instance, but all the settings can be overriden both at the Git project level and at the repository level. So, it allows for very fine tuned settings.

I am unable to block specific file types from pushing to the central repository.

Though there's an option to block certain file types as below. But it's not working at all.

I tried with *.pdf and pdf and .pdf conventions. But nothing is working. Still, I am able to commit pdf files to the repository without any trouble.

a-large.png

But this plugin is able to block larger files but not specific file types.

 

Any alternate plugin, incase if this plugin can't do this job ?

 

Thanks in advance.

I have done a quick test and unfortunately I confirm it's not working for me either (version 2020.02.07)

 

Are you sure?  This is working for me (I'm the primary maintainer of this plugin):



remote: Control Freak: Push rejected.
remote:
remote: Control Freak
remote: -----
remote: Push rejected.
remote: All files larger than 500 bytes are not permitted.
remote: Specific files [*.exe, *.pdf, *.zip] larger than 200 bytes are also not permitted.
remote: The following file is too large:
remote:
remote: test.pdf (1723760 bytes)
remote:
remote: You can push large files to this repository using 'git lfs':
remote: git lfs track <path>
remote:


 

There is one important loophole to know about with this control:  any file that already exists in the repo (on any branch or tag, including unreachable tags or branches, and including files deleted many commits ago - they are still present in the repo) cannot be blocked (based on contents, not based on filename).  The control only gets triggered during the phase where git-receive is calculating missing files to accept during the push, and since git-receive skips files that already exist on the server, regardless of their name, there is no way to block those.

This means developers can also easily get around the control:  "git push random-file.allowed" and then rename to "random-file.pdf" in a later push.

Is there any chance you guys are hitting against this loophole, where byte-for-byte identical files (regardless of name) cannot be blocked if they are already present in bitbucket?

p.s.  the control happily accepts all of these:  pdf, .pdf, *.pdf

(they all evaluate to the same file type)

I have done a few more tests this morning and in my case it's possible I was into this loophole in my previous tests.

Please, note however that if you don't specify a maximum size (it is not very clear that it is a mandatory field), I believe it should take a size of '0' bytes as default. I don't know if there is a default size at the moment, but it doesn't seem to block anything unless you specify the size (also there is just one pattern both for the filename and for the size; this limits the possibilities).

You say above that all the pdf, .pdf, and *.pdf evaluate to the same file type. Could you also indicate if it is possible to use more complex patterns or even regular expressions ? Thank you.

I know this addition is not very old in the Control Freak plugin, and I just try to suggest improvements. This is not criticism, only suggestions.

Thank you very much for your work on this feature, and on this plugin in general.

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

Calling any interview participants for Bitbucket Data Center

Hi everyone,  We are looking to learn more about development teams’ workflows and pain points, especially around DevOps, integrations, administration, scale, security, and the related challeng...

442 views 4 4
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you