Load balancing Stash-Git

[Deleted user]

We are evaluating Stash as one of the SCM choices for future self-hosted Git management. We understand that the heavy lifting is done by Git, so the deployment model that is being considered is one where Stash connects to multiple Git instances on geographically separated servers. Is such configuration possible in Stash? If not, has anyone attempted to do the same?

[Kelly Schoenhofen · Accepted solution]

We're still in the process of rolling out Stash and we hit some throttling back in April, so we looked into how to increase the scalability/performance of the product.

Atlassian has a couple pages of documentation on this, if you haven't read them yet:

https://confluence.atlassian.com/display/STASH/Scaling+Stash

and

https://confluence.atlassian.com/display/STASH/Using+Stash+in+the+enterprise

So the bottleneck is the git executable, not Stash itself (like you surmised already).

What we had already done:

1) Don't run Stash behind a proxy.

2) Give the Stash server 2-cpus.

3) Move the DB to an difference, dedicated server.

What we did after the throttle banner came up and we read up on scaling Stash:

4) Give Stash 4-cpus.

5) Upgrade Stash to 2.3 (then 2.4 and soon 2.5).

6) Use https when possible for our build servers, vs ssh. While allegedly ssh uses abit more cpu in the decrypt/encrypt vs https, more importantly Atlassian said the caching does not function over ssh, just over http/https.

I believe in 2.5 the caching is enabled by default.

Going to 4-cpus was probably our biggest bang for our buck. I'll probably go to 6-cpus if we have another throttle event before looking at splitting the server up.

[Deleted user]

Thanks Kelly. The second link on enterprise use is the one I haven't seen. It will be a big addition to the analysis we are pursuing among many other apps out there. Appreciate it.

[Kelly Schoenhofen]

One more, not sure if it made a difference. I think keeping up with the latest git might also keep performance high, and match what the developers are using (or stay a tiny step ahead). I saw some odd issues on our Fisheye server not able to understand part of a commit it was processing until I upgraded from git 1.7.x to 1.8.x.

Since this is an Ubuntu 12.04.2 LTS server, using apt-get gets you Git 1.7.9 I believe ; I updated to the latest git ppa on the server, which brought it to 1.8.3 (as of last week at least).

For Ubuntu, that's something like this:

sudo apt-get install python-software-properties
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:git-core/ppa
sudo apt-get update
sudo apt-get upgrade

- git 1.8.x should be on the upgrade list now.

[Deleted user]

Thank you both.

[Deleted user]

I found a reply to a post on forking. https://blogs.atlassian.com/2013/05/stash-git-forking-development-workflow/

Directly writing to a git repository managed by Stash is probably not a good idea. Hmm..

[Jens Schumacher [Atlassian]]

Most definitely not.

[Jens Schumacher [Atlassian]]

Hi Tosak,

I would love to discuss your requirements in more detail, but answers is probably not the right forum. Please feel free to reach out to me via "jens at atlassian.com".

Cheers,
Jens

[Deleted user]

Hi Jens, thanks for the reply.

We are seeing more and more internal users moving toward DVCS in favor of speed and other merits. Many chose Git as their favourite and we want them in a centrally subsidized platform. If this was launched in full, I'd imagine an increment of a thousand users per year base on current SVN user base.

Centralized Stash and repositories was the first model being looked at. However, to make a decision alternatives must be made available, be it simplistic or complicated. I trust that Stash has been designed to scale decently with increasing demands, but I do not want to impose Git's scalability on Stash. Hence the question.

The objective of the distributed setting here is to keep all administrative functions in one place, while allowing users to push changes to a different git servers. Is this covered in the mirror scenario mentioned?

Thanks,
Tosak

[Jens Schumacher [Atlassian]]

Hi Tosak,

It is not possible to connect to multiple Git repositories located on different servers at this stage. That being said, you could setup a Git mirror and run heavy operations like CI against the mirrored repository.

What scale you are looking at? Stash has been build to scale well on a single machine. Therefore I would suggest to evaluate whether a complicated deployment is required in the first place.

Cheers,
Jens

[Deleted user]

I've seen that one. It's a different question. Wojciech was asking about Disaster Recovery (failing over to the second Stash instance). 2 Stash + 1 Git.

My question is on load balancing Git and whether Stash can handle that. So 1 stash + multiple Git.

[dleng]

Already been asked here: https://answers.atlassian.com/questions/128693/stash-high-availability