Stash slowdowns

Marc-Andre November 5, 2014

Hi,

we've been using Stash for a while (upgraded to multiple versions, we're now on 3.1.3)

We're having an issue where Stash will just slowdown, and the web interface responses really slowly and then just grinds to an halt. Running a TOP -C will show us that there is almost 100% memory usage. We can't find anything in the logs as well.

We have a license for 500 users but we have about 300 accounts created.

We are running Linux RedHat 6.4 with 2 VCPUs and 8 gigs of ram.

Any help would be appreciated,

 

Thanks!

 

Marc,

4 answers

2 votes
Michael Heemskerk
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 5, 2014

Hi Marc,

The type of slowdowns you're seeing and the fact that almost 100% memory is being used is an indication that the Stash process/memory is being swapped in and out of RAM.

In your top -C can you see which processes are using the memory? My guess is that it might be the git processes that are being executed when you clone a repository. The Stash process itself is designed to be able to run with a modest footprint, but the git processes that are forked by Stash are much more variable. Do you by any chance have large repositories that are cloned at (relatively) high frequency?

Because of the CPU and memory  usage of these git processes, Stash limits the amount of git processes that it will execute concurrently. See https://confluence.atlassian.com/display/STASH/Scaling+Stash for more background information. If you've changed the configuration option that controls the amount of concurrency (throttle.resource.scm-hosting), you'll either have to scale that value back or add more memory to the machine.

Other questions:

  • How much memory have you allocated for the Stash process? 1GB should be enough.
  • Is the server dedicated to Stash or do you have other applications running on it as well?
  • When Stash is experiencing this type of slowdown, how may git processes do you have running? Please note, that each 'clone' operation typically results in a tree of 5 git processes (git itself forks child processes).
  • Have you tweaked any of the settings in STASH_HOME/stash-config.properties? For instance, throttle.resource.scm-hosting?

Cheers,

Michael

 

0 votes
Marc-Andre November 5, 2014

Hi Michael,

Thanks for helping out (thanks Mike as well)

Give me a few moments to answer all your questions, i'll get the info from the server right now.

0 votes
Marc-Andre November 5, 2014

Hi Mike, that's exactly the page that I'm reading right now. I haven't seen anything in the logs other than error message from the license that can't reach the marketplace website (We run the STASH machine in a envinronment without internet).

0 votes
Mike Friedrich
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 5, 2014

There a few plugins which can use some CPU (especcially those who scan repositories for information).
Thats one possible source, which do you have installed?

Second, 2 VCPUS could be just too less, checkout this page: https://confluence.atlassian.com/display/STASH/Scaling+Stash

 

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events