We have happily been using Stash for almost 5 months, with no real issues. However, I have just had to reboot our Stash Server, and since doing so, Stash has become almost unresponsive.
When digging into it, we are seeing very high CPU usage by the git-upload-pack process. This seems to be triggered when we try to start a build from TeamCity. This process has now been running for over 40 minutes, and doesn't show any sign of stopping. I have read that the need for running this process is to clean up loose objects in the repository, before sending over the wire to clients, like TeamCity. Is my understanding here correct. If so, is there anything that can be done to improve this?
The system, as is, is almost unusable
I have read also about using git gc to clean up loose objects in the repository, which should make the need for an extensive git-upload-pack less, if this is the case, how can I run this on Stash? Should I also run it on the cloned repository in the TeamCity working folder?
Any guidance you can provide in getting this resolved would be greatly appreciated!
Thanks for reviewing this.
I think we need to analyse what is going on with your instance and we need to collect data on your environment for that. Could you please raise an issue at https://support.atlassian.com and quote this question there?
Please have a look at the documentation below:
If your CI is polling constantly Stash, turning on ref advertisement and making sure you are caching for SSH and HTTP might help you.
Before posting here I saw that article, but I didn't see anything there that would directly help me, unless I missed something. Also, the need to scale concerns me. We really only have 1 repository in Stash, which is being used by two developers (this is a prototype setup to see how things will work). Do I really need to scale up my server for this sort of setup?
Can you point me in the direction of how to setup and configure:
We are not using SSH.
Also, any thoughts on the last part of my question, i.e. with regard to cleaning up the git repository? Is this something that we should actively be doing, or is this something that Stash is doing for us?
This morning, I have been through the article that you mention, and I have set up the settings that make sense to me:
I have also upgraded Stash from 3.1.1. to 3.5.0, however, none of this seems to have made a difference
Having just started TeamCity and Stash again, I am still now seeing a very high CPU usage when triggering a build. I started a build at 08:58 and immediately CPU usage on Stash server jumped to 80-90% and after about 1 hour 20 minutes, the CPU usage returned to normal. During this time, the TeamCity build had failed due to timeout, but starting it again, resulted in a successful build.
Subsequent builds then also work as I would like them to, i.e. reasonably fast. I can only assume that they are now using the pack file which was generated during the first failed build.
I am still concerned about the length of the time that the first build takes. I can "fix" this by scheduling a build to run every morning before anyone comes into the office, and by the time that is finished, we should be able to use the cached pack file during our working day. This really doesn't seem like a great solution though. Can you suggest anything that would take down the length of time to create the initial pack file?
This community is celebrating its one-year anniversary and Atlassian co-founder Mike Cannon-Brookes has all the feels.Read more
Bitbucket Pipelines helps me manage and automate a number of serverless deployments to AWS Lambda and this is how I do it. I'm building Node.js Lambda functions using node-lambda ...
Connect with like-minded Atlassian users at free events near you!Find a group
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no AUG chapters near you at the moment.Start an AUG
You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs