We recently moved the home drive for Bitbucket Server to an NFS mount for easier expandability and data recovery. Shortly after, users started reporting performance problems. The page times out when trying to look at diffs on a pull request page or just waiting for the "Merge" button to resolve.
At the times people complain, I see the following in the profiler log:
[2.8ms] - Page com.atlassian.bitbucket.user.UserService.findGroupsByUser(String,PageRequest)
[4.1ms] - InternalPullRequest com.atlassian.stash.internal.pull.PullRequestDao.findByRepositoryScopedId(int,long)
[148745.4ms] - void com.atlassian.bitbucket.pull.PullRequestService.streamChanges(PullRequestChangesRequest,ChangeCallback)
[3.8ms] - InternalPullRequest com.atlassian.stash.internal.pull.PullRequestDao.findByRepositoryScopedId(int,long)
[60.2ms] - Map com.atlassian.stash.internal.comment.InternalCommentService.countCommentsByLocation(CommentSearchRequest)
[60.2ms] - Map com.atlassian.stash.internal.comment.CommentDao.countsByLocation(CommentSearchCriteria)
[148325.4ms] - git: with pull request lock(603, 276)
[2.0ms] - InternalPullRequest com.atlassian.stash.internal.pull.PullRequestDao.findByRepositoryScopedId(int,long)
[2251.5ms] - git: resolve effective diff(276@1)
[311.2ms] - git: resolve merge-base(276@1)
[302.2ms] - nio: /usr/local/bin/git merge-base 2ad034b1d2dc3626922821f15177f07ba98f13cf e53dcda7c018040f91de3d288ad135f78efff1a3
[1684.3ms] - git: update refs(276@1)
[343.4ms] - nio: /usr/local/bin/git update-ref stash-refs/pull-requests/276/from 2ad034b1d2dc3626922821f15177f07ba98f13cf
[1.1ms] - InternalGitPullRequestCachedMerge com.atlassian.stash.internal.scm.git.pull.GitPullRequestCachedMergeDao.findByPullRequest(PullRequest)
[343.5ms] - nio: /usr/local/bin/git diff-tree -C -r --format=%H 4a382d6f0ebe01aff3d7b7b68270bd531614a991 2ad034b1d2dc3626922821f15177f07ba98f13cf --
Did you move the entire home directory to NFS or just the shared directory? Moving the entire home directory to NFS is strongly discouraged since caches reside in this directory, and having those on NFS defeats the purpose of the caches (since they're supposed to locally cache files from NFS for faster access).
Premier Support Engineer
The whole home directory.
Changing the nfs mount options to the following may have fixed it.
But I would like to investigate your suggestion. Is there a guide? Do I just mount the nfs folder to "shared"?
Yep, you can just mount NFS to <BitbucketHome>/shared.
You can find more at https://confluence.atlassian.com/bitbucketserverm/latest/install-bitbucket-data-center-1096089986.html under Step 1. Configure file share mounts.
Premier Support Engineer
Beginning on April 4th, we will be implementing push limits. This means that your push cannot be completed if it is over 3.5 GB. If you do attempt to complete a push that is over 3.5 GB, it will fail...