It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

how to diagnose why crucible is slow

Paul Wagland Jan 29, 2016

This is a simple enough question, but it probably has a complicated answer.

Given that we have a Crucible installation, running against subversion, how can we determine why it is so slow. We regularly have complaints that the developers need to wait for minutes (literally, not figuratively) for the diff results to show. The rest of the product seems fairly fast, but displaying a diff frequently takes more than 20 seconds, and on a regular basis even longer.

How can we work out what is taking the time in the Crucible stack? What sorts of things should be tuned?

We have 1GB allocated to the heap, and it is running on a 2GB machine, which according to the docsprovides performance "headroom". Our subversion repository is faster than this, but that shouldn't even be relevant given that Crucible has already indexed everything anyway?

What can we check and/or modify to improve this performance?

1 answer

1 vote
Marek Parfianowicz Atlassian Team Jan 31, 2016

Hi Paul,

There may be many reasons, why an instance is slow. I would like to suggest checking the following things first:

1) Ensure that you have blame indexing enabled (the system property 'blame.calc.enable=true'; it's true by default). If not, then whenever FishEye has to display a diff with blame, it has to fetch it directly from SVN, which can be very time consuming. In case you have blame enabled, then it may happen that due to some indexing errors, blame data is not persisted for some files/revisions. In such case FishEye will also have to call SVN to fetch this data. There is an option to disable this fallback as well ('blame.scm.fallback.enable=false'). More details:

2) Please check memory usage of your FishEye/Crucible instance. Check garbage collector logs. It's worth to ensure that 2GB of memory is indeed sufficient and that there are no long pauses caused by frequent full GCs. How many repositories do you have? What is their total size (GB)? 

3) Please measure your network performance. For instance, try running a web browser directly on your FishEye server to see if overall performance is similar to one observed on workstations.


Paul Wagland Feb 01, 2016
  1. We have blame indexing enabled. That said, it doesn't seem to work, we have been advised by Atlasssian to turn on enable the fallback, otherwise we have no blame calculation, which is not a workable solution.
  2. I do not have access to the machine directly. Are the GC logs available through the user interface anywhere? According to our sys admins, most of the time the machine is idle, with occasional spikes of high CPU usage.
  3. The network is not the issue. Most of the Crucible UI is perfectly functional, it only takes minutes to load the diff results, everything else is "fast", or at least "fast enough". For example, clicking on the admin page shows it in well under a second, opening the 'projects' overview shows the result in about a second on the first load. Going to "My Reviews" shows the page in under a second.
Marek Parfianowicz Atlassian Team Feb 01, 2016

ad 1. Please check if FishEye/Crucible logs contain any error messages related with repository indexing, such as:

"Problems calculating blame for <path>@<rev>, will fetch from repository instead"
"Problems calculating blame for <path>@<rev>, blame won't be calculated"
"Unable to annotate <path>@<rev> as the content is not accessible"
"Problem committing author linecount, rolling back"
"Error reading database in linecount slurp"



  • enable debug logging first (in the administration panel)
  • it would be good to pick and reindex one repository (if this is not blocker for your team), to see how indexing was made for all commits (and not only new ones)

Counting amount of such errors will give you an indication how many commits are missing blame in FishEye's index.


Please also check how frequently the SCM fallback is used and how long it takes:

grep -E 'qtp.*<- getSvnBlameChunks.*' atlassian-*.log

Output will look like this (I wrapped the line for readability):

"DEBUG [qtp<thread id> ] fisheye Timer-output - <- getSvnBlameChunks(
rev=<rev>, contentKey=<path and rev>)
 chunks=110 time 144 millis <memory statistics>"

The "time XXX millis" says how long it took to ask SVN for blame.


Marek Parfianowicz Atlassian Team Feb 01, 2016

ad 2. There's no way to get it from UI. You'd have to add proper flags to a JVM and restart the server. 

Marek Parfianowicz Atlassian Team Feb 01, 2016

ad 3. Judging from your description, fetching blame is the main bottleneck. 

Could you tell how your SVN is configured in FishEye? Does your SVN server run on the same machine as FishEye? What hard disk is used? It it SSD?

Paul Wagland Feb 02, 2016

ad 1&3, as these are both going in the same direction.

None of the error messages mentioned at the top of the answer are found in our logs. Debugging has been enabled, but it won't be possible to re-index the repository until this evening. Our smallest repository can still be indexed overnight, so I will do that one and report back.

Looking at one of the files in one of the change sets that I know to be "slow". This file has 7009 lines, and is made up from 353 commits. Doing a 'time svn blame' on this file takes 3.5 seconds. Through the crucible UI this page takes over a minute to show.

This was done with debugging enabled, I'm waiting for our sysadmin to grant access to the file, and will investigate it then to look at the time taken for the svn blame.

Our SVN server is serving the data from a repository that is on normal HDD, and is not co-located on the same virtual machine as Crucible, although it is on a machine in the same VMWare cluster.

Paul Wagland Feb 02, 2016

ad 2. Is there a page indicating where/what these changes should be?

Paul Wagland Feb 03, 2016

Thank you for the follow up information. At this point it is clear that the problem is that Crucible is not indexing our revisions for some reason. Because of this, the fallback is enabled, and that in turn is excruciatingly slow. This issue is being followed up with by a support issue.

Suggest an answer

Log in or Sign up to answer
This widget could not be displayed.
This widget could not be displayed.
Community showcase
Published in Next-gen

Introducing subtasks for breaking down work in next-gen projects

Teams break work down in order to help simplify complex tasks. This is often done iteratively, with tasks being broken down into smaller tasks and so on until the work is accurately captured in well-...

2,262 views 14 17
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you