SVN Indexing slow on changesets involving a large number of files

Hi all,

I've just started using Fisheye 3.0.1.

I noticed that SVN repository indexing is slow AND CPU intensive which sounded strange to me, so I did some sampling.

The SVN repository is available via HTTPS which is known to be slow, so I expected bad IO performance but not a steady 100% CPU usage.

I found out that during SVN indexing Fisheye is often executing com.cenqua.fisheye.svn.util.ChangePathUtil.isSubPath(..), which consumes 100% of a CPU.

Typical stack trace:

at java.lang.String.equals(String.java:1017)
	at com.cenqua.fisheye.svn.util.ChangePathUtil.isSubPath(ChangePathUtil.java:18)
	at com.atlassian.fisheye.svn.Svn2MessageContext.isApplicableChangePath(Svn2MessageContext.java:109)
	at com.atlassian.fisheye.svn.Svn2MessageContext.getApplicableChangePaths(Svn2MessageContext.java:153)
	at com.atlassian.fisheye.svn.Svn2MessageContext.isPartOfAdd(Svn2MessageContext.java:428)
	at com.atlassian.fisheye.svn.Svn2Infill2Processor.getInfo(Svn2Infill2Processor.java:363)
	at com.atlassian.fisheye.svn.Svn2Infill2Processor.infillChangeSet(Svn2Infill2Processor.java:187)
	at com.atlassian.fisheye.svn.Svn2Infill2Processor.access$100(Svn2Infill2Processor.java:78)
	at com.atlassian.fisheye.svn.Svn2Infill2Processor$1.perform(Svn2Infill2Processor.java:121)
	at com.atlassian.fisheye.svn.Svn2Infill2Processor$1.perform(Svn2Infill2Processor.java:110)
	at com.cenqua.fisheye.cache.BaseRevisionCache.withDbWriteLock(BaseRevisionCache.java:967)
	at com.atlassian.fisheye.svn.Svn2Infill2Processor.process(Svn2Infill2Processor.java:110)
	at com.atlassian.fisheye.svn.Svn2Scanner.slurpRepository(Svn2Scanner.java:210)
	at com.atlassian.fisheye.svn.Svn2Scanner.doSlurpTransaction(Svn2Scanner.java:177)
	at com.cenqua.fisheye.rep.BaseRepositoryScanner.ping(BaseRepositoryScanner.java:85)
	at com.cenqua.fisheye.rep.BaseRepositoryEngine.doSlurp(BaseRepositoryEngine.java:92)
	at com.cenqua.fisheye.rep.RepositoryEngine.slurp(RepositoryEngine.java:408)
	at com.cenqua.fisheye.rep.ping.IndexingPingRequest.doRequest(IndexingPingRequest.java:25)
	at com.cenqua.fisheye.rep.ping.IncrementalPingRequest.doRequest(IncrementalPingRequest.java:30)
	at com.cenqua.fisheye.rep.ping.PingRequest.process(PingRequest.java:58)
	at com.cenqua.fisheye.rep.RepositoryHandle.processPingRequests(RepositoryHandle.java:228)

Is there any fine tuning that I can do via configuration of the repository?

Thanks in advance.

Best regards,

Marco

4 answers

Hi Nick,

thanks for your reply. I started indexing from a higher changeset and solved the problem in this way.

BTW, changesets that count 10K files, as an order of magnitude, are still slow but Fisheye processes them without major issues.

The problem with 300k files changesets is that CPU is almost constantly at 100%, suggesting me that com.cenqua.fisheye.svn.util.ChangePathUtil.isSubPath(..) is working on in-memory objects. Also, heap occupation grows and exceeds -Xmx1g settings (no user activity). This makes me suspect that com.cenqua.fisheye.svn.util.ChangePathUtil.isSubPath (or the code block that invokes it) could be further optimized.

Slow indexing because of poor IO is not currently a major concern for me, what's more important IMHO is not having a direct dependency between changeset size and heap occupation.

Thanks again, regards.

Marco

I found out that this is happening while processing a huge changeset (300k files).

Hi,

There are some configurations that can be done to tune your FishEye instance.

They are described here:

https://confluence.atlassian.com/display/FISHEYE/Tuning+FishEye+performance

Please give it a try. :D

0 vote
Nick Pellow Atlassian Team Jul 01, 2013

Hi Marco - 300k files is indeed a huge changeset.

Has FishEye finished processing this changeset now and your CPU cooled down a little ?

Further, indexing will be much faster for you if you can use svnsync to sync your svn repo local to the fisheye server.

Cheers,

Nick

Suggest an answer

Log in or Sign up to answer
How to earn badges on the Atlassian Community

How to earn badges on the Atlassian Community

Badges are a great way to show off community activity, whether you’re a newbie or a Champion.

Learn more
Community showcase
Published Monday in Jira Software

How large do you think Jira Software can grow?

Hi Atlassian Community! My name is Shana, and I’m on the Jira Software team. One of the many reasons this Community exists is to connect you to others on similar product journeys or with comparabl...

442 views 6 11
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you