Rule of thumb for load on SVN with Crucible + Fisheye

[Steffen Breitbach]

Hi everyone!

We're planning to use Crucible + Fisheye but have no idea about the load that this might cause on our SVN server. This is something our SVN admins will want to know ;-)

Is there some kind of rule of thumb that we could use to estimate the additional load on SVN by using Crucible + Fisheye? Something like "each project every ten minutes" or every commit times two?

Thanks in advance!

Steffen

[Norman Abramovitz]

You did not state if fisheye is on the same machine or different machine from SVN. You also did not state how many repositories you have either.

Assuming 1 repository with fisheye on the same machine as SVN.

The Java environment has more effect on the server than fisheye itself. The native vs bundled svn support for me had the greatest effect on system performance depending on the hardware environment you have.

https://confluence.atlassian.com/display/FISHEYE/Native+support+for+SVN

The initial sync will give your SVN some load, but it would be equivalent to someone asking for the logs for every file revision in the respository. You might need to restart fisheye multiple times during the initial sync since SVN seems to lose protocol synchronization.

The incremental syncs just pulls the revision comments from the last sync, so it depends on the method of retrevial (poll or triggered) what type of effects it might have on the server.

You will need to get an idea of how many commits are done on a daily basis and how many commits are actually close it time to each other.

I have found if there was a great deal of commits per minute (more than 5) then polling was better because it avoided having the commit hooks running for every commit. If you are already doing commit hooks then the added logic has communications costs to fisheye.

If the commits are far and few between (two or three commits per 10 mintues) then triggering might be a better choice.

Having the machine poll when no commits are done is generally not an issue but that depends on what else is running on the machine.

[Norman Abramovitz]

Oh, and the real way to know is actually to do some measurements. You might find you need to improve things before adding the new software.

[Steffen Breitbach]

Hi Everyone!

Thanks for your input, that really was helpful when talking to our SVN guys.

Regards
Steffen

[Pablo Beltran]

I never used FishEye, however the following is based on my guesses only.

FishEye seems to use caches extensivenly, so Subversion itself should not be too much overhead by using FishEye except during the initial loading because FishEye fetches all the versioned data:

• all the revisions and their metadata (properties): revision number, author, date and commit message
• all the changed files and directories for each revision
• the file contents to extract the key words (full text search - Lucene).

[I do not know whether Fisheye fetches all the files for each version or only the initial version and all the deltas]

Once FishEye is loaded and synchronized with Subversion, you should not see any overhead on Subversion unless maybe some peak if you make a big commit (you import a large project, for instance).

Despite FishEye can pull the changes every X time (minutes or seconds) or you can configure it to push the changes once that have been commited only, this is not an expensinve task for Subversion. The performance cost would depend on the commit size as I mentioned above.

So a quick and easy answar for your Subversion admin would be: you might see some overhead during FishEye initial loading/synchronization, so try to do it during the weekend, and for the daily basis you sould not see any extra overhead, maybe just the opposite as developers will fetch data from Fishye caches rather than Subversion directly.

Any correction/suggestion/improvement to my guess would be very appreciated.

[Nick]

Hey Steffen,

As well as what Sergety suggests by using a local mirror accessed via file:///, you should also consider configuring a Commit Hook inside the svn repositories so that FishEye can poll far less frequently for new commits. see https://confluence.atlassian.com/display/FISHEYE/Configuring+commit+hooks

Cheers,

Nick

[Sergey Svishchev]

The app polls each repository every X minutes (configurable globally and per-repository) and, if a new commit is found, fetches its metadata and diffs.

Consider creating a replica of repositories you will be indexing -- local filesystem access is always cheaper than remote.