FishEye 3.1 stuck at 99% indexing and will not finish

[logiformscore]

After upgrading to 3.1, Fisheye is hanging (with no errors in the log) when re-indexing our existing SVN repo:

<label class="GBRQCM4CCC"> Current Activity:</label>Full indexing in-progress<label class="GBRQCM4CCC"></label>

<label class="GBRQCM4CCC"> Status Message:</label>Expanding paths of changeset 3559

It says its99.9% complete.. but will not finish. New committs are not shown. If I restart fisheye, the last commit will then appear and it then gets stuck again at the same point.

[Chad Barnes]

We found that our indexing operation was hitting the default subversion timeout in fisheye. Single changesets were taking more than an hour to complete (timing out at an hour). This was confirmed by the indexing log (in the UI) showing one operation taking >3600000ms. We increased the timeout to a few hours and got through the repo (slowly).

[Hippo CMS Infra Team]

HI @TomB,

Unfortunately we didn't find a solution. It still happens sometimes with the most recent version and the most recent java versions but not very frequently and only with the file:// protocol. As workaround I temporary switch to the http:// protocol until the indexing is past the problematic revision and then back to the file protocol for the extra speed.

Bart

[Tom Bell]

Thanks. I may try your http workaround as well since increasing the timeout doesn't seem to help.

[Tom Bell]

To @Hippo CMS Infra Team, have you had success tracking down the cause of the indexing timeout?  We're attempting to reindex a local copy of a single repo and it's been stuck at 77% for two weeks now while expanding paths of changesets.

Tom

[Hippo CMS Infra Team]

Now I get I timout after 20 hours :-( I'll raise an issue with atlassian.

[Hippo CMS Infra Team]

I increased the timeout two 2 hours, but now I see the following error every 2 hours (instead of every hour):

2014-04-08 16:28:38,063 ERROR - Problem processing revisions from repo public due to class com.cenqua.fisheye.rep.RepositoryClientException - java.util.concurrent.TimeoutException

We setup our fisheye instance to use a local mirror of our subversion tree from disk with "file:///var/svn-repo". So I'm not sure if increasing the timeout will actually help. It might be some svn locking issue.

For now I've set the timeout to 20 hours and hopefully it will finally process the specific commit (which isn't anything much bigger than normal).

I'll report back when I've more info.

Some more info on our system:

Fisheye 3.3.3

JAVA:

java version "1.7.0_51"

Java(TM) SE Runtime Environment (build 1.7.0_51-b13)

Java HotSpot(TM) Server VM (build 24.51-b03, mixed mode)

SVN repo:

# cat format
5
# cat db/format 
6
layout sharded 1000

# svn --version

svn, version 1.8.5 (r1542147) compiled Feb 13 2014, 00:11:36 on x86_64-unknown-linux-gnu

[Hippo CMS Infra Team]

Thanks! I'll try that.

[Hippo CMS Infra Team]

Any progress on this issue? We're having the same issue..

[francis]

Same here, will file a support issue

[Chad Barnes]

We're seeing similar behavior. Fisheye 3.1.4. Hanging expanding the same changeset.