Fisheye/Crucible - incremental indexing after pulling each commit from Mercurial repository.It lasts for two hours.
Why do we need this?How can I disable reindexing?Why don't we index only the files from the commit?
Fisheye/Crucible Version:4.4.1
Incremental indexing and full reindex are two different things:
* incremental indexing is the thing you'd like to have -> pull newest changes and index new commits only
* reindex means dropping current repository indexes and indexing repository from scratch
In order to see updates on a repository in Fisheye you need incremental index. In order to determine why it takes 2h first of all determine if Fisheye commences incremental indexing or full reindex. You'll be able to find it in debug logs.If it's really reindex then I suppose somebody is triggering that in some way (plugin, rest webhook etc) If it's incremental index then I suggest enabling debug logs and determining which part takes 2h. Is it connectivity issue and Fisheye is having hard time to pull changes up ? Is this a big changeset with massive changes therefore indexing takes its time ? Hard to tell at the moment.The third option that comes to my mind is the fact that potentially you have repository polling set to every 2 hours in Fisheye. This means Fisheye will look for new changes in repository every 2h, effectively it seems like a lot of time for Fisheye to pick up changes but in fact it's not, it's jus the polling period.
Thank you, Grzegorz.
Mercurial - Non-CVS Repository.
Polling Interval - 60s
No user-installed add-ons found in your FishEye instance.
logs:
2017-09-22 18:33:56,572 DEBUG [153:StdOutHandler DPC ] fisheye RepositoryStatus-setMessage - Status change [DPC]: pulling from http://....
...
2017-09-22 18:33:56,572 DEBUG [153:StdOutHandler DPC ] fisheye RepositoryStatus-setMessage - Status change [DPC]: added 1 changesets with 1 changes to 1 files2017-09-22 18:33:56,772 DEBUG [153:StdOutHandler DPC ] fisheye RepositoryStatus-setMessage - Status change [DPC]: (run 'hg update' to get a working copy)2017-09-22 18:33:56,888 DEBUG [IncrPing3 DPC ] fisheye DvcsProcessMonitor-onAfterFinished - Finished process: /usr/bin/hg --config ui.verbose=false pull http://..... took 569ms2017-09-22 18:33:56,889 DEBUG [IncrPing3 DPC ] fisheye DvcsProcessMonitor-onBeforeStart - Starting process: /usr/bin/hg --config ui.verbose=false branches -c2017-09-22 18:33:57,123 DEBUG [IncrPing3 DPC ] fisheye DvcsProcessMonitor-onAfterFinished - Finished process: /usr/bin/hg --config ui.verbose=false branches -c took 234ms2017-09-22 18:33:57,124 DEBUG [IncrPing3 DPC ] fisheye RepositoryStatus-setMessage - Status change [DPC]: Updated from http://....2017-09-22 18:33:57,124 INFO [IncrPing3 DPC ] fisheye BaseRepositoryScanner-ping - processing repository DPC (DPC)2017-09-22 18:33:57,124 INFO [IncrPing3 DPC ] fisheye LifecycleRepositoryScanner-doSlurpTransaction - Starting slurp of DPC (DPC)2017-09-22 18:33:57,124 DEBUG [IncrPing3 DPC ] fisheye RepositoryStatus-setMessage - Status change [DPC]: Indexing has commenced for repository DPC2017-09-22 18:33:57,125 DEBUG [IncrPing3 DPC ] fisheye Timer-output - -> slurp of DPC mem_usage = 401219968 heapsize = 608698368 freemem = 207478400
and then a lot of lines...
2017-09-22 18:35:07,413 DEBUG [IncrPing3 DPC ] fisheye DvcsProcessMonitor-onAfterFinished - Finished process: /usr/bin/hg --config ui.verbose=false annotate -r 1b3fc7713b094f6fbe4df8d484752768ef7439e1 -c -f -l ....cs took 151ms2017-09-22 18:35:07,439 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-3:22760, 3-4:22943, 4-11:22760, 11-14:22913, 14-15:22760, 15-18:32712, 18-23:22760, 23-24:22913, 24-28:22760, 28-29:22913, 29-30:22760, 30-31:25436, 31-33:22760, 33-34:22913, 34-35:22760, 35-36:25436, 36-37:22913, 37-38:25436, 38-40:22760, 40-41:22943, 41-42:22760, 42-46:22943, 46-47:25436, 47-70:22943, 70-71:22760, 71-75:22943, 75-76:22760, 76-78:22943, 78-79:22913, 79-81:22943, 81-82:22913, 82-83:22760, 83-91:22943, 91-103:22760, 103-104:23030, 104-116:23086, 116-117:23030, 117-118:32712, 118-121:23030, 121-122:32712, 122-124:23030, 124-126:22760], hunks=[<5:0, 5:1>, <30:1, 31:2>, <35:1, 37:5>, <37:1, 43:1>], hunksRevId=718562017-09-22 18:35:07,451 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-1:2072, 1-2:25438, 2-3:2072, 3-4:2088, 4-5:2057, 5-8:22224, 8-9:32717, 9-10:28228, 10-14:25438, 14-15:24257, 15-16:53075, 16-17:22224, 17-18:22765, 18-20:24257, 20-21:22765, 21-22:22947, 22-23:23653, 23-25:28228, 25-26:22224, 26-27:2057], hunks=[<11:1, 11:0>, <15:1, 14:0>, <17:1, 15:0>], hunksRevId=718572017-09-22 18:35:07,455 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-39:54472, 39-40:60038, 40-46:54472, 46-47:57607, 47-58:54472, 58-59:60038, 59-100:54472], hunks=[<96:1, 96:1>], hunksRevId=718582017-09-22 18:35:07,459 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-2:53538, 2-3:54490, 3-4:53538, 4-5:54490, 5-7:53538, 7-8:54490, 8-10:53538, 10-11:57053, 11-13:53538, 13-14:60046, 14-16:53538, 16-17:60148, 17-18:60248, 18-19:66655, 19-22:53538, 22-23:54490, 23-24:53538, 24-25:60046, 25-31:53538, 31-32:58192, 32-34:53538, 34-35:56582, 35-36:53538, 36-39:65913, 39-43:53538, 43-76:54490, 76-78:53538, 78-79:54490, 79-92:53538, 92-93:60046, 93-95:53538, 95-96:65970, 96-97:54336, 97-99:59471, 99-100:54336, 100-102:59471, 102-103:54336, 103-104:59471, 104-105:65970, 105-107:54336, 107-108:65970, 108-109:54336, 109-111:59471, 111-112:54336, 112-113:59471, 113-114:65970, 114-115:54336, 115-147:54464, 147-155:57053, 155-156:55516, 156-160:54464, 160-161:60242, 161-162:55516, 162-164:54464, 164-165:58192, 165-169:54464, 169-172:66655, 172-176:58192, 176-177:66655, 177-187:58192, 187-188:56582, 188-194:54464, 194-195:59477, 195-196:54464, 196-197:59477, 197-198:54464, 198-215:59477, 215-217:54464, 217-219:59477, 219-220:54464, 220-227:59477, 227-229:54464, 229-230:58192, 230-231:54464, 231-232:58192, 232-233:54464, 233-234:55516, 234-235:58192, 235-236:54464, 236-253:54490, 253-255:53538], hunks=[<10:0, 10:1>, <241:0, 242:1>, <242:1, 244:1>], hunksRevId=718592017-09-22 18:35:07,467 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-2:2098, 2-3:1900, 3-4:2098, 4-5:2111, 5-8:2098, 8-9:52626, 9-10:32832, 10-11:39831, 11-12:37875, 12-14:39831, 14-15:32832, 15-17:39831, 17-18:37875, 18-19:52626, 19-20:39831, 20-36:68903, 36-37:32832, 37-38:39831, 38-39:52626, 39-40:22773, 40-41:39831, 41-42:52626, 42-43:25218, 43-47:52626, 47-48:25218, 48-50:39831, 50-51:22773, 51-53:39831, 53-54:22773, 54-56:39831, 56-58:22773, 58-59:39831, 59-60:52626, 60-61:22773, 61-62:39831, 62-63:52626, 63-64:2485, 64-65:52626, 65-66:54325, 66-67:22773, 67-69:52626, 69-70:39831, 70-71:52626, 71-72:25218, 72-76:52626, 76-77:28241, 77-79:52626, 79-80:2499, 80-81:39831, 81-82:39869, 82-83:52626, 83-84:39017, 84-85:2557, 85-86:2499, 86-87:39831, 87-88:54325, 88-100:53643, 100-101:58313, 101-102:53080, 102-103:58201, 103-104:39831, 104-108:58201, 108-109:53080, 109-111:58201, 111-112:53080, 112-115:58201, 115-116:54325, 116-120:52626, 120-121:39831, 121-123:52626, 123-125:25218, 125-126:52626, 126-127:58313, 127-137:52626, 137-138:39869, 138-139:39831, 139-146:52626, 146-147:39869, 147-148:39831, 148-149:52626, 149-151:39831, 151-153:52626, 153-154:39831, 154-158:52626, 158-159:39869, 159-160:39831, 160-167:52626, 167-168:39869, 168-169:39831, 169-170:52626, 170-172:39831, 172-174:52626, 174-175:39831, 175-179:52626, 179-180:39869, 180-181:39831, 181-188:52626, 188-189:39869, 189-190:39831, 190-191:52626, 191-193:39831, 193-195:52626, 195-196:39831, 196-200:52626, 200-201:39869, 201-202:39831, 202-209:52626, 209-210:22231, 210-211:39831, 211-218:52626, 218-219:22773, 219-220:39831, 220-223:52626, 223-224:39831, 224-227:52626, 227-228:28241, 228-229:39831, 229-230:52626, 230-232:39831, 232-233:52626, 233-235:39831, 235-236:52626, 236-237:22773, 237-238:39869, 238-241:52626, 241-242:39831, 242-244:52626, 244-245:39831, 245-248:52626, 248-249:22773, 249-250:39869, 250-251:52626, 251-252:39869, 252-254:52626, 254-255:39831, 255-257:52626, 257-258:39831, 258-267:52626, 267-268:39869, 268-269:39831, 269-276:52626, 276-277:23661, 277-278:39869, 278-281:52626, 281-282:39831, 282-285:52626, 285-286:30122, 286-287:39869, 287-288:52626, 288-289:39831, 289-290:39869, 290-291:52626, 291-292:39831, 292-293:39869, 293-294:52626, 294-295:23661, 295-296:39869, 296-297:52626, 297-298:39831, 298-300:52626, 300-301:39831, 301-303:52626, 303-304:39831, 304-313:52626, 313-314:39869, 314-315:39831, 315-322:52626, 322-324:39869, 324-325:52626, 325-326:39831, 326-327:39869, 327-328:52626, 328-329:39831, 329-330:39869, 330-333:52626, 333-334:58313, 334-340:52626, 340-341:28821, 341-347:58297, 347-348:58313, 348-349:39831, 349-350:52626, 350-351:22773, 351-353:52626, 353-354:39831, 354-355:52626, 355-356:53634, 356-357:39831, 357-358:53643, 358-359:53634, 359-360:39831, 360-361:52626, 361-362:53634, 362-363:22773, 363-364:52626, 364-365:25218, 365-370:52626, 370-372:25218, 372-373:52626, 373-374:22773, 374-376:52626, 376-377:22773, 377-378:52626, 378-379:25447, 379-384:52626, 384-386:25447, 386-387:52626, 387-388:25447, 388-393:52626, 393-394:25447, 394-395:22773, 395-396:52626, 396-397:22950, 397-398:39831, 398-399:52626, 399-400:22987, 400-401:39831, 401-402:52626, 402-403:22987, 403-404:39831, 404-405:52626, 405-406:22987, 406-407:39831, 407-408:52626, 408-409:22987, 409-410:22950, 410-411:52626, 411-412:22987, 412-413:39831, 413-414:52626, 414-415:22987, 415-416:39831, 416-417:52626, 417-418:30116, 418-419:23052, 419-420:52626, 420-421:22773, 421-422:22950, 422-423:52626, 423-424:22927, 424-425:39831, 425-426:39869, 426-427:22927, 427-428:39831, 428-429:39869, 429-430:22927, 430-431:39831, 431-432:39869, 432-434:22927, 434-435:22950, 435-436:52626, 436-437:23661, 437-438:39869, 438-439:22950, 439-440:39831, 440-441:39869, 441-442:24264, 442-443:39831, 443-444:39869, 444-445:23052, 445-446:23661, 446-447:52626, 447-448:22773, 448-449:52626, 449-450:22773, 450-463:66269, 463-465:2098, 465-467:1900, 467-468:2098, 468-475:24138, 475-478:1900, 478-479:24138, 479-480:64464, 480-481:28241, 481-482:1900, 482-483:2098, 483-484:1900], hunks=[<142:1, 142:0>], hunksRevId=718602017-09-22 18:35:07,473 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-13:44880, 13-14:62369, 14-18:44880, 18-19:45186, 19-20:62704, 20-23:45186, 23-29:44880, 29-43:45186, 43-44:62369, 44-48:44880, 48-49:45186, 49-50:62704, 50-53:45186, 53-60:44880, 60-73:45186, 73-79:45692, 79-80:62704, 80-81:45692, 81-82:45229, 82-93:45692, 93-94:45229, 94-96:44880], hunks=[<29:2, 29:0>], hunksRevId=718612017-09-22 18:35:07,480 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-3:29250, 3-4:33159, 4-5:29250, 5-6:71677, 6-9:44459, 9-10:29250, 10-11:29851, 11-12:29250, 12-13:71677, 13-14:31838, 14-19:29250, 19-20:71677, 20-23:29851, 23-24:62705, 24-26:29851, 26-27:62705, 27-41:29851, 41-46:31281, 46-48:29851, 48-49:71677, 49-61:29851, 61-62:62705, 62-68:29851, 68-69:31838, 69-70:60461, 70-71:29250, 71-73:31838, 73-74:34505, 74-76:31838, 76-80:34505, 80-83:43666, 83-84:69440, 84-86:43666, 86-87:69440, 87-88:43666, 88-89:65517, 89-90:43666, 90-91:62705, 91-94:69440, 94-96:43666, 96-97:34505, 97-98:71677, 98-104:34505, 104-107:31838, 107-108:29851, 108-109:71677, 109-110:29250, 110-111:29421, 111-112:71697, 112-114:31838, 114-115:71677, 115-119:71697, 119-120:71677, 120-121:31281, 121-123:29421, 123-124:71697, 124-125:31838, 125-126:29421, 126-127:71677, 127-131:71697, 131-132:71677, 132-135:31838, 135-136:71677, 136-138:31838, 138-139:71677, 139-141:71634, 141-142:71677, 142-146:71634, 146-148:31838, 148-151:71634, 151-152:71677, 152-153:71634, 153-155:31838, 155-157:71634, 157-161:31838, 161-172:71677, 172-173:71697, 173-175:31838, 175-176:71677, 176-178:31838, 178-182:71697, 182-184:31838, 184-186:71634, 186-188:31838, 188-190:29250, 190-193:34090, 193-194:71677, 194-196:34090, 196-197:62705, 197-199:44249, 199-201:34090, 201-204:60461, 204-209:71677, 209-214:60461, 214-224:37483, 224-232:37585, 232-233:44459, 233-234:71677, 234-235:44459, 235-236:71677, 236-240:44459, 240-243:71677, 243-250:44459, 250-255:71677, 255-261:44459, 261-262:71677, 262-265:44459, 265-266:62705, 266-271:44459, 271-272:71677, 272-280:44459, 280-281:62705, 281-295:44459, 295-296:62705, 296-310:44459, 310-311:62705, 311-316:44459, 316-317:71677, 317-325:44459, 325-326:62705, 326-340:44459, 340-341:62705, 341-346:44459, 346-347:71677, 347-348:44459, 348-349:29250, 349-350:71677], hunks=[<6:0, 6:1>, <172:0, 173:11>], hunksRevId=718622017-09-22 18:35:07,533 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-3:21752, 3-4:43049, 4-5:21866, 5-6:21752, 6-7:33160, 7-8:21752, 8-9:38163, 9-10:43049, 10-13:21752, 13-15:70491, 15-16:43049, 16-17:50824, 17-18:43049, 18-21:21752, 21-22:71609, 22-24:21752, 24-27:51734, 27-35:21752, 35-36:21866, 36-39:70491, 39-40:43049, 40-45:70491, 45-65:43049, 65-68:43563, 68-82:43049, 82-84:43563, 84-85:43049, 85-101:43563, 101-106:43049, 106-111:34506, 111-120:23600, 120-121:32671, 121-122:28282, 122-128:23600, 128-129:23550, 129-132:28427, 132-135:55024, 135-136:28427, 136-141:55024, 141-142:28427, 142-143:50824, 143-144:29852, 144-146:70491, 146-148:28427, 148-160:50824, 160-162:51734, 162-166:50824, 166-180:51734, 180-183:50824, 183-192:32671, 192-194:65754, 194-208:32671, 208-210:65754, 210-216:32671, 216-217:38163, 217-222:32671, 222-224:38163, 224-235:32671, 235-237:65754, 237-247:32671, 247-249:65754, 249-265:32671, 265-267:65754, 267-273:32671, 273-275:65754, 275-286:32671, 286-313:50824, 313-318:70960, 318-320:50824, 320-325:70960, 325-333:50824, 333-360:70491, 360-362:21752], hunks=[<8:0, 8:1>], hunksRevId=718632017-09-22 18:35:07,535 DEBUG [IncrPing3 DPC ] fisheye DefaultBlameChunksCalculator-applyHunks - Applying hunks; oldBlame=[0-2:71683, 2-3:71610, 3-5:71683, 5-12:71610, 12-14:71683, 14-16:71610], hunks=[<5:0, 5:1>, <14:0, 15:1>], hunksRevId=71864
Grzegorz, how to fix it?
Sorry, I'm not sure what's there to fix.
Name of the thread IncrPing3 DPC indicates it's incremental indexing, so I think Fisheye indexes only new changes.
I don't know how much repository slurp takes on your instance. There should be a INFO logs indicating how much time it's take for Fisheye to index a single set of changes (it may be multiple commits at once)
fisheye BaseRepositoryScanner-ping - done slurp for repository repository (repository) time = 8732 ms
The logs you've linked (DefaultBlameChunksCalculator) are related to a blame calculations. You can turn them off if you really want to but it will impact some of the Fisheyes features.
It looks like you're new here. Sign in or register to get started.