Repository index failed due to error (malloc)...on a GIT respository

[Jonathan Belanger]

I have a couple of repositories that are getting the same error.  It appears that Fisheye does not like something about a .doc file that it thinks is not a Word file, but possibly a Rich Text Format file.  I have put in an exclude on the entire folder, as well as all items with a .doc extension.

Here is the error information:

Repository index failed due to error com.cenqua.fisheye.rep.RepositoryClientException: com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: com.atlassian.utils.process.ProcessException: While executing: "D:\RoRPrograms\Git\bin\git.exe whatchanged --always --reverse --date-order -m --no-abbrev --no-renames e80a314e0f115de6bd84f4fa31f0edf5686162cd..af2fe82d2733eb22a745bbabd4eaef71d25ce1df --pretty=format:C:%H%nP:%P%nA:%aN%nE:%aE%nR:%cN%nF:%cE%nD:%at%nS:%s%nB:%b%n@@fe_body_end@@ " in [D:\FisheyeData\var\cache\ImageProcessing\clone] com.atlassian.utils.process.ProcessException: com.atlassian.fisheye.dvcs.handler.DvcsProcessRuntimeException: com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: d:\TEMP/eFtzMa_Version 6 Eclipse Build and Release Procedure.doc is not a Word Document. It is probably a Rich Text Format file d:\TEMP/eFtzMa_Version 6 Eclipse Build and Release Procedure.doc is not a Word Document. fatal: Out of memory, malloc failed (tried to allocate 310979025 bytes) com.atlassian.fisheye.dvcs.handler.DvcsProcessRuntimeException: com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: d:\TEMP/eFtzMa_Version 6 Eclipse Build and Release Procedure.doc is not a Word Document. It is probably a Rich Text Format file d:\TEMP/eFtzMa_Version 6 Eclipse Build and Release Procedure.doc is not a Word Document. fatal: Out of memory, malloc failed (tried to allocate 310979025 bytes) com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: d:\TEMP/eFtzMa_Version 6 Eclipse Build and Release Procedure.doc is not a Word Document. It is probably a Rich Text Format file d:\TEMP/eFtzMa_Version 6 Eclipse Build and Release Procedure.doc is not a Word Document. fatal: Out of memory, malloc failed (tried to allocate 310979025 bytes) com.atlassian.utils.process.ProcessException: While executing: "D:\RoRPrograms\Git\bin\git.exe diff --no-renames --no-abbrev --full-index -U0 8c22f07f8570d334940c9705161799eef15aada7..a16db74f1f9ab724dcd82c646510c779ee5fbfbd " in [D:\FisheyeData\var\cache\ImageProcessing\clone] com.atlassian.utils.process.ProcessException: Non-zero exit code: 128

[Jonathan Belanger]

This is the result of the command run on the Crucible server:

D:\FisheyeData\var\cache\ImageProcessing\clone>D:\RoRPrograms\Git\bin
\git.exe diff --no-renames --no-abbrev --full-index -U0 8c22f07f8570d33
4940c9705161799eef15aada7..a16db74f1f9ab724dcd82c646510c779ee5fbfbd
WARNING: terminal is not fully functional
diff --git a/.gitignore b/.gitignore
index
2f2d8f7019e20d88dfa4797dd002233a1235c3ef..0db20818ab1e644f1598af496df5bca9c6b34f5e
 100644
--- a/.gitignore
+++ b/.gitignore
@@ -40 +40,3 @@ InstallUtil.InstallLog
-*.o
+CaptureConsoleUnitTest.opensdf
+#Ignore help temp files
+*.shfbproj_*
@@ -41,0 +44,27 @@ InstallUtil.InstallLog
+#Ignore 3rdParty
+/ek/capture/imaging/common/3rdParty
+#Ignore image files
+*.raw
+*.dcm
+#Ignore dll files
+*.dll
+#Ignore DATA
+/ek/capture/imaging/RibSuppression/Data
+#Ignore auto generated html document
+/ek/capture/imaging/*/*/*/html
+#Ignore mcl files
+/ek/capture/imaging/RibSuppression/*/mcl
:


At that point it scrolls through all of the diffs, at some point during the
scroll, the same error from Crucible/Fisheye does reappear in the text.

Result of git config --list:

core.symlinks=false
core.autocrlf=true
color.diff=auto
color.status=auto
color.branch=auto
color.interactive=true
pack.packsizelimit=2g
help.format=html
http.sslcainfo=/bin/curl-ca-bundle.crt
sendemail.smtpserver=/bin/msmtp.exe
diff.astextplain.textconv=astextplain
rebase.autosquash=true
core.repositoryformatversion=0
core.filemode=false
core.bare=true
core.symlinks=false
core.ignorecase=true
core.hidedotfiles=dotGitOnly
remote.origin.url=ssh://Git@git01:29419/repo.git


When changing the command to include --no-textconv, I did NOT see the error
message pop up during the diff scroll.

[Piotr Swiecicki]

Seems like this option: diff.astextplain.textconv=astextplain causes the trouble. Can you find it in your ~/.gitconfig file and remove it please, then try again? I hope that would help. I've raised an improvement issue https://jira.atlassian.com/browse/FE-5638 to have FishEye ignoring such extension, but in general I would advise to keep standard git client configuration for the user FishEye is run on behalf of. Hope that helps, Piotr

[Matt Preall]

I'm working with Jon to solve this issue. If I remove the lines [diff "astextplain"] textconv = astextplain From the gitconfig file, it crashes because that format is used to diff many different text file types. This is the suggested configuration for .doc conversions everywhere I've looked online as well. When I try to set .doc/.DOC files to diff as binary in the .gitattributes file, I get the following error, which looks similar to the previous one, but this time it does not have any additional information about what is causing the crash: Repository index failed due to error com.cenqua.fisheye.rep.RepositoryClientException: com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: com.atlassian.utils.process.ProcessException: While executing: "D:\RoRPrograms\Git\bin\git.exe whatchanged --always --reverse --date-order -m --no-abbrev --no-renames 6f9d51d6f7568916460bd9c1d69c007ec2cb8419..5740552331105b662d40f87fc3c0c8ff8275ff6c --pretty=format:C:%H%nP:%P%nA:%aN%nE:%aE%nR:%cN%nF:%cE%nD:%at%nS:%s%nB:%b%n@@fe_body_end@@ " in [D:\FisheyeData\var\cache\ImageProcessing\clone] com.atlassian.utils.process.ProcessException: com.atlassian.fisheye.dvcs.handler.DvcsProcessRuntimeException: com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: fatal: Out of memory, malloc failed (tried to allocate 310979025 bytes) com.atlassian.fisheye.dvcs.handler.DvcsProcessRuntimeException: com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: fatal: Out of memory, malloc failed (tried to allocate 310979025 bytes) com.atlassian.fisheye.dvcs.handler.DvcsProcessException: Error while communicating with VCS: fatal: Out of memory, malloc failed (tried to allocate 310979025 bytes) com.atlassian.utils.process.ProcessException: While executing: "D:\RoRPrograms\Git\bin\git.exe diff --no-renames --no-abbrev --full-index -U0 8c22f07f8570d334940c9705161799eef15aada7..a16db74f1f9ab724dcd82c646510c779ee5fbfbd " in [D:\FisheyeData\var\cache\ImageProcessing\clone] com.atlassian.utils.process.ProcessException: Non-zero exit code: 128 I've been trying to figure out how to modify the syntax of the git diff call to add no-textconv, but I don't see where that is set. Thank you for the help.

[Piotr Swiecicki]

Seems like textconv option is still in effect, although you removed it from the configuration as explained. Can you remove whole ~/.gitconfig file as a temporary solution and see if that helps? Ideally you shouldn't interfere with git configuration for the user running fisheye instance. If you need to use some extensions for your other work, perhaps you want to consider using dedicated system user to host fisheye with clean git configuration? As explained earlier I've opened an improvement issue to get FishEye improved so it ignores git configuration settings like this. Cheers,

[Matt Preall]

The above error came when I removed all relevant changes from the gitconfig file. Also, the gitconfig being used for this is essentially the default that comes with the git utility with only the asplaintext stuff added, which, based on my investigation is the standard way to handle git diffs of Word documents. When the asplaintext stuff is in there and the gitattributes file tells the tool to use textconv for .doc/.docx files, I get the error Jon put at the top (d:\TEMP/eFtzMa_Version 6 Eclipse Build and Release Procedure.doc is not a Word Document. It is probably a Rich Text Format file) When I remove the asplaintext info and delete the gitattributes file, which should just cause the tool to perform the standard default diffs for all files, I get the error above, which unfortunately doesn't actually show a fatal cause like the other. My other confusion is that I continue to try and put the folder that this culprit .doc file is located in as an Exclude in the repository but it never seems to work. I want to be able to exclude the entire /Build folder in the base level of the repository from indexing and I've used the following Exclude strings: Build/ /Build/ /Build* /Build/* /Build/*.doc **/*.doc Based on the documentation I've read, each one of those strings should work to exclude the Build/ folder, but each time I get the same error.

[Matt Preall]

A little more information - the gitconfig and gitattributes file we were using that caused the initial error are actually the default files that come with the standard Git for Windows utility. The textconv options are the default method Git uses to handle Word/RTF/PDF diffs.

[Piotr Swiecicki]

Hi @Jonathan Belanger,

It seems like the error is coming from antiword library (see http://repo.or.cz/w/antiword.git/blob/HEAD:/main_u.c#l185), as a result of running 

"D:\RoRPrograms\Git\bin\git.exe diff --no-renames --no-abbrev --full-index -U0 8c22f07f8570d334940c9705161799eef15aada7..a16db74f1f9ab724dcd82c646510c779ee5fbfbd "

command in the D:\FisheyeData\var\cache\ImageProcessing\clone folder.

Can you try running the same command from Windows prompt and see if you get the same error?

Do you have any git extensions installed on your machine?  Can you run 

git config --list

and attach the output (ensure no sensitive data is attached, e.g. your emails etc.)

Finally, if running command locally bring the same output, can you try running same command with --no-textconv option to see if that helps?  I mean 

"D:\RoRPrograms\Git\bin\git.exe diff --no-textconv --no-renames --no-abbrev --full-index -U0 8c22f07f8570d334940c9705161799eef15aada7..a16db74f1f9ab724dcd82c646510c779ee5fbfbd "

Regards,