universal-wiki-converter crashes when exporting MindTouch Wiki

Question

I am trying to export MindTouch Wiki 09.08.3 with UWC.

user-mac% ./run_cmdline.sh -v
Universal Wiki Converter 3.13.0

With ignore.mindtouch=true uwc will run sucessfully , but without any exported data:

gregs-mac% ./run_cmdline.sh -e conf/exporter.mindtouch.properties
2011-09-12 10:32:48,628 INFO  [main] - Exporting Mindtouch...
2011-09-12 10:32:48,629 INFO  [main] - Getting page tree.
2011-09-12 10:32:53,769 INFO  [main] - basic authentication scheme selected
2011-09-12 10:32:54,085 INFO  [main] - Getting page content.
2011-09-12 10:32:54,085 INFO  [main] - Getting page tags.
2011-09-12 10:32:54,085 INFO  [main] - Getting page comments.
2011-09-12 10:32:54,085 INFO  [main] - Getting attachment info.
2011-09-12 10:32:54,086 INFO  [main] - Creating output directory: /tmp/exported_mindtouch_pages
2011-09-12 10:32:54,086 INFO  [main] - Writing to file system.
2011-09-12 10:32:54,125 INFO  [main] - Export Complete.

user-mac% ls -al /tmp/exported_mindtouch_pages 
total 0
drwxr-xr-x  2 gregs  wheel   68 Sep 12 10:32 .
drwxrwxrwt  8 root   wheel  306 Sep 12 10:32 ..

With ignore.mindtouch=false uwc will fail with the following error:

gregs-mac% ./run_cmdline.sh -e conf/exporter.mindtouch.properties
2011-09-12 10:38:07,895 INFO  [main] - Exporting Mindtouch...
2011-09-12 10:38:07,896 INFO  [main] - Getting page tree.
2011-09-12 10:38:13,036 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,256 INFO  [main] - Getting page content.
2011-09-12 10:38:13,298 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,421 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,447 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,499 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,539 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,564 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,588 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,612 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,635 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,680 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,703 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,727 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,751 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,776 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,800 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,844 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,868 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,893 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,916 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:13,958 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,011 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,041 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,071 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,107 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,160 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,214 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,251 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,294 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,346 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,376 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,405 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,437 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,480 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,529 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,557 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,580 INFO  [main] - basic authentication scheme selected
2011-09-12 10:38:14,594 ERROR [main] - Problem encountered when making rest call: 404: HTTP/1.1 404 Not Found
2011-09-12 10:38:14,594 ERROR [main] - Problem occured when making rest request to: http://wiki-srv/@api/deki/files/515/info/?authenticate=true
com.atlassian.uwc.exporters.MindtouchExporter$BadResult: 404
	at com.atlassian.uwc.exporters.MindtouchExporter.handleError(MindtouchExporter.java:145)
	at com.atlassian.uwc.exporters.MindtouchExporter.getFileInfo(MindtouchExporter.java:539)
	at com.atlassian.uwc.exporters.MindtouchExporter.getAttachmentParents(MindtouchExporter.java:233)
	at com.atlassian.uwc.exporters.MindtouchExporter.getContent(MindtouchExporter.java:217)
	at com.atlassian.uwc.exporters.MindtouchExporter.getContent(MindtouchExporter.java:206)
	at com.atlassian.uwc.exporters.MindtouchExporter.getContent(MindtouchExporter.java:208)
	at com.atlassian.uwc.exporters.MindtouchExporter.getContent(MindtouchExporter.java:208)
	at com.atlassian.uwc.exporters.MindtouchExporter.export(MindtouchExporter.java:87)
	at com.atlassian.uwc.exporters.MindtouchExporter.export(MindtouchExporter.java:75)
	at com.atlassian.uwc.ui.UWCCommandLineInterface.export(UWCCommandLineInterface.java:95)
	at com.atlassian.uwc.ui.UWCCommandLineInterface.main(UWCCommandLineInterface.java:48)
2011-09-12 10:38:14,596 ERROR [main] - Problem while exporting
java.lang.NullPointerException
	at java.util.regex.Matcher.getTextLength(Matcher.java:1140)
	at java.util.regex.Matcher.reset(Matcher.java:291)
	at java.util.regex.Matcher.<init>(Matcher.java:211)
	at java.util.regex.Pattern.matcher(Pattern.java:888)
	at com.atlassian.uwc.exporters.MindtouchExporter.getAttachmentParents(MindtouchExporter.java:234)
	at com.atlassian.uwc.exporters.MindtouchExporter.getContent(MindtouchExporter.java:217)
	at com.atlassian.uwc.exporters.MindtouchExporter.getContent(MindtouchExporter.java:206)
	at com.atlassian.uwc.exporters.MindtouchExporter.getContent(MindtouchExporter.java:208)
	at com.atlassian.uwc.exporters.MindtouchExporter.getContent(MindtouchExporter.java:208)
	at com.atlassian.uwc.exporters.MindtouchExporter.export(MindtouchExporter.java:87)
	at com.atlassian.uwc.exporters.MindtouchExporter.export(MindtouchExporter.java:75)
	at com.atlassian.uwc.ui.UWCCommandLineInterface.export(UWCCommandLineInterface.java:95)
	at com.atlassian.uwc.ui.UWCCommandLineInterface.main(UWCCommandLineInterface.java:48)

The similar problem is beeing discussed in the following tread:
http://forums.atlassian.com/thread.jspa?messageID=257379663&#257379663

Any ideas to resolve the problem ?

dimitri de grae · Answer

It seems to me that if you write an exporter , it should at least be able to authenticate (basic authentication seems to be a problem) and if possible at least export some pages.

concerning twiki and mindtouch i have seen many recurring questions on these forums but never solutions, it always seems to boil down to "fix it yourself"

Laura Kolker2 · Answer

As far as I can tell there's probably two issues:

1) Some of your attachments that are linked don't exist anymore. The mindtouch module was created off sample data (didn't have access to real data), and so this case seems to trip people up occasionally. So your missing attachments cause Mindtouch to pass back 404 responses.

2) The exporter really should be outputting each file as it downloads it, and just fail out on a page by page basis rather than cause the whole export to fail when it hits this sort of problem. That's my bad, but I don't have time to fix it right at the moment.

Your options:

a) Use the export logs to identify the missing attachments, and clean up the mindtouch content?

b) Update the mindtouch exporter code to handle these situations more gracefully? I'm happy to answer dev questions, but you should start here: uwc dev doc

Cheers,

Laura