Has anyone managed to migrate TWiki/FOSWiki to Confluence 5.X without spending thousands?

[Desiree' Abbott]

My company is consolidating to a single Wiki platform, and my team is coming from FOSWiki (aka TWiki).  We have about 800 pages and I need to preserve links, inline images, attachments, formatting, and page hierarchy.  Theoretically the UWC is supposed to do this but it is no longer supported and doesn't work with Confluence 5 - it completely breaks when there is any HTML on the page.  We're a small team so we can't afford to pay thousands to tens of thousands for this service.  

Any ideas would be GREATLY appreciated.

[John Kern · Accepted solution]

My company went through this.  We consolidated 3 different TWiki's into Confluence.  It was a massive undertaking dealing with 30,000+ pages.

UWC gets you started.  But it was NOT enough.  And you are right - it crashes all of the time.  I realized there were lots of broken links, poorly formatted items, and I could identify what was going to cause it to crash.

So I wrote a PERL parser that took the TWiki files and cleaned them up before I ran it through the UWC. It only worked on one WEB at a time.  After each new WEB I imported, I would go back and tighten my rules and add new items that found that it did not format well.  Lots of regular expressions.

The Parser:

1) The output of my parser had three possible directories: NoChanges, AutoCorrect, Review.

2) I would then spend time on the Review directory and hand edit those files and/or import them and watch UWC crash as I imported into a TEMPORARY space.

3) Once I was happy all of my TWiki to Confluence files, I would import into new SPACE.  I would then  review and possible hand edit some of the remaining files.  Mainly hyperlinking files from new space to other already created spaces.

Yea - Lots of work.  But we ended up with a very good product.

Good luck.

[Tom Mac]

Could you please share this PERL parser?

[Normann P_ Nielsen _Netic_]

I would like the parser too 

[Andrew Clark]

Anyone still need help migrating Foswiki to Confluence? I volunteered to migrate 400 pages to Confluence and in the end wrote my own Java code to do it.

Yes I encountered all the issues mentioned above and more. For example, Foswiki supports Base64 embedded images but Confluence silently ignored them. I've completed my task now but the code is still in a good state to be modified accordingly.

The way I wrote the app was to utilise space properties to maintain migration state which works really well. Especially when you have multiple "releases".

I recommend any migration should go to a test server/space first so that you can effectively test it as you go along. The code for debugging reasons also creates a page that contains an index of migrated pages including the original URL and destination URL and includes logging for each page including warnings of links and anchors so that you know which pages need more testing.

As mentioned before the migration has been complete now for a few months and was an overall success.

[Andrew Clark]

Oh and unlike the solution mentioned previously I simply used the rendered Foswiki HTML as my page source. As such it pretty much page scraped and ended up in Confluence as HTML too.

[RyanM]

@Andrew Clark I'd be very interested in your converter, I've been asked a similar question and am scoping the challenge as we speak. Please let me know if you can help me with writing this code, I'd be most appreciative! Kind regards, Ryan

[Andrew Clark]

Hi @Ryan Meintjes I certainly can help except I wouldn't want to go through and rewrite what I'd done. If you have foswiki pages accessible to me I could run through some of them with the converter to see how it goes if you like? Otherwise how it's written is a lot of analysis of the Foswiki page structure, understanding the confluence API and resolving issues as they occur.

[RyanRoRo]

Hi @Andrew Clark & @Ryan Meintjes

I think you tagged the wrong Ryan. Don't blame you because the user search is really bad on this platform

Tagging the other Ryan in so he can see your reply. Have a good day

[Tracy Lau]

Hi John,

I am currently working on a TWiKi to Confluence migration project and come across lots of issues. The web I am working on involved 1500+ of pages and attachments. My issues include:

(1) Hundreds of  files failed to convert by UWC. Some of those can be converted after I removed all the html tag or curly brackets.

(2) Those was converted, are either in a chuck of raw text which need to be reformatted by hand or with lots of broken links.

All these involved manual fix and reformat which is very time consuming.

I am interested in your PERL Parser. Would it be possible to share it with me ?

Many thanks,

Tracy

[Brian M Thomas]

Desiree:

This may be far too late to help you, but I've just seen an example of this, and the problem was with the html macro that is generated.  By default, the macro is disabled, and so the macro invocation tag is output verbatim and the HTML content in its body is passed to the wiki markup converter, which ignores all of the tags.

If this is your situation, you must omit the html macro and substitute the equivalent "safe" macro equivalents for each HTML tag for which Confluence has one, passing any attribute settings as parameters.  Links, which don't have an equivalent, can be changed directly to wiki link markup. That could be done in the output formatter if you're up to that, or take the easier path and script it.

Then again, if 800 pages is all you've got, it may well be easier to edit them by hand...