nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

UWC can't upload pages to Confluence. Fails on internal null pointer exception

Doug Schmidt

I'm having a horid experience using the UWC to attempt to import our company's FogBugz wiki into Confluence. I'd really appreciate some pointers / tips on how to import pages that are already in Confluence markup format.

<rant>

Every cryptic failure I've hit makes the UWC look more and more like an incomplete, unprofessional, unsupported product. It feels like Atlassian couldn't care less about winning new business and solving actual customer problems.

There really needs to be basic quick start guide with examples to get newbies going. The current documentation of the UWC is a mess. It is half-baked. It hints that it will describe things in detail, but the links to the hints you need go nowhere, or are not written from a new users point of view (which, BTW, are the *only* type of users for UWC: ie. People who just want to use the tool once to get their data into Confluence! Grrr!)

</rant>

Since FogBugz is not a supported wiki, I wrote a Ruby script to create the "exported" wiki according to this UWC page

My exported wiki has about 400 pages (stored as UTF-8 text files in Confluence Wiki Markup), and some pages have attachments. We don't care about importing the history of each page. Just the current status of each page is fine by us.

somePath/mywiki/page1.txt
somePath/mywiki/page1/attachment1.ext
somePath/mywiki/page1/attachment2.ext
somePath/mywiki/page2.txt
somePath/mywiki/page3.txt
somePath/mywiki/page3/attachment1.ext
somePath/mywiki/page4.txt
...

I've configured my confluenceSettings.properties file to point to my empty Confluence space, and have tested the connection successfully, from both the UWC GUI and from the command line.

Properties of interest from my confluenceSettings.properties file:

wikitype=NoSyntaxConversions # Because *.txt are already in Confluence markup format
pages=somePath/mywiki
attachments= # Because UWC assumes attachments are stored in a page-relative folder
sendToConfluence=true
uploadOrphanAttachments=false

When I attempt a UWC conversion operation, UWC appears to process each of the 400 .txt files in the pages folder as its own page, since uwc.log shows a "Converting page file: foo.txt" message for each file. So far so good.

But then UWC attempts to start uploading the converted pages to my Confluence space, and UWC fails quickly, after about 4-6 successful page uploads. Here is the relevant content from the final 5 lines of my uwc.log:

2013-04-02 14:04:00,760 INFO [Thread-5] - ::: total time to convert files: 8 seconds.
2013-04-02 14:04:00,762 INFO [Thread-5] - Uploading Pages to Confluence...
2013-04-02 14:04:08,735 ERROR [Thread-5] - Unexpected problem while converting: null
2013-04-02 14:04:08,752 ERROR [Thread-5] -
Conversion Status... NONE

That's it, just a cryptic null exception message. So helpful! But on the console that launched UWC, I see the exception stack trace. Why isn't the exception stack trace part of the uwc.log? Cuz that's what the developer will want to know if/when my questions gets investigated. Here is that stack trace, BTW:

2013-04-02 14:04:00,760 INFO [Thread-5] - ::: total time to convert files: 8 seconds.
2013-04-02 14:04:00,762 INFO [Thread-5] - Uploading Pages to Confluence...
2013-04-02 14:04:08,735 ERROR [Thread-5] - Unexpected problem while converting: null
java.lang.NullPointerException
at java.util.Hashtable.put(Unknown Source)
at com.atlassian.uwc.ui.ConverterEngine.createPageTable(ConverterEngine.java:2112)
at com.atlassian.uwc.ui.ConverterEngine.sendPage(ConverterEngine.java:2014)
at com.atlassian.uwc.ui.ConverterEngine.sendPage(ConverterEngine.java:1719)
at com.atlassian.uwc.ui.ConverterEngine.writePages(ConverterEngine.java:1356)
at com.atlassian.uwc.ui.ConverterEngine.convert(ConverterEngine.java:421)
at com.atlassian.uwc.ui.ConverterEngine.convert(ConverterEngine.java:215)
at com.atlassian.uwc.ui.UWCGuiModel.convert(UWCGuiModel.java:180)
at com.atlassian.uwc.ui.listeners.ConvertListener$Worker.construct(ConvertListener.java:277)
at com.atlassian.uwc.ui.SwingWorker$2.run(SwingWorker.java:110)
at java.lang.Thread.run(Unknown Source)
2013-04-02 14:04:08,752 ERROR [Thread-5] -
Conversion Status... NONE

But there is so much more missing from the UWC log:

Should log the stack trace for any exceptions that occur
Should log the page name for each upload attempt (just like it already does for each page during the converter page), so we can figure out which pages work and which one it is crashing on.
Should log the name of the top-level wiki type, and probably the path of that wiki's .properties file, just to confirm what type of conversions were going to be performed.

So I'm totally in the dark about why the UWC is failing. I'm guessing I'm doing something wrong, but I have no way of diagnosing what that is. Is my filesystem layout wrong? Is my configuration wrong? Is the specific data in one of the *.txt files wrong?

Any guidance would be appreciated.

Accepted answers

Doug Schmidt

Compiled using JDK 1.6, got the minimal pages imported, and then abandoned the conversion process. 300 pages (some of which were stale and didn't need conversion) was just not a big enough peice of content to justify outsourcing the migration. We're doing it manually.

Overall, very disappointed with the realities of using the UWC. It's a very difficult task, and it requires (very unexpectedly) an engagement with a 3rd party company (the authors of UWC) to successfully convert from other formats. We're a small company with a limited budget, and simply could not afford to go further down the rabbit hole for the small amount of critical content.

All comments

Ellen Feaheny [

Hi Doug

If you would like some support on the UWC, please contact us at info@appfusions.com - and we can certainly get you there.

There's ALOT to migrations, since every flavor of migration to Confluence has different tricks - and the UWC is a tool, not the solution. It helps - but not always perfect depending on the content, formats, and flavors of content being moved.

It's just not as simple as it may look - or maybe actually that is what you are figuring out?

We can likely save you alot of time and headaches with some direct support.

Ellen

Doug Schmidt

Yep, I understand that migrations are complex, and I also understand that UWC is only one tool in the complete migration solution. I get that. I've written other tools in our solution (like the Ruby script that takes our FogBugz wiki data and exports the pages in Confluence markup, and exports all the attachments to the filesystem).

So I think I've reduced our particular migration project into a very limited and managable subset. We don't care to migrate the page history, or the user audit trail of who created which page and when. We just want the current FogBugz wiki content in Confluence as close as practical.

All the FogBugz markup (html) has been transformed into Confluence markup, including:

Attachment links and inline images.
Links to FogBugz case numbers have been transformed into JIRA issue macros (our bug DB has already been succefully imported into a JIRA project)
Page hierarchy: Links to other FogBugz wiki pages have been transformed into Confluence [pageTitle] links

Our entire exported wiki is on the small side:

280 pages, stored as UTF-8 text in Confluence markup. 940 KB total.
310 attachments (82 MB in total, most are small PDFs or PNGs)

It seems like this type of constrained problem should be something that the UWC should handle quite well.

There is an existing converter (NoSyntaxConversions) which seems to imply it will do what we need (ie. the page is already in Confluence format and needs no syntax conversions).

I'm not 100% sure that my page-specific attachments are stored in the location UWC expects it (uploads are failing after the first 6 pages, none of which had any attachments), so maybe I need to store attachments elsewhere for UWC, or configure my master settings file a bit differently, but I'm not seeing any error messages or other guidance about that.

So in conclusion, I think my constrained project is something that should be supportable through the forums, and not require a costly "engagement" with a consultancy.

Doug Schmidt

OK, making some progress here now.

I enabled extra debug logging in the log4j configuration by uncommenting the line:

log4j.logger.com.atlassian.uwc=DEBUG

Now I can see each attempt to upload the page.

It also appears that the suggested file system layout proposed in the UWC wiki is not correct.

I moved all the attachments to a separate folder, and dropped the ".txt" file extension from each page file.

The new file system structure I am using is:

somePath/mywiki/pages/page1
somePath/mywiki/pages/page2
somePath/mywiki/pages/page3
somePath/mywiki/pages/page4
somePath/mywiki/attachments/page1/attachment1.ext
somePath/mywiki/attachments/page1/attachment2.ext
somePath/mywiki/attachments/page3/attachment1.ext

Now all the 280 pages are able to be uploaded (10 markup errors detected. I can patch those).

None of the attachments are being uploaded though, so I'll have to figure that out still.

But at least there is some progress.

Ellen Feaheny [

Doug -

Pleasure to meet you.

Hmm - interesting perspective.

Here's a few points from our engineering and why it's not a simple answer - and never is with migrations (and who knows what else is under the covers..) -

1) FogBugz is not in the list of supported wikis. Could it evolve to work with it, likely - but not a guarantee out of the box. So not a good assumption.

2) On NPEs, well - we try to put in some helpful things for a dev - sorry that it is different than what you expected. Hard one.

3) One thing - you are not using the latest code - which is here:

https://bitbucket.org/appfusions/universal-wiki-converter/commits/272b4014fd9171da2df1a9ef7e9c53cef7c5d96f

How do we know - b/c line 2112 of the ConverterEngine isn't in the createPageTable method at this time.

4) NoSyntaxConversions is from pre-4.0, so it assumes wiki markup, not confluence xhtml. And it certainly doesn't support attachments. Attachments are handled by a designated AttachmentConverter

We're guessing you imported you content to something that you believe is Confluence xHTML?

There's a property: someprefix.0001.engine-markuptoxhtml.property=false that might help you.

5) Using run_cmdlin.sh - perhaps set which converter properties the UWC is going to be used. For example:

$ run_cmdline.sh confluenceSettings.properties converter.NoSyntaxConversions.properties

https://migrations.atlassian.net/wiki/display/UWC/UWC+Command+Line+Interface for details

Oh, and the link your pointed to is an example recommendation for development architecture. It's not user documentation, and is not intended to be used with a particular converter properties file. It's equivalent to us saying: hey, when you write your exporter, have a sensible file structure so its easy for you to write something that will parse that.

As for other things - likely a bunch, yet consulting a migration and the many trial runs, script tweaking without eyes to the real environment is a lose-lose proposition, we have learned.

info@appfusions.com if you would like to be constructive and get expertise and a fast successful migration.

Otherwise, at only 280 pages, small enough to do manually?

Good luck!

Doug Schmidt

Thanks Ellen, those are very helpful tips.

We are definitely right on the edge of doing stuff manually. Unfortunately, Confluence 4.3.x does not appear to allow you to manually paste their older wiki markup into their online editor. If you typed the wiki markup, char by char, instead of pasting it, then AutoComplete kicks in and everything works. But simple pasting doesn't.

As for the imported content, it is in the form of Confluence Wiki Markup, not Confluence xHTML (which I think is also called Confluence Storage Format?).

---

To be super clear, I think AppFusions is doing fine by providing UWC tool (with source, yay!) and building a business model around migrations.

What I object to is Atlassian's delegation of this (admittedly hard) problem to a 3rd party and then implying to potential customers (during their sales cycle), that migration is covered because UWC exists.

As an evaluating user, when you read all the Atlassian-written copy about UWC (and their site has tons of references to UWC), everything is written as if the migration will be straightfoward, when in fact the only truthful statement about migrating stuff into Confluence should really be "Prepare to spend some money outsourcing your migration to AppFusions. You might get lucky and be able to use a canned migration, but probably not."

But to all the non-developers at our company who looked at UWC during the decision process to move to the Atlassian platform, they all thought migration would be easy enough. It would have been nice to plan for this migration in a more realistic manner.

Ellen Feaheny [

We are working on a Confluence Migration Toolkit -

Simplified from the current UWC.

We'll see if this helps the situation going forward. It is tricky problem.

Contact info@appfusions.com for more info.

Best,

Ellen

Doug Schmidt

Now that unit test compiles, but I now get different errors:

$ ant
Buildfile: c:\Sites\universal-wiki-converter\build.xml

init:

compile.module.uwc.production:
[javac2] c:\Sites\universal-wiki-converter\build.xml:156: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac2] Compiling 626 source files to c:\Sites\universal-wiki-converter\target\uwc\classes
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\ui\FeedbackWindow.java:296: error: incompatible types
[javac2] method = (State.Type) methodObj;
[javac2] ^
[javac2] required: java.awt.Window.Type
[javac2] found: com.atlassian.uwc.ui.State.Type
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\ui\FeedbackWindow.java:321: error: incomparable types: java.awt.Window.Type and com.atlassian.uwc.ui.State.Type
[javac2] if (method == State.Type.NOTE) {
[javac2] ^
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\ui\FeedbackWindow.java:337: error: incomparable types: java.awt.Window.Type and com.atlassian.uwc.ui.State.Type
[javac2] if (method == State.Type.STEP) {
[javac2] ^
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\ui\FeedbackWindow.java:341: error: incomparable types: java.awt.Window.Type and com.atlassian.uwc.ui.State.Type
[javac2] else if (method == State.Type.MAX) {
[javac2] ^
[javac2] Note: Some input files use or override a deprecated API.
[javac2] Note: Recompile with -Xlint:deprecation for details.
[javac2] Note: Some input files use unchecked or unsafe operations.
[javac2] Note: Recompile with -Xlint:unchecked for details.
[javac2] 4 errors

BUILD FAILED

c:\Sites\universal-wiki-converter\build.xml:156: Compile failed; see the compiler error output for details.

Total time: 4 seconds

Doug Schmidt

Latest UWC from BitBucket doesn't compile from source.

Doug.Schmidt@DOUG-VM2008 /c/Sites/universal-wiki-converter (master)
$ ant
Buildfile: c:\Sites\universal-wiki-converter\build.xml

init:

compile.module.uwc.production:
[javac2] c:\Sites\universal-wiki-converter\build.xml:156: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac2] Compiling 626 source files to c:\Sites\universal-wiki-converter\target\uwc\classes
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\converters\IllegalPageNameConverterTest.java:480: error: unmappable character for encoding Cp1252
[javac2] input = "Detta èr en sida med îèÜ och ?Çà";
[javac2] ^
[javac2] 1 error

BUILD FAILED

c:\Sites\universal-wiki-converter\build.xml:156: Compile failed; see the compiler error output for details.

Total time: 1 second

I don't particularly care if that unit test passes, so I just changed that input string to "foo".

Doug Schmidt

Ellen Feaheny [

Hello Doug

There is no Atlassian backing or support on the Wiki Migration process, at all - we have endlessly been trying to get some level of firm support on this, as a priority, from Atlassian since June of 2011.

We believe it would also be great for their sales, having both internally and externally aligned owners. Migrations are high risk, requiring very controlled processes, repeated iteration, and solid technical and consultative skills to do well for clients.

Experience matters on these for sure, when it will impact 100s/1000s of users!
(We know this for sure!)

Atlassian's business model does not have in-house professional services, at current.

Now I will tell you the backstory, and a digression.

As you can see on the Marketplace listing, AppFusions didn't have UWC rights to fully take it over - it says "Atlassian Labs". The original developer of it (since 2005) - and consistent maintainer - is a key member of AppFusions team.

For the last 2 yrs, AppFusions provided "open source" / owner-like contributions and many many updates, including the latest one for 4.X - and tons of man hours to progress it forward.

As Atlassian grew, it became clear that the program needed "a program" - not just throwing darts at questions without full environment context. Migrations are complicated.

We have done many flavors of Wiki migrations, as paid gigs - but as free, it is not a wise risk for AppFusions at the technical consult, content translations, or even for our reputation level. This is because, without eyes and better visibility on migration content and environments, the room for error / fail is very high!

When 100s/1000s of pages and users are at stake, this is not a consideration to take lightly. Managed controlled migrations - like any data migration - are critical for success.

In late Jan 2013, we (AppFusions) finally had a breakthrough:

Atlassian emailed us that we could take UWC over fully, and we were planning to rebrand and start over. Make it a real supportive product. We finally had full PM rights to do so, as well as Atlassian blessing for success, which was most important.

Everything was a go... but... THEN suddenly - completely unexpected, in early March 2013:

+ after being requested for a meeting, with positive hints (not negative)
+ and AppFusions' most successful Q4 and Q1 ever on the Atlassian Marketplace (which Atlassian had all stats for too)
+ and record breaking Q4 and Q1 Atlassian sales invoices (and very high NEW customer ratio)
+ and solid strategic integration partner relations with some of the largest systems/brands in the industry that we have/are building Atlassian integrations/solutions for (i.e., IBM, Jive, Box, Google, Dropbox, Alfresco, UserVoice, LingoTek, iRise, MixPanel, Parature, ... and others coming!)
+ as well as our to-play-out-plans on UWC evolutions, that we'd emailed Atlassian about a week or two prior..

... SUDDENLY, Atlassian gave us 30 days termination on our 3yr experts partnership and instantly locked us out of the Atlassian Marketplace.

Now, with the 30-days just passed, our integrations and plugins are no longer available or searchable through the Marketplace. They can be found through "google search".

Given that we have (had) 43 Marketplace submissions (and many others fastly coming..), new and old customers alike are in a bit of a confusion foray on this situation.

Customers ask me why, and honestly, I do not know. The notice only referenced contractual sections that "we can terminate without cause":

+ Maybe Atlassian had lingering upsets that I fought hard for the customer on the UWC case (like you are here, Doug).
+ I also fought hard for Atlassian to back up strategic integrations, vs. leave it to the customer to flail around on what works and what doesn't. Integrations are hard enough. And we work with some pretty large corps, and this always comes up. Of course continual reliablity is critical.

Admittedly, we were competitive, yet we tried to do it in right ways i.e., hard work, continually delivering, giving back in Answers and new solutions, solid expertise, and high Atlassian sales for them.

Maybe in the process we pushed too hard, ruffled too many feathers - but I will say this, I and AppFusions are customer advocates beyond belief. Endless passion on this and *trying* to do the right thing for customers.

And our success, to date, is proof of that. You do not have success if you endlessly piss off or be unfair to customers.

AppFusions is looking forward - and the team is solidly in tact to continue our mission of "connecting the enterprise" and providing best cutting-edge packaged solutions and talent professional services.

We have moved our licensing to AppFusions licensing. We will further automate. Many customers are finding us already, via google or other - and evals are picking up again! THANK YOU!

We are also working on a new AppFusions community site so our customers can get deep support for our solutions and we can further share our expertise *across the board*. We have alot, as you have seen by AppFusions folks here on Answers.

Follow @appfusions on Twitter for more/continual news.

We request Atlassian customer (YOU!) support - directly (joining our community when opens, or email us at info@appfusions.com), or tell Atlassian if you feel strongly about the efforts to eliminate us from the community. (In fact, I hope this post is not deleted - for being honest!)

We'd LOVE to work out differences with Atlassian - if they will consider (please).
For the customers, a joint solution should happen. We are reasonable people and fair people.

Many of you have known the AppFusions team on Answers - we are a very strong technical team. In addition to our large plugin and integration work, we also perform ALL levels of professional services. There really is not that much in the Atlassian space that is breaking our technical back with our customers, and we pride ourselves in delivering amazing results for all our customers.

Thanks for your understandings on why it was important to share this note here!

Best,

Ellen

p.s.

My co-founder, Patrick Li released an updated edition of his JIRA Essentials book for JIRA 5.2 yesterday - if interested.

Save yourself some consulting $$s and buy a quick reference book to speed up your learning!

Be the Change
You Seek
~ Atlassian values

Ellen Feaheny | CEO, co-founder

ellenfeaheny (skype) | ellen@appfusions.com
831.247.9417 (cell) | 888.516.2890 (toll free or fax)

AppFusions … Connecting the enterprise.
http://www.appfusions.com

geheitmann

Hi Doug, little late but did you find any solution? Same issue here but even building as JDK 1.6 has no effect.

David Skreiner2

I somewhat agree with the original poster: UWC is not easy to use. The documentation is sparse, and so on ... on the other hand, it quite clearly says "this is not a WYSIWYG tool -- UWC may help you spend less time working on PHP scripts to fix the stuff you want to import". And if, like me, you can't script, then you'll either need to find a programmer for your database migration, or hire consultants e.g. appfusions.

I'm not sure if appfusions is still in business: info@appfusions.comnever replied when I emailed to ask about the "simplified UWC" they had promised in here, so they may be bankrupt or something.

Peter Brydon

It's just a shame this isn't a maintained piece of software. Recent attempt at even installing the thing on a modern copy of opensuse is just a headache. Ant starts off looking in the wrong place for javac (/usr/lib64/jvm/jre/bin/javac instead of /usr/bin/javac)...

Next after I patch that there are UTF8 unmappable errors. Found out they go away if I add 'encoding="iso-8859-1"' into the <javac2 segment of build.xml.

Lastly the incomparable types argument comes in and leads me here. Of course my package manager doesn't have JDK 1.6. My work is considering using Confluence if we can import our content properly as an alternative to MediaWiki but it's a little disappointing when the mere installation procedure of a helper program that may not even work fails.

Why is UWC even listed on the admin page again?

Peter Brydon

Next after I patch that there are UTF8 unmappable errors. Found out they go away if I add 'encoding="iso-8859-1"' into the <javac2 segment of build.xml.

Why is UWC even listed on the admin page again?

John Moore6

Hello Atlassian,

how far would you go to make Confluence attractive. If I see this article being not solved over 2 years, I slowly loose my attention to work with Confluence. In most cases people are using other Wiki's and want to export content into Confluence, to see how fit this product their needs.

Could you please re-touch this simple leaks and provide professional guidance.

Rg,

Confluence lover

John Moore6

Hi Doug,

its quite long time since last post.

I just want to know how to build the new uwc. I get the same error as like you above in your post.

I am migrating from Twiki to Confluence and having hierarchy issues. Thus I want to do some changes and rebuild the uwc.

Can anyone please help.

Doug Schmidt1

UWC is total abandonware by Atlassian in my opinion. Just a blatant marketing lie so that Atlassian can claim "Yep, you can import your exisitng data". Technically correct (the best type of correct!), but totally misleading.

UWC is maintained by one developer at AppFusion (as best I can tell from the source logs). And looking at the change log, it is one developer's sandbox project with little oversight. It is not ready as an open source project for others to take / fork / improve. It is "open source" in that I can see the code, but there is no community around the project, nor any real support (AppFusion has made that explicitly clear in this thread). It is that one developer's project, whose source code happens to be posted on the internet.

You will likely need to cough up at least $20K to pay the AppFusion company to do the migration for you. Anything less and their economics don't make sense.

My company decided not to throw good money after bad. We only had 300 pages to import from a FogBugz wiki, and only 40% of the FogBugz pages were still relevant. So we had funky script just import our 120 relevant pages, losing all markup and images in the process, and manually corrected things by hand. That was the least bad way forward for us.

If we hand an active, thriving, multiple-thousand page wiki to import, then paying AppFusion to import our wiki might have made sense.

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

confluence

cloud

confluence-cloud

universal-wiki-converter