HTML to Wiki Markup: Since Jira 9.10 nekohtml library not found by atlassian renderer

[Gabriele Talarico]

Hi All,

A method that worked up until Jira 9.9 no longer works. The error occurs from Jira 9.10 onwards.
The method is as follows:

public String fromHtmlToWiki(String txtHtml) {
         DefaultWysiwygConverter converter = new  DefaultWysiwygConverter();
         return converter.convertXHtmlToWikiMarkup(txtHtml);
}

This method throws a NoClassDefFoundError on the nekohtml library:

java.lang.NoClassDefFoundError: org/cyberneko/html/parsers/DOMFragmentParser
at com.atlassian.renderer.wysiwyg.converter.DefaultWysiwygConverter.convertXHtmlToWikiMarkup(DefaultWysiwygConverter.java:346)

going to see the jira folders atlassian-jira/WEB-INF/lib in versions prior to 9.10 there were both jars:
 -    atlassian-renderer-9.0.1.jar
 -    nekohtml-1.9.19.jar
while from jira 9.10 the nekohtml jar no longer exists in lib folder but exists in atlassian-bundeld-plugins folder.
I think this causes the above error.

How to solve it? Thank you!

[Stefan Aschendorf]

Same issue here and a fix:

we put nekohtml.jar into /opt/atlassian/jira/lib/

after that, xerces is missing. get xerces from apache and put it in the lib folder too.

then restart jira

[Muhammad]

Hi @Stefan Aschendorf thanks for sharing the trick, can you please share little more details on this

are both jar files and from where you downloaded those

[Stefan Aschendorf]

Hi Muhammad,

yes, both are jar's and you find all here: https://nekohtml.sourceforge.net/

[Slava Dobromyslov]

To make atlassian-renderer-9.0.1 work in Jira 9.10+ you don't need to add anything to /opt/atlassian/jira/lib/. Nekohtml is provided by OSGi container in Jira. You can find the system plugin named Neko HTML. This plugin exports package org.cyberneko.html.parsers to the OSGi environment.

So, the main reason is that atlassian-renderer can't load the nekohtml dependency from OSGi.

You should set scope to compile so that the renderer get bundled into your plugin. After doing so it will successfully resolve the required class via OSGi.

Add it to your plugin's pom.xml like this:

<dependency>
    <groupId>com.atlassian.renderer</groupId>
    <artifactId>atlassian-renderer</artifactId>
    <version>9.0.1</version>
    <scope>compile</scope>
    <exclusions>
        <exclusion>
            <groupId>*</groupId>
            <artifactId>*</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Make sure that version 9.0.1 is provided by your Jira. If it differs then change the version to provided by Jira.

And yes, you have to exclude all the transitive dependencies to avoid OSGi errors related to different class versions in your plugin and provided by OSGi.

IMPORTANT NOTE: If you set this dependency scope to 'compile' you will not be able to correctly instantiate V2RendererFacade and use in 'DefaultWysiwygConverter.setWikiStyleRenderer' to make 'DefaultWysiwygConverter.convertWikiMarkupToXHtml' work. It will happen due to difference between renderer classes/inteffaces compiled into your plugin and the same classes provided by OSGi:

@Bean
V2RendererFacade v2RendererFacade(
  RendererConfiguration rendererConfiguration,
  LinkRenderer linkRenderer,
  EmbeddedResourceRenderer embeddedResourceRenderer,
  Renderer renderer,
  UnicodeEscapingTool unicodeEscapingTool
) {
  new V2RendererFacade(
    rendererConfiguration,
    linkRenderer,
    embeddedResourceRenderer,
    renderer,
    unicodeEscapingTool,
    true
  )
}

[Robert Varga]

This was useful, thanks!

[Peter-Dave Sheehan]

I built my own groovy class to help with that. It may not work in all use cases, but it might be a starting point for yours.

import groovy.util.logging.Log4j
import org.apache.log4j.Level
import org.jsoup.Jsoup
import org.jsoup.nodes.*
/**
 * This is quickly put together class to convert HTML (typically from an Insight/Asset object rich text attribute)
 * to a jira wiki string (to be used in comments).
 * Not all HTML tags are supported.
 * This was implemented because DefaultWysiwygConverter.convertXHtmlToWikiMarkup stopped working in Jira 9.12.0
 * https://community.atlassian.com/t5/Jira-Software-questions/HTML-to-Wiki-Markup-Since-Jira-9-10-nekohtml-library-not-found/qaq-p/2553062
 */
@Log4j
class HtmlWikiConversionUtil {
    StringBuilder wiki
    Document doc
    String convertHtmlToWiki(String html){
        wiki = new StringBuilder()
        doc = Jsoup.parseBodyFragment(html)
        doc.body().childNodes().each{
            processChildNode(it)
        }
        wiki.toString()
    }

    void processChildNode(Node node, Integer listDepth=0, String listChar=''){
        if(node instanceof TextNode) {
            wiki << node.text()
            return
        }
        switch (node.nodeName()){
            case 'p':
                wiki << "\n\n"
            break
            case 'ul':
                listDepth++
                listChar = '*'
                break
            case 'ol':
                listDepth++
                listChar = '#'
                break
            case 'li':
                wiki << "\n${listChar*listDepth} "
                break
            case 'strong':
                wiki << "*"
                break
            case 'em':
                wiki << '_'
                break
            case 'u':
                wiki << '+'
                break
            case 'a':
                wiki << '['
                break
            case 'br':
                wiki << '\n'
            default:
                log.warn "${node.nodeName()} tag not currently supported. Only text will be kept out of this HTML: ${node.outerHtml()}"

        }

        node.childNodes().each{
            processChildNode(it, listDepth, listChar)
        }

        // close the tag/markup
        switch (node.nodeName()){
            case 'p':
                wiki << "\n"
                break
            case ['ol', 'ul']:
                listDepth--
                break
            case 'strong':
                wiki << "*"
                break
            case 'em':
                wiki << '_'
                break
            case 'u':
                wiki << '+'
                break
            case 'a':
                def url = node.attr('href')
                wiki << "|$url]"
                break
        }

    }

}

[Brent Nye]

Peter-Dave, thank you for contributing this. It looks like a great solution, but I'm having trouble figuring out how to leverage it in a ScriptRunner Behavior. Sorry to be needy, but any tips? Sincere thanks.

[Peter-Dave Sheehan]

@Brent Nye can you give me a bit more context on what you are trying to achieve in your behaviour?

But here are some generic tips.

1. Make sure your scriptrunner/jira installation is configured to use script roots correctly
2. Create a new file via the Script Editor in a location of your choice and paste my code. The file name must be exactly "HtmlWikiConversionUtil.groovy"
3. Include, at the top of the file a package declaration that matches the path where you installed the file. So if your file is stored as  com/acme/util/HtmlWikiConversionUtil.groovy, the package line will be

   package com.acme.util

4. Now you are ready to import and use the class in a behaviour script:

   import com.acme.util.HtmlWikiConversionUtil 
   
   def assetField = getFieldByName('asset field')
   def desc = getFieldById('description')
   
   def assetObject = Assets.getByKey(assetField.value[0]) //watch it here, there are some context where single values are returned instead of list, also check for empty, and figure how you want to handle multiple
   def assetHtmlText = assetObject.getString('rich text attribute')
   
   def wikiText = HtmlWikiConversionUtil.convertHtmlToWiki(assetHtmlText)
   
   desc.setFormValue(wikiText)

[Brent Nye]

@Peter-Dave Sheehan , thank you for the detailed instructions. I'd obviously not implemented my own classes yet, so your instructions were extremely helpful.

Before upgrading to 9.12, I had a working solution (ScriptRunner behavior) which used 

com.atlassian.renderer.wysiwyg.converter.DefaultWysiwygConverter. Our help desk technicians would select a value from an Assets custom field. The Behavior would grab the Assets object and a specific field value from it, convert that to Jira wiki format, and dump it into the Jira field.

For testing purposes, I put your class script on the root and modified your solution to fit what I had previously. Here's what I tried:

import HtmlWikiConversionUtil

def assetField = getFieldByName('Service Desk Template')
String assetValue = assetField.getValue()

def assets = Assets.search('Name = \'' + assetValue + '\'')
def assetObject = assets[0]
def assetHtmlText = assetObject.getString('Remedy Information')

def wikiText = HtmlWikiConversionUtil.convertHtmlToWiki(assetHtmlText)
def RI = getFieldByName("Remedy Information")
RI.setFormValue(wikiText)

That resulted in the import line being flagged with:

The [HtmlWikiConversionUtil] import is never referenced

And the line where I called your code was flagged with:
[Static type checking] - Non-static method HtmlWikiConversionUtil#convertHtmlToWiki cannot be called from static context

Google helped me realize I hadn't made an instance of that class. I did so as follows:

HtmlWikiConversionUtil htmlWikiConversionUtil = new HtmlWikiConversionUtil()

I'm thinking I shouldn't need to, as I initially used your instructions quite specifically. Either way, I and our help desk are deeply in your debt. Sincere thanks for your excellent solution, which converts bold/strong and bullet/UL tags into Jira wiki markup perfectly!

[Peter-Dave Sheehan]

No, my bad, the instructions were slighly incorrect. 

I had forgotten how the class was implemented (non-static), and wrote the example from memory.

This should work

def wikiText = new HtmlWikiConversionUtil().convertHtmlToWiki(assetHtmlText)

[Peter-Dave Sheehan]

It took me a while working with scriptrunner to understand the difference between static and non-static classes and methods (I don't have any formal software development education).

The short and long of it is that if a class has global variables that are accessed by the methods then you need to create an instance of that class. 

A static method makes no use of class-wide variables (unless those variables are constant/final).

[Muhammad]

Hi @Peter-Dave Sheehan that's great but has some missing tags like table

[Peter-Dave Sheehan]

@Muhammad yeah, that's a starting point.

So far, in the way I have had to use this (copying Asset rich text into Jira wiki rich text field), I've never needed tables.

Feel free to expand on it.

[Peter-Dave Sheehan]

I have since had a need to develop support for table tags for myself, here is how I did it:

import groovy.util.logging.Log4j
import org.apache.log4j.Level
import org.jsoup.Jsoup
import org.jsoup.nodes.*
/**
 * This is quickly put together class to convert HTML (typically from an Insight/Asset object rich text attribute)
 * to a jira wiki string (to be used in comments).
 * Not all HTML tags are supported.
 * This was implemented because DefaultWysiwygConverter.convertXHtmlToWikiMarkup stopped working in Jira 9.12.0
 * https://community.atlassian.com/t5/Jira-Software-questions/HTML-to-Wiki-Markup-Since-Jira-9-10-nekohtml-library-not-found/qaq-p/2553062
 */
@Log4j
class HtmlWikiConversionUtil {
    StringBuilder wiki
    Document doc
    String convertHtmlToWiki(String html){
        wiki = new StringBuilder()
        doc = Jsoup.parseBodyFragment(html)
        doc.body().childNodes().each{
            processChildNode(it)
        }
        wiki.toString()
    }

    void processChildNode(Node node, Integer listDepth=0, String listChar=''){
        if(node instanceof TextNode) {
            if(node.text() != ' ') {
                wiki << node.text()
            }
            return
        }
        switch (node.nodeName()){
            case 'p':
                wiki << "\n\n"
            break
            case 'ul':
                listDepth++
                listChar = '*'
                break
            case 'ol':
                listDepth++
                listChar = '#'
                break
            case 'li':
                wiki << "\n${listChar*listDepth} "
                break
            case 'strong':
                wiki << "*"
                break
            case 'em':
                wiki << '_'
                break
            case 'u':
                wiki << '+'
                break
            case 'a':
                wiki << '['
                break
            case 'br':
                wiki << '\n'
                break
            case ['table','tbody', 'td']:
                listChar =  '|'
                break
            case ['thead', 'th']:
                listChar = '||'
                if(node.nextSibling()?.nodeName() == 'tr'){
                    listChar =  '|'
                }
                break
            case 'tr':
                def firstCell =  node.childNodes().find{!(it instanceof TextNode)}
                if(firstCell.nodeName() == 'th'){
                    listChar = '||'
                }
                wiki << "\n$listChar"
                break
            default:
                log.warn "${node.nodeName()} tag not currently supported. Only text will be kept out of this HTML: ${node.outerHtml()}"

        }

        node.childNodes().each{
            processChildNode(it, listDepth, listChar)
        }

        switch (node.nodeName()){
            case 'p':
                wiki << "\n"
                break
            case ['ol', 'ul']:
                listDepth--
                break
            case 'strong':
                wiki << "*"
                break
            case 'em':
                wiki << '_'
                break
            case 'u':
                wiki << '+'
                break
            case 'a':
                def url = node.attr('href')
                wiki << "|$url]"
                break
            case ['th','td']:
                wiki << "$listChar"
                break
        }

    }
}