Handling nested macro discovery at scale in Confluence

[Brian Hill]

I'm busy beavering away in a customer cloud migration, where they make extensive use of macro nesting in historical content, know there are plenty of other customers who also leverage this in Server, DC. Others working in this context will understand some of the challenges of this. You're likely already familiar with https://jira.atlassian.com/browse/CONFCLOUD-70746

Atlassian recently released this guidance article: https://confluence.atlassian.com/confkb/migrating-from-confluence-server-data-center-to-cloud-nested-bodied-macros-1209866774.html which posits this as a customer problem and to start dealing with it in pre-migration work. That can and does involve potential transformation of a lot of content, depending on Confluence instance size, content age and other factors. Content age, relevance and analytics naturally plays a role here, along with business prioritisation approaches for legacy content. Quite the challenge.

Previous related posts have noted the need to do this discovery by brute-force, page by page. That's a little impractical in large instances. I'm interested to hear from others who've tackled this programmatically to help deal with the scale. What's worked? What hasn't? What trade-off approaches have you needed to consider?

[Derek White · Accepted solution]

@Brian HillHonestly, we put it on the client.

When nesting is a major problem, it has usually been due to tab macros - macros that let you add tabs and switch between them on the page (e.g. Navitabs).

We provide lists of pages based off (poor) CQL macro searches and they identified critical pages. The critical pages were then checked for nesting and if it was a problem (with this new query, they should all be nested). Most of these were adjusted on Server/DC to migrate without issue, others were updated in a Test Cloud and copied to Prod Cloud. Remaining pages were left up to users post-migration - IT was trained on how to respond to the users.

Given the page content is not something a partner can usually make a decision on, there is not much we can do to directly fix each page.

Our long-term plan is to come up with pathways for various Apps/macros that we can use to advise the customer. We hope to be able to implement some page updates via db/api to handle the simpler macros.

[Mark Ahnell]

The article being referenced in several of these comments (https://confluence.atlassian.com/confkb/identifying-nested-macros-in-confluence-data-center-server-1207190999.html) no longer works.  Does anyone have a copy of the query?

cc: @Giuliano C_ 

[Gabriel Muller]

SELECT ct.contentid, ct.title, s.spacename FROM bodycontent b JOIN content ct ON (ct.contentid = b.contentid) JOIN spaces s ON s.spaceid = ct.spaceid WHERE ct.contenttype IN ('PAGE','BLOGPOST') AND ct.content_status = 'current' AND ct.prevver IS null AND b.body ~ '<ac:structured-macro[^>]*ac:name="([^">]*)"(?:(?!</ac:structured-macro>).)*<ac:structured-macro[^>]*ac:name="([^">]*)"(?:(?!</ac:structured-macro>).)*<ac:(?:rich-text-body|plain-text-body)';

[Fred L]

Recently faced issues like that (well, I think it's related) - where native TOC macro doesn't pick up level heading (h2 elements) generated by a custom macro (ACE based).

https://jira.atlassian.com/browse/MIG-771?focusedCommentId=3241877&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-3241877

More context (screenshots) here: https://support.atlassian.com/requests/JST-841243/

[Giuliano C_]

Hi @Brian Hill

I appreciate for bringing up this important topic!

As the author of the article, after going through your post, I have restricted its visibility until we have a clear version to help understand which nesting combinations will not work in Cloud for migrators.

The reason is that the article itself focuses on any sort of nesting and what we want to do is to ensure that customers understand which combinations still work. Thanks to this post, I see the need to improve the article.

Aligned with these reasons, when you migrate to Cloud, the pages will still remain in the legacy editor (which supports nesting). However, some apps did not build the functionality for the legacy editor as well. The current/new editor, on the other hand, does not support nesting for most apps.

Nested-bodied macros in Cloud

Supporting additional nested-bodied macros is something under consideration and we do understand how important this is for migrators, which is one of the reasons for creating articles like the one above!

Even though we are still improving it, whenever you have questions or concerns tied to migration, support will be more than happy to clarify them before going forward with a production migration.

We are interested to know the opinions of other customers and partners regarding the questions that you posted as well! Thank you again for the opportunity, my friend.

Wish you the best,
Giuliano C.
Atlassian Team

[Derek White]

Can you at least open it up to partner portal or something? I didn't grab the queries before you locked it down - have had a backlog ticket to write queries for a few months ;)

[Giuliano C_]

Fair point, @Derek White!

I remember you from your comment in CONFCLOUD-70746, which is what the new changes for the KB article will be going through. What a small world, my friend.

After reading your question, I've thought about sharing the queries here, but having it on a page with a warning confirming that this is for all nested-bodied occurrences would benefit other customers and partners without actually confusing them, which is why I just created the KB below:

https://confluence.atlassian.com/confkb/identifying-nested-macros-in-confluence-data-center-server-1207190999.html

Can you check it and let me know if it helps? Also, if you have any thoughts about it, I will be more than happy to add them to the KB.

Thank you for the opportunity as well, Derek!

Wish you the best,
Giuliano C.
Atlassian Enterprise Team

[Derek White]

Yeah, I'm all over the place! Co-workers and customers are always telling me they saw a comment of mine somewhere.

---

We have a migration customer in Test phase, right now, that I can try it out on (this week or next). They use PostgreSQL. Our plan is to, over time, compile a problematic Apps/macros list and be able to attack the most problematic macros earlier in the migration.

Will let you know how it goes, what our process was, what we found, etc.

Thanks,
-Derek