Is it possible to create a sitemap of Confluence?

[Tom]

Our confluence is more than a decade old and in that time it’s become a bit of a free for all with no constraints on who can create what and where. We’ve also never had a tidy of it so it’s an unwieldy beast!! Ahead of a move to Confluence Cloud we’d like to properly understand how it’s become structured, so that if there are parts of it that are well organised we don’t lose that and can encourage others to follow a similar directory structure when on cloud. The problem is that building a map of its structure manually will take forever - is there a better way to map it please?

[Tom]

Test

[Radek Dostál]

Not quite sure what your aim is. Who's to say which "structure" is good and which is bad, and what even does structure mean.

You can generate page trees w/ a macro, and you could, given some tweaking with javascript, be able to automatically expand all dropdowns in content tools view. But I don't think Confluence gives any such a "structure" view that easily otherwise, the macro is probably the closest to it but even that is an overhead and is limited to levels.

What I don't see is, even if you did manage to get it, how that would be of any use. Who is the arbitor of which structure is right and who's to say a page outside this "desired structure" is not being used and/or is not a critical component of a Space.

[Tom]

Thanks. The page tree macro you mention sounds like a step in the right direction- does it have a name please?

Our CTO has already designed a high level structure for how he’d like to see our new cloud confluence organised. Right now we have so much content we can’t even see if anyone at all organises content in this way, or if perhaps maybe there’s a better way that he hasn’t considered. 

ideally we would have an ‘as is’ sitemap / page tree showing what we currently have and a ‘to be’ sitemap / page tree showing where we are heading next so we can discuss the proposed change at a suitable level of detail 

[Tom]

Let me give you another example to help illustrate why this is needed. As part of the CTOs proposed structure he’s keen that Squads have their own places to put content relating to themselves, but do so in a consistent manner. 

I estimate that we have had maybe 400 squads during the last decade (some are long lived whereas others are short lived). Let’s assume that 50% of them created Squad specific pages in Confluence. Each would have done so largely independently. That’s 200 squads page structures we need to a) locate b) review and c) consolidate.

For example…

Squad 1 might have pages called:

who we are

how to engage with us

meet the team

our expertise

Squad 2 might have pages called:

meet the team

onboarding

how we operate

planning cycles

You can see two different approaches being taken, both equally valid. With visibility of just those two sources you could propose that a Squads pages should consist of:

who we are

our expertise

how we operate

meet the team

onboarding

how to engage with us

planning cycles

(There might be duplication or consolidation opportunities in there - but having a proposed ‘to be’ structure will spark debate, which is needed)

We need visibility of what’s there so we can create ‘best of’ structures like this but at a large scale. 

[Radek Dostál]

https://confluence.atlassian.com/doc/page-tree-macro-163414255.html

Which has "Expand all" as well in the options.

You would need to do this per space (which could be automated w/ a cURL or REST to update the page and switch the space key). The storage format contains

<ri:space ri:space-key="{{ SPACEKEY }}" />

which is easy enough to replace and post as a page update. You can place this macro on any page really, ideally an empty one so that the content can then be read/stored/parsed, so this would be one way to get there.

But still, it's overhead, probably kind of annoying and could take half a day to automate and build a map out of, not exactly.. elegant, quick, or good, but, that's probably the closest to getting it out of box if we're not talking more complex methods.

It would be possible - and be more efficient - to create a user macro which would iterate over all spaces, build the map, and print it all at once on a single page. That would take a bit of coding though.. Theory is "Macros can also access objects available in the default Velocity context" (source: https://confluence.atlassian.com/doc/user-macro-template-syntax-223906130.html), which in turn means the possibility to use $spaceManager and $pageManager (source: https://developer.atlassian.com/server/confluence/confluence-objects-accessible-from-velocity/), both of which can be used to get and iterate over the pages the build a tree.

I don't have any code at hand nor do I see any through a few searches though, but all in all this would probably be the most optimal approach, but either one of them would take time.

Another option I can think of is SQL, but the db design is such spaghetti I'm writing that one off already, too much work and headache.

Another one would be to crawl through the spaces with REST (relying on content/search), but that's more of a pain than a user macro, plus it takes far more steps.

So overall a few options. Unless somebody else has a better suggestion or already did something like this, I'll take a look at the user macro and see if I can scramble it together next week (if you're not in that much of a hurry).

[Tom]

Thank you so much Radek. If you were able to find time to look at the user macro option that would be amazing!!

[Radek Dostál]

So, I did try.

Go To -> User Macros -> Create User Macro

Visibility: Visible only to system administrators in the Macro Browser (!!!IMPORTANT!!! Otherwise a regular user could create the trees. And this is pretty resource taxing.)
fill out the rest as you see fit, e.g.:
Macro Name: space-trees
Macro Title: Space Trees

Definition of User Macro:
- no macro body

Template:

## Macro title: Space Trees
## Macro has a body: No
##
## Developed by: Radek Dostal (Tietoevry)
## Date created: 24/04/2023
## Confluence version: 7.19.6
## @noparams

## This is deprecated, but (probably) the only way to get to "All Spaces" in a User Macro? Not sure if or how we can inject SpaceService here.
#set ($spaces = $spaceManager.getAllSpaces())

## Remove this to run without limit 1/3
#set ($limit = 5)
#set ($at = 0)

#foreach ($space in $spaces)

    ## Remove this to run without limit 2/3
    #if ($at == $limit)
        #break
    #end

    <h3>${space.getName()} - (${space.getKey()})</h3>

    #set ($homePage = $space.getHomePage())
    #if (!$homePage)
        #continue
    #else
        <a href="$homePage.getUrlPath()">$homePage.getDisplayTitle()</a>
        #set ($children = $homePage.getChildren())
        #if (!$children.isEmpty())
            <ul>
            #foreach ($child in $children)
                <a href="$homePage.getUrlPath()"><li>$child.getDisplayTitle()</li></a>
                #set ($children2 = $child.getChildren())
                #if (!$children2.isEmpty())
                <ul>
                    #foreach ($child2 in $children2)
                        <a href="$child2.getUrlPath()"><li>$child2.getDisplayTitle()</li></a>

                        ## Uncomment below to get more levels, but beware too many could crash Confluence on memory, and will take exhuberant amount of time to build..
                        ## #set ($children3 = $child2.getChildren())
                        ## #if (!$children3.isEmpty())
                        ## <ul>
                        ##     #foreach ($child3 in $children3)
                        ##         <a href="$child3.getUrlPath()"><li>$child3.getDisplayTitle()</li></a>
                        ##         #set ($children4 = $child3.getChildren())
                        ##         #if (!$children4.isEmpty())
                        ##         <ul>
                        ##             #foreach ($child4 in $children4)
                        ##                 <a href="$child4.getUrlPath()"><li>$child4.getDisplayTitle()</li></a>
                        ##                 #set ($children5 = $child4.getChildren())
                        ##                 #if (!$children5.isEmpty())
                        ##                 <ul>
                        ##                     #foreach ($child5 in $children5)
                        ##                         <a href="$child5.getUrlPath()"><li>$child5.getDisplayTitle()</li></a>
                        ##                         #set ($children6 = $child5.getChildren())
                        ##                         #if (!$children6.isEmpty())
                        ##                         <ul>
                        ##                             #foreach ($child6 in $children6)
                        ##                                 <a href="$child6.getUrlPath()"><li>$child6.getDisplayTitle()</li></a>
                        ##                                 #set ($children7 = $child6.getChildren())
                        ##                                 #if (!$children7.isEmpty())
                        ##                                 <ul>
                        ##                                     #foreach ($child7 in $children7)
                        ##                                         <a href="$child7.getUrlPath()"><li>$child7.getDisplayTitle()</li></a>
                        ##                                     #end
                        ##                                 </ul>
                        ##                                 #end
                        ##                             #end
                        ##                         </ul>
                        ##                         #end
                        ##                     #end
                        ##                 </ul>
                        ##                 #end
                        ##             #end
                        ##         </ul>
                        ##         #end
                        ##     #end
                        ## </ul>
                        ## #end

                    #end
                </ul>
                #end
            #end
        </ul>
        #end
    #end

    ## Remove this to run without limit 3/3
    #set ($at = $at+1)

#end

Go to any empty page (e.g. personal space), insert macro and save - this may take a LONG time to save, give it some time to breathe. The code is.. simplistic to put it mildly.

Please do note that - the code above is limited to 5 spaces only, to just demo/confirm if it works. I have no idea if this could crash Confluence on memory, if you removed the limit. It should be fine provided you are not already bleeding RAM before running this, but I would suggest to, if possible, to have some healthy reserve or to test this on non-production first, I really have no clue how memory taxing this is, I don't know how to meter it. If you have a few hundred spaces, it's probably not much at all, but with thousands of spaces I would be worried.

Well this is my first shot at it, there might be a few things to do better and I'm not exactly a pro with velocity besides the basic syntax (would love to make a recursive function for children but this is just quick and it's not that terrible.. yet).