The Devil hides in the details: the importance of migrating Confluence user settings

[#1]

Confluence is more than just a wiki-based platform; it's an example of a complex system - one comprised of multiple interacting components which define the end-user experience, and impact both individual and team productivity. As such, even a seemingly small and innocent change to its existing configuration can negatively affect the work of thousands of users and inflict financial losses for companies. The following blog post will delve into the details of retaining user settings during a Confluence migration/merge/consolidation  - an aspect of the overall configuration that is often overlooked due to its seemingly lower-priority status.

“We have a bigger fish to fry” - why compromising on Confluence user preferences is a bad idea

A typical Confluence migration, especially in regards to enterprise instances, requires a careful examination of the existing configuration and data to ensure its consistency and integrity won't be compromised during the migration process. Unfortunately, frequently teams faced with the Herculean task of a complex migration initiative tend to prioritize certain aspects over others. Individual user preferences are one such case. “Users will manually reconfigure those preferences, we have a bigger fish to fry!” is an attitude one encounters quite often when it comes to Confluence migration cases.

To better illustrate how this seemingly lower-priority Confluence configuration can have a much more significant impact than initially anticipated, I'd like to introduce three main characters- Frustrated Tony (end-user), Overwhelmed Nick (the Confluence admin), and Anxious Joe (IT manager who led the Confluence migration effort). 

Monday morning, the downtime weekend of the Confluence migration is over, everything went smoothly, and the new instance is up and running. Frustrated Tony (who, at the very beginning of this anecdote is not frustrated at all) walks into the office and fires up his computer. Once he opens Confluence, it immediately struck him that something is wrong - the Confluence configuration he has been operating on until Friday evening was no longer available to him on Monday morning. Below is a list of the most common user preferences that can turn your ordinary Tony into a Frustrated Tony, which will quickly impact both Overwhelmed Nick (who will be flooded with user tickets), and Anxious Joe (who will become very anxious realizing the productivity & financial impact of this seemingly small, yet devilish detail):

• Tier I: One time settings - static user settings 
• Email: users subscribe to specific notifications, updates and activity changes via their emails. If the default settings of the Target instance are set to notify all users for all updates, then you get to first deal with a volume of spam worthy of competing with the GDPR fiasco of mid-2018. 
• Editor: autocomplete & auto-formatting settings which impact the overall content creation & editing experience for users, and oftentimes it leads to an avalanche of tickets for Overwhelmed Nick because users perceive this change as a bug, rather than a user setting. 

The chances are that the vast majority of users are unaware of these settings - they have configured them at the very beginning of their Confluence experience; therefore these settings have become such an integral part of their day-to-day Confluence activities that they typically take them for granted. 

• Tier II: Personal preferences - dynamic user settings
• Saved for later: imagine losing your browser bookmarks? How do you restore them? You don't, because you can't - you work slower and less productively, looking deep into your neural-network of fading memories, trying to recover whatever information you have retained about your library of saved Confluence content. The data loss is irreversible, so is the long-term impact on user productivity.
• Watches: Confluence users often subscribe to be notified about a single page, a page tree, or an entire Confluence space. This user preference mostly impacts people who are involved in tracking, and they realize this preference has been lost when they don't receive notifications from their watched spaces for some time. Or, worst case scenario, when they are reminded about a deadline for a specific activity they didn't know existed.

Additionally, there are other seemingly small details about such a migration initiative, which end up causing a whole lot of headaches for everyone involved, including:

• Differences in default settings on Source & Target instances: oftentimes, both the Source and the Target Confluence instance have different default settings configurations. For example, the default user setting on Source has disabled auto-watch email notification, whereas the Target instance has enabled this setting (or vice-versa). Either way, users might end up with flooded email inbox or stop receiving notifications altogether. Both scenarios have potentially negative consequences for users and their daily work.
• Differences in database ID on Source & Target: when migrating multiple Confluence spaces, the IDs of their pages and dependent objects change, which is the root cause of a lot of broken macros, references, and other related dependencies.
• Differences in usernames on Source & Target: this challenge is true for all types of migration initiatives - Confluence, Jira, Bitbucket, etc. When there is a difference between the usernames on Source and those on Target, Confluence cannot resolve these conflicts and comments, mentions, attachments are all listed as anonymous users. This creates a significant problem, especially for companies operating in highly regulated industries, where audibility and traceability are crucial for their compliance. 

Ultimately, this small detail which is often overlooked due to its seemingly low-priority ends up impacting multiple people:

• Frustrated Tony (and all other end-users) who lose their original user settings and need to spend 1-2 hours in restoring whatever they can;
• Overwhelmed Nick (and other Confluence admins) who starts his Monday morning with an overwhelming amount of ticket requests for a variety of problems associated with lost user settings;
• Anxious Joe, who needs to answer to the VP or Product Manager (depending on who is in charge of the budgeting) about the financial losses inflicted by this productivity blocker (if 10 000 users need to spend 1-2 hours restoring their user settings, the financial loses from a managerial point of view will be in the hundreds of thousands). 

Our methodology for averting user preference-related disasters

With over 6000 migrated Confluence spaces, and 3+ million pages, trust me - we've seen it ALL. Which is why we ensure that the smallest details - such as Confluence user settings - is carefully mapped, analyzed and properly migrated.

While our process is based on extensive work with Confluence Export/Import, we have developed additional in-house tooling (somewhat influenced by our existing app - Configuration Manager for Jira). This in-house tool enables us to capture and seemingly migrate user settings in a self-contained, portable file which is deployed on the Target system once all the spaces, their configuration, and data have been successfully migrated. There are three major phases to this process:

• Phase I Analysis: from a functional point of view, we analyze the default settings of both instances to ensure that they are consistent for all users and that the work of the users on both systems won’t be disturbed following the merge. We alert the client about any inconsistencies, which helps us proceed to the next phase
• Phase II Mapping: our tooling enables us to do a comprehensive mapping on a micro-level (e.g., comment, attachment, page ID, etc.). Which is why we can quickly identify and fix a wide variety of Confluence bugs associated with spaces and their broken references. 
• Phase III Post-processing: while we do export spaces one at a time (Confluence Export/Import does not support the migration of multiple spaces), there is a crucial phase of post-processing, where we ensure the consistency across the entire spectrum of settings after we have migrated all Confluence spaces, pages, and their referenced objects. 

Based on our experience, our advice - whether you are a Tony (end-user), Nick (Confluence admin), or a Joe (team leader of the initiative) - is to treat your user preferences with a medium to a high-priority status. The Devil hides in the details, and something as small as a default user setting can have a significant impact on the entire complex system. 

References:

Confluence documentation regarding the topics covered in the article:

• Your profile & settings
• Save for later
• Edit your user settings
• Watch Pages, Spaces & Blogs

Related Atlassian bugs:

• Favorites missing after importing (CONFSERVER-36348) 
• Mentions are broken after space XML Import (CONFSERVER-34994)
• Watches are not included in XML export (CONFSERVER-24443)

Author: Bianka Banova