You're on your way to the next level! Join the Kudos program to earn points and save your progress.
Level 1: Seed
25 / 150 points
1 badge earned
Challenges come and go, but your rewards stay with you. Do more to earn more!
What goes around comes around! Share the love by gifting kudos to your peers.
Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!
Join now to unlock these features and more
Merging data from one Jira/Confluence server into another Jira/Confluence server is one thing. Merging data from Jira/Confluence Cloud into a Jira/Confluence server environment is another. However, I was tasked with this fun & challenging project at one of our customers, DPG Media.
At our customer, DPG Media, several teams had started using both Jira & Confluence over the last couple of years, resulting in a big Jira and Confluence server environment setup with over 2000 active users all logging in with their active directory credentials. However, some teams had been working on a different Jira or Confluence environment being it due to company merges or other causes, and some of these instances were living in the cloud.
The downsides here are obvious. Certain users had to go to different URLs to work on different projects in different Jira/Confluence instances. On top of this, there are some major differences in Cloud vs Server deployments (think UI and plugins) which made it unclear to some users why they saw things differently in what they thought was the same tool. Oh yeah, and don't forget the double license cost of course.
In the vision of bringing the teams that used Jira or Confluence closer together, it was time to merge the different environments.
Each scenario is different, each company is different, but also each migration path is different. However, there are some questions you need to ask yourself when starting a migration or merger project like this.
These 2 are major factors. There are plenty of plugins available for each deployment option, but not every plugin that you currently use, is available for both Cloud and Server.
Other things to worry about is the impact of the data. In most cases it will make more sense to migrate the smallest environment into the bigger one. If however, you choose to merge a bigger Cloud instance into a 'smaller' Server instance, because of certain plugins/functionalities for example, you may need to think about the extra load on the server. Do you have to upgrade your server hardware as well for example?
And of course, it's definitely worth comparing the license cost for both. In our case, we had enough licenses left on our server instance to absorb all the extra users that came from cloud (some users already had an account in the server instance AND the cloud instance).
Before developing a migration plan, we first had to fully understand how our requirements could impact the merging and had to tackle some problems before starting the process.
Many other different problems occurred, such as status categories being overwritten, issue linking descriptions, etc. All this had to be fixed for the eventual production migration. Simply put:
These two requirements were simple, but required a lot of testing and attention.
Our migration path was actually pretty simple, but required some extra work. Jira was obviously the biggest challenge as we could simply use Confluence's default space export and import. But for Jira, we chose to use the amazing Project Configurator plugin by Adaptavist. The plugin is amazing, the support is extraordinary!
"But the plugin is server only?"
"You migrated from cloud to server?"
Our secret in the migration process, is to simply provide a middle server which hosts a clean Jira installation, a clean database and a clean Confluence installation. Look at the picture below to better understand our process.
Having a server in the middle was a helpful tool for several reasons. Let it first of all be clear that the middle server was a clean install of Jira and Confluence and didn't contain any production data, configuration or schemes.
On this middle server we imported the full site backup that was generated from cloud (both for Jira and for Confluence). We used the developer license for Jira and Confluence. On the middle server, we created a Jira User server and linked it to the Confluence instance. This allowed us to re-use the exact same user base as existed on Cloud. The Jira User server was put to the top of the list in the Confluence user directories, so Confluence would use the usernames from Jira's directory rather than its own internal directory. Remember that both Jira and Confluence's internal directory hold the same users and usernames as they are included in the cloud export. Linking them together allows you to change usernames in Jira and automatically apply that to Confluence.
This should ring a bell and remind you of the mismatching username problem. We created a script that ran over a spreadsheet of usernames and email addresses and applied changes through the Jira API. Via the API, each email address was put in a search query and returned the available user on the middle server (which came from cloud) and then updated this cloud-imported username to the username defined in the spreadsheet (which were the usernames used in the active directory).
Because of this change, all data in Jira was mapped correctly to the new username. And as the cherry on top, because of the Jira user server setup, all Confluence data was also automatically updated to the new username.
Nope! We also used the middle server to update all configuration that came from cloud. We matched the (from cloud imported) configuration to the configuration that was used on the production instance. Think of changing project keys, issue types, statuses, resolutions, priorities, etc.
It was important to change these settings here for many reasons. First, it allowed us to apply the configuration changes without interrupting the cloud instance setup (as they continued to use the cloud instance until the actual merge). But next to this, it also allowed us to see that for example a priority scheme doesn't exist on cloud yet, and we therefor had to create a default one on the server (so that the existing default on production wouldn't be overwritten).
And of course, remember that the Project Configurator plugin was only available on server.
On top of this, we had a safe way to use database queries to fix Jira macro's that pointed to the wrong application links (they still pointed to cloud).
Last but definitely not least, we got to install the same plugin versions as the production server to increase the chances of a clean plugin data transfer.
Now it was time to export all projects (data and configurations) and upload these configurations to production (which should have no impact on existing configurations as we updated everything on the middle server). You'll get an overview by Project configurator of the changes that are being applied to existing configuration, go through these carefully to see if you missed something.
In the first batch, we choose not to import boards, filters and dashboards as they need extra attention. For example, usernames that had been changed, would break filters. Same goes for renamed projects or project keys. We fixed these by extracting the config.xml file out of the full export zip and updating all references that were used in the <queries> tags with the find & replace tool which is available in most text editors.
Import the updated XML file and you're done... with Jira.
Confluence at this point is easy (if you don't have corrupted spaces or duplicate space key problems). Simply go and export the XML for each space individually and import them, one by one, on the production Confluence instance.
For duplicate space keys, we used the ScriptRunner for Confluence plugin's "copy space" functionality. This allowed us to copy the entire space to a new key.
The merging of the cloud data was successful for our instance. However, no merge or migration goes without some bumps in the road. The advise here is to plan enough time for testing and provide some buffer for unexpected errors or problems that you didn't think of before hand. And again, test, test and test. (You can do a full test migration if you have a staging server that is a copy of your production instance.) We did a full migration test multiple times as we had a staging server available.
I couldn't include all problems or attention points in this article (you see it's already quite long). A migration or merger is a big thing, never expect this to go smoothly from the first try. If you need help with migrations, have any questions questions or would like to learn more, feel free to post something in the comments.
Thanks for reading!