The Bridge Between Clouds: A Fair Dinkum Framework for Xray Data Migration

[#1]

Moving a Jira instance can be a right royal pain in the neck. While Atlassian’s built-in tooling handles native data and configurations quite well, things get trickier when it comes to Marketplace apps and third-party data. The current landscape of cloud-to-cloud migration and Atlassian organisation restructuring is complex, with evolving platform capabilities and limited native support for fully automated migrations that cover both core and Marketplace app data.

The Xray community has long faced this issue. Many of us have followed the feature requests for a backup re-import function to complement the existing data backup export and native support for cloud-to-cloud migrations of app data. The lack of these native solutions led me to "build my own".

Now, I could have gone the primitive route, messing about with UI-based test case exports and importers. But let's be honest: while those might get you part of the way there, they fall short when you need near-identical data replication. Manual, UI-heavy processes simply don't scale; they’re tedious, error-prone, and a bit of a worry when you're moving thousands of records. Instead, I decided to build a custom pipeline. Some might say obsessing over this level of fidelity is a silly challenge to set for oneself, but I’ve found it offers a worthy quest to deeply understand technical problem-solving beyond a "click and hope" approach.

The Foundation: Data Extraction and the "Gist" of the Data

The journey starts with data extraction. We put Xray’s native Backup feature to work as it’s already there, and it happily produces a full instance archive without any drama. With the data handed to us on a silver platter, we roll straight into Canonical Bundle Assembly.

I introduced the Canonical Bundle to act as the "gist" of the data—a strict, simplified representation of everything we need. We keep the bundle weight light by externalising references to attachment blobs, focusing instead on a normalised format that captures structural relationships using Jira work keys. Once assembled, we run a Canonical Bundle Validation pass to ensure the data conforms to our schema and that the semantic hierarchy is consistent. By creating this "source of truth," the extraction process becomes entirely decoupled from the rebuild.

Identity Management: The State Machine

Before we can move data, we have to solve the "Identity Gap." We use two distinct seeding processes to populate our PostgreSQL state engine:

• Issue ID (Work Item ID) Seeding: We run live JQL lookups in both the source and target Jira instances. This establishes a "ground truth" lookup table mapping Jira Work Keys to their specific Issue IDs in both environments.

• Synthetic Issue ID Seeding: We pre-seed the state database with synthetic Work Keys from the canonical bundle (see note¹). We then materialise these as real issues in the target Jira project, replacing the synthetics with their newly created Issue IDs and updating the state machine accordingly.

---

¹ Note: Synthetic issue ID seeding is the trick that makes cross-tool migrations possible, but it’s not magic. It’s an essential identity bridge, and it works especially well in DC→Cloud moves where the source testing tool isn’t Xray (e.g., TestRay in Jira Data Center) and you’re consolidating into Xray on Jira Cloud. By minting placeholder work keys from the canonical bundle and then materialising them as real issues in the target, we get stable, addressable anchors in Jira. That lets us realign the coupling between vendor data models and Jira’s native structures in-flight—so relationships, memberships, and history can be re-mapped deliberately and consistently.

---

Migration at this scale is rarely a "one-shot" success. Network blips or API rate limits are inevitable. To combat this, our pipeline utilises a Tracking Database to maintain state at the individual asset level.

• Granular Tracking: Every test, plan, and execution status is recorded (e.g., Definition Created, Folders Assigned, Results Ingested).

• Idempotency: Because every phase is tracked, the process is fully resumable. If the migration is interrupted, the pipeline identifies exactly where it left off and avoids duplicating data or creating redundant Jira issues.

It’s a fair bit of plumbing, but it saves a massive mess later.

Rebuilding the Assets: Definitions and Hierarchies

With the identity bridge built, the pipeline begins the heavy lifting of reconstruction. I found it best to separate these into two distinct functional streams to ensure nothing was lost in the shuffle:

• Re-hydration of Test Asset Definitions: We rebuild the "guts" of the tests, restoring test steps, definition-level attachments, datasets, and parameter tables, alongside precondition logic.

• Reconstruction of Test Planning Memberships and Hierarchies: Here, we restore the organisational fabric. The pipeline recreates memberships for Test Plans, Test Sets, and Test Executions, while also re-establishing the Test Repository folder associations so the team finds a familiar structure on day one.

The History Move: Transformations and Imports

The final stages focus on historical integrity and dealing with the realities of Cloud APIs.

• Execution Result Payload Transformation: We derive chunked execution result payloads from the canonical bundle. This is a delicate process: dataset-driven tests and parameterised runs can fan out into many iterations of result data for a single Test Execution, so we have to carefully reconstruct per-iteration outcomes. This ensures that when payloads are created for Xray import, the full context of every test run remains intact.

• Large Attachment Management: The Xray API has strict payload size limits. To avoid timeouts, we handle large evidence files as a dedicated post-import pass. We upload these individually to the previously loaded results, with the metadata driven directly from the execution bundle generation.

The Bottom Line

Building a custom migration pipeline isn't just about moving bits; it's about bringing the whole story across, warts and all, to a new environment. By focusing on a state-driven architecture and a strict canonical contract, we turned a high-risk event into a series of predictable, repeatable wins.

---

Disclosure: All images featured in this article are AI-generated using Google Gemini. Image generation prompts were crafted directly from codebase context within JetBrains IDE and GitHub Copilot.