BUG? - total loss of app data on space import (loss of CustomContent and CustomContentEntityObject)

Question

Dear community,
Dear Atlassian,

Situation:

* I have a Confluence Cloud with different apps from the marketplace (vendors are Atlassian Gold - Platinum partners and no small / unheard-of companies in the Atlassian realm) 
* I created some data with those apps
* I confirmed via /wiki/api/v2/custom-content API endpoint that the app data is present as custom content
* I export the space
* I confirmed all app data is present in the entities.xml file in the backup
* I imported the space to another instance for testing and development purposes having the same apps installed prior to install
* I confirmed that all pages and attachments are successfully imported

Issue:

* API endpoint returns empty array -> no data
* apps report "no data" (each in their way) -> data not present
* app macros come up empty (since no data is there to be queried)

Conclusion:

* import of spaces DO NOT restore app data stored in custom content entities!
(I tried the same with a complete site import, no joy there either!)
* the result is total data loss of app data

I contacted some vendors who confirmed the issue to be present on their systems as well.
-> this is not related to my site or instance.

I don't think it should make a difference: apps are Atlassian Connect apps.

Some reported that they had tested this before and claim it worked, pointing the finger on Atlassian. (which, in my 20 years of experience with Atlassian, seems quite reasonable, since the export clearly has the content present and the import does not import them)

Did anyone experience the same?
Is there a known fix for this?
Can I recreate the data via an API endpoint reading the entitites.xml? Does that work? (I played around with it a little and I am not sure this will work...)

Thx,
Jay

Oday Rafeh_2jhprw · Answer

Hi @Janek Schumann

I understand your frustration, and it's true that the approach I mentioned can be quite challenging, especially when app vendors don't provide dedicated APIs or their data structures are not easy to reverse engineer.

I apologize for not being able to provide a straightforward solution to your issue. However, engaging Atlassian Support and app vendors may help you find a more suitable approach to importing app data.

Good luck

Regards

Oday

Janek Schumann · Answer

I was playing around with this.

first off, most vendors don't have a dedicated API. And since neither Forge nor Connect can add one to the Atlassian product directly... I am out of luck.

Looking at the Atlassian Cloud API gives me idea. Tried to create custom content, worked. Tried to update custom content, sometimes works? Deleting custom content, does not work for me. Not sure why. Getting a 404 on IDs which are clearly there. (maybe only apps can use the delete endpoint?)

Trying to reverse engineer a vendor's data structure, trying to recreate it with custom scripts in the hopes the app will work as before, ... I'd rather shoot me in the foot or wait for Atlassian to fix the import issue. (backups cannot be imported to 100%? really? who does that?!)

Knowing Atlassian, however, the most basic features customers want or need takes a decade or more. Just saying "CLOUD-6999" >.<

Oday Rafeh_2jhprw · Answer

Hi @Janek Schumann

Recreating the data using API endpoints by reading the entities. xml file can be a possible workaround, but it might not be straightforward and may require a custom script.

The success of this approach depends on the APIs provided by the app vendors and your familiarity with using those APIs.

- You can follow:

-Parse the entities. xml file:

You'll need to write a script or use a tool to parse the entities. xml file and extract the custom content entities related to the app data.

-Use the app vendor's APIs:

For each custom content entity, refer to the app vendor's documentation to understand their APIs and find the appropriate endpoint to recreate the data.

-Post the data using API endpoints:

Use the API endpoints provided by the app vendors to send the extracted data to the corresponding app in the new Confluence Cloud instance.

-Verify the data import:

After recreating the data, check the apps in the new Confluence Cloud instance to ensure the data has been imported correctly.

Please note that this approach can be time-consuming and may require a deep understanding of the app vendor's APIs. It's highly recommended to reach out to the app vendors and Atlassian support for assistance or more efficient solutions.