Automation for confluence cloud space export in .xml

[Radoslav Yordanov]

I am searching for CLI tool or way using CLI to automate the export of one space or many spaces from my confluence cloud instance into .xml so i can back it up or import into another instance
I’d like to automate this on a schedule (e.g. every night) using a script or integration.

My goal is to regularly back up either the entire space or a specific space/page tree without having to manually initiate the export through the Confluence UI.

[Darryl Lee]

Oh boy, I think I went down this rabbit hole a while ago, using Chrome Developer Tools to watch what was happening after manually kicking off a Space Export from the UI.

I also took a peek at what Appfire's Confluence CLI is doing under the covers (that's the first workaround that @Tomislav Tobijas referenced from CONFCLOUD-37995).

Finally, I dug around atlassian-python-api to try and find the get_space_export function and ooof, I do not know how building PIPs works. Eventually I finally found the actual code, and it confirms what I was able to figure out.

So to make sure I don't forget to write it down, I present to you...

How to programmatically export your space as XML in 7 complicated steps (should also work for CSV and HTML) (PDFs are a whole 'nother story)

1. You have to make GET request to:

https://YOURSITE.atlassian.net/wiki/spaces/exportspacexml.action?key=YOURSPACE

2. You have to scrape the returned HTML to find the atl_token that is returned in that call (in a hidden form variable!)

3. You then have to use that token to make a call to:

https://YOURSITE.atlassian.net/wiki/spaces/doexportspace.action?key=YOURSPACE

The data you'll need to POST as a form (content-type: application/x-www-form-urlencoded) is:

atl_token=YOURTOKEN
exportType=TYPE_XML
contentOption=all
includeComments=true
confirm=Export

4. The returned HTML will include a <meta> value that looks like this:

<meta name="ajs-pollURI" content="rest/internals/1.0/io/export/spe-918849974">

5. You will have to set up a polling function to hit this URL:

https://YOURSITE.atlassian.net/wiki/rest/internals/1.0/io/export/spe-918849974

Each time you hit that endpoint, you should see something like this:

{
  "id": "spe-918849974",
  "elapsedTime": 1314,
  "percentageComplete": 14,
  "successful": false,
  "skipped": false,
  "complete": false,
  "message": "Archiving backup descriptor",
  "statusCode": 0
}

So yeah, you can keep hitting that endpoint -- in the Confluence UI it loads like every 5 seconds or so.

{
  "id": "spe-918849974",
  "elapsedTime": 102604,
  "percentageComplete": 100,
  "successful": true,
  "skipped": false,
  "complete": true,
  "message": "Export complete. Download <a class=\"space-export-download-path\" href=\"/wiki/download/temp/filestore/c7d557d9-3631-4acc-8e25-662754aa47ef\">here</a>.",
  "result": "/wiki/download/temp/filestore/c7d557d9-3631-4acc-8e25-662754aa47ef",
  "statusCode": 0
}

Depending on the size of your space, it could take a while, but eventually it'll hit 100% percentageComplete and successful = true, AND you end up with "result": "/wiki/download/temp/filestore/c7d557d9-3631-4acc-8e25-662754aa47ef"

6. So yeah, if you do a GET of

https://YOURSITE.atlassian.net/wiki/download/temp/filestore/c7d557d9-3631-4acc-8e25-662754aa47ef

You will get redirected (302 response with location) to something like:

https://api.media.atlassian.com/file/MEDIUMSTRING/binary?token=LONGSTRING&client=SHORTERSTRING&name=&max-age=2940&dl=true

That URL returns binary data, but it also contains a header with the filename:

content-disposition: attachment; filename="Confluence-export-space-YOURSPACE.zip"

7. Web browsers recognize that header and use it to name the file that is being sent. If you're going to write a program you could probably just name it however you like.

(Oh the curl flag to use the filename found in content-disposition is -J or --remote-header-name. Ugh, looks like for Python it's complicated...)

BUT ANYWAYS YEAH, NOW YOU HAVE YOUR SPACE EXPORT, IN 7 COMPLICATED STEPS! 

SO, the usual disclaimers apply:

THESE APIs are UNDOCUMENTED. ATLASSIAN CAN CHANGE THEM AT ANY TIME! YOU SHOULD PUT IN ERROR-CHECKING and ALERTING IN CASE THINGS CHANGE.

That being said, it's in the Python module. If and when it breaks, those guys will probably fix it. Maybe some day I'll understand where the actual code is hidden in the repo. :-}

Have fun!

[Darryl Lee]

Oh yeah, @Jayesh Raghuvanshi's suggestion is a good one, because Appfire CLI is using the same process as above to do its magic.

The nice thing about paying for an app is that if/when Atlassian changes the APIs and breaks things, Appfire (should) get to fixing it pretty quickly.

Especially if you're a paying customer and you file a ticket about it. :-}

[Darryl Lee]

OH I have an even better suggestion, which is free:

Write a Python script that uses the Atlassian Python API's get_space_export method.

With a little help from Cursor.ai I came up with this:

#!/usr/bin/env python3

import requests
from atlassian.confluence import Confluence
from credentials import HOSTNAME, USERNAME, TOKEN

# For Confluence Cloud
confluence = Confluence(
  url=HOSTNAME,
  username=USERNAME,
  token=TOKEN
)

space_key = "HELP"
export_type = "xml"

# Get Space export download url
export_url = confluence.get_space_export(space_key, export_type)

# Download the export
response = requests.get(export_url)
if response.status_code != 200:
  print(f"Failed to download export: {response.status_code}")
  print(response.text)
  exit(1)

# Save the export to a file
output_filename = f"Confluence-export-space-{space_key}.zip"
with open(output_filename, "wb") as f:
  f.write(response.content)
  print(f"Export saved to {output_filename}")
  exit(0)

Credentials are stored in credentials.py:

# Confluence API Credentials
HOSTNAME = ""
USERNAME = ""
TOKEN = ""

The downside to this approach is that get_space_export can take a while without any visual indicator that it's working (think about the progress bar showing percentage in the UI).

BUT, when I chose a smaller space because I was getting impatient, it indeed worked!

[Darryl Lee]

Oh, and because a backup is only useful if you can restore it, I just tested a modestly sized (80MB) space to a test site, and it restored PERFECTLY, attachments and all.

I'm now pondering using this as a backup strategy. :-}

[Jayesh Raghuvanshi]

Hello @Radoslav Yordanov 

While there are few request from atlassian to create a feature for space exports but they are currently in gathering interest phase https://jira.atlassian.com/browse/CONFCLOUD-40457 https://jira.atlassian.com/browse/CONFCLOUD-37995 

My suggesstion:

Confluence Cloud doesnot support fully supported REST API endpoint to trigger XM space export, we are left with CLI automation option

Use the Confluence Command Line Interface available in Atlassian Marketplace by appfire

• This is a paid plugin which provided pre-defined functions to for multiple options for the space like creating a new space, creating a page, copy the content from one page to another, space export, page export, page permissions ext
• You can write a script using the CLI tool's command and authenticate using the token created by atlassian admin account.
• Schedule the script on server as cron job

Below is an example of sample script

#!/bin/bash
# Configuration
CONFLUENCE_URL=""
SPACE_KEY=""
ATLASSIAN_USER=""
API_TOKEN=""
OUTPUT_DIR=""
TIMESTAMP=$(date +%Y%m%d%H%M%S)
EXPORT_FILE="${OUTPUT_DIR}/${SPACE_KEY}-backup-${TIMESTAMP}.xml.zip"

# Use the CLI tool (command will vary based on the specific tool)
confluence-cli --action exportSpace --server ${CONFLUENCE_URL} \
--user ${ATLASSIAN_USER} --password ${API_TOKEN} \
--space ${SPACE_KEY} --exportType xml --outputFile ${EXPORT_FILE}

# Add error handling and log creation here
if [ $? -eq 0 ]; then
echo "Successfully exported space ${SPACE_KEY} to ${EXPORT_FILE}"
else
echo "Export failed for space ${SPACE_KEY}"
fi

Let me know if you need any other help, We also provide freelancing support for the same.

Thanks

[Tomislav Tobijas]

Hi @Radoslav Yordanov ,

You might want to check this open suggestion (and workarounds listed there): CONFCLOUD-40457: Add REST API endpoint for generating space exports 

This other one is also relatively relevant to what you're looking for: CONFCLOUD-37995: Ability to schedule space exports to HTML, XML, or PDF 

Basically, no out-of-the-box solutions are currently out there, but you might want to try these:

1. The Confluence CLI provides such functionality, but it is a paid third-party add-on. The above functionality would be important to automate migration/backup tasks for administrators.

2. Use the atlassian-python-api library, which provides a method called get_space_export in the confluence.py module. This method allows you to trigger space exports through REST API requests. Please note that this library is community-maintained and is not officially supported by Atlassian

Cheers,
Tobi