nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

🛡️ Automate Jira & Confluence Cloud Backups with AWS (Open-Source Python Scripts)

Aaron Morris_49u8f7

Hi everyone! 👋

I just wanted to share some Python scripts I wrote to automate Jira Cloud and Confluence Cloud backups using AWS.

🔗 https://github.com/agile-innovations-tech/atlassian-cloud-backups

There are four scripts:

Jira Backup Trigger: Initiates the creation of a new Jira Cloud backup file using Jira's API.
Jira Backup Download: Downloads the latest backup file from Jira Cloud and stores it in AWS S3.
Confluence Backup Trigger: Initiates the creation of a new Confluence Cloud backup file using Confluence's API.
Confluence Backup Download: Downloads the latest backup file from Confluence and stores it in AWS S3.

The scripts are designed for AWS Lambda, but you could adapt the scripts to run on other platforms.

These scripts are designed using the APIs described here:

By using AWS, you can use:

Lambda to execute the scripts
EventBridge to schedule the scripts
S3 to store the backups
and CloudWatch to monitor for backup failures.

The README in the repository contains some YouTube links for a full tutorial.

👉 If you try it out, I’d love to hear how it works for you!

💬 Feedback, ideas, or even code contributions are very welcome.

Thanks for checking it out!

Comments

Natasha Richardson

Hi Aaron,

Just a quick question. Does this solution work similar to backup tools in the marketplace for example revyz?

Aaron Morris_49u8f7

@Natasha Richardson -- Great question! These scripts only automate Jira and/or Confluence's native backup capabilities.

They do not provide any of the advanced features you might get from the Marketplace.

Zachary Howard

Always great to see Community members offer solutions like this! I am curious though, as someone who hasn't really considered backing up our Jira and Confluence Cloud (Premium) instances is this something that I should look into? I had assumed that since the apps were cloud-based Atlassian would have their own backups in place in case of any critical system outages.

I've been trying to look into this and found a few sources talking about the lack of a native auto-backup feature, and ways around this:

https://jira.atlassian.com/browse/CLOUD-6498
(In typical Atlassian fashion this ticket from 2013 is still unresolved, and I don't see a backup feature anywhere in my Admin portal)
https://support.atlassian.com/jira/kb/automate-backups-for-jira-cloud/
(A KB article that states the instructions are not supported by Atlassian and rely on Community Support)
https://support.atlassian.com/confluence-cloud/docs/create-a-site-backup/
(Manual Confluence Cloud backup)
https://support.atlassian.com/jira-cloud-administration/docs/export-issues/
(Manual Jira Software Cloud backup)

So I have a few questions:

What are the benefits of performing our own Cloud backups vs relying on Atlassian's backups?
@Aaron Morris you mentioned that "These scripts only automate Jira and/or Confluence's native backup capabilities" compared to the available 3rd party apps. I'm not sure what this means though. What do the native backup capabilities not capture that 3rd party options do?
How does your Python solution differ from the functionality in this KB article that claims to not require any scripts?

Sorry if my questions are complicated. Genuinely excited to see this free solution that you've created! Thank you!

Aaron Morris_49u8f7

Hi @Zachary Howard -- This is kind of a heavy subject, but I'll give some brief answers:

What are the benefits of performing our own Cloud backups vs relying on Atlassian's backups?

Having your own backups provides a little more insurance. Specifically, Atlassian's disaster recovery capabilities only protect you against system-level incidents that (likely) impact many customers. They can't help you if something happens to your data locally within your site. For example, a buggy automation script deletes too many records, a disgruntled employee sabotages your data, etc.

Having site-level backups might save you after a localized incident like these.

You mentioned that "These scripts only automate Jira and/or Confluence's native backup capabilities" compared to the available 3rd party apps. I'm not sure what this means though. What do the native backup capabilities not capture that 3rd party options do?

Natasha was asking how these backups compare against using something like Revyz or Rewind. The answer is that Jira's native backups only provide a monolithic backup of your entire Jira site that can be restored to a temporary site or to your original site.

Third-party apps make it easier to restore individual projects, individual records, specific configurations, and so forth. They also support a longer history of backups. You'll have to research each individual app for a comprehensive comparison of capabilities.

And to be clear, those apps are great. The only downsides are that they are more expensive and tie you into their technology if you want to maintain your history of backups.

You'll have to do your own cost-benefit assessment to decide if those solutions are right for you.

How does your Python solution differ from the functionality in this KB article that claims to not require any scripts?

The solution in the KB is incomplete. Specifically, the following warning is listed near the top of the KB.

Importantly, in this approach, we skip downloading the created site backups to any local machine simply because it's better than nothing. It is recommended to download site-backups if you can. To do so, use the bash scripts listed below:

Backup script for Jira Cloud

Backup script for Confluence Cloud

So, for my own purposes, I've adapted the scripts linked by the KB to run in AWS and make use of AWS's secure storage, monitoring, etc.

And I've shared my scripts here. 🙂

I hope this helped clarify things a bit. Again, this is a big subject and would require an eBook to answer in full. 😆

Zachary Howard

Thanks for the detailed response! I really appreciate it, and I'll certainly be looking into this more for our company!

Matt Smith

This is what we have done for a long time

Darryl Lee

@Aaron Morris this is amazing work.

Can I ask you and @Matt Smith how big your backup files are? Or I guess in Storage, how much space are your instances using?

Our usage is apparently (but understandably) way too big to create a Zipped XML file:

Jira Premium - 7.9 TB of unlimited GB used
Confluence Premium - 1.9 TB of unlimited GB used

We migrated long-standing on-prem instances, and we did NOT do any clean-up or selective migration. Everything (clearly) came with us to Cloud. :-{

Matt Smith

Jira is about 50GB and confluence is about 90GB

Lucas Slowik

Hi Aaron,

Great job overall, but I can see some significant limitations in your Lambda-based approach.

1. Execution time limit
AWS Lambda has a maximum execution time of 15 minutes. For large backup files, this makes it unsuitable.

2. Memory usage and transfer efficiency
As I understand your implementation, the file is first fully downloaded into memory and only then uploaded to Amazon S3. This creates two issues:

- For large backup files, a substantial amount of memory must be allocated to the Lambda function.

- Since the download and upload happen sequentially rather than as a stream, the total transfer time is not optimized.

I’ve been working on a similar solution and ended up using an Amazon Elastic Container Service container with a Python application that streams the Jira backup directly into S3.

With this approach:

- There is no execution time limit.

- There is effectively no file size limit.

- It runs on a very small container (0.25 vCPU and 500MB RAM).

- A 60GB backup transfer completes in about 1 hour.

- The monthly cost is approximately $0.50 if the backup runs daily (excluding data transfer costs).

Aaron Morris_49u8f7

Feel free to share your solution with the rest of us. :-)

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

jira-cloud

confluence-cloud

automation

backup