Automate online site-backups for Jira and Confluence without programming

Here I'll introduce a way to automate site backups of Jira Cloud and Confluence Cloud with Jira Automation

 

(0) Why site-backup is important

Atlassian doesn't officially provide data rollback based on customer requests. So it's essential to take site backup regularly. For example, the backup will help you restore data when any of the site data is lost by a user's interaction. Refer to the following part in Data storage FAQ for more details:

Atlassian cloud sites don't support the use of backup data to roll back changes.

First, let me clarify what is covered by this article. The approach introduced here is only good as a temporary workaround as there are some constraints, such as:

  • Disclaimer: This approach is not officially supported and is currently maintained by the community. Consider voting for CLOUD-6498 to show your interest in official support.
  • Prerequisite: You need to subscribe to at least one Jira family product; meaning it's unable to follow this instruction if you only with Confluence subscription as we'll utilize Jira Automation feature which is only available for Jira. Now Confluence is going to have Automation as well.
  • Importantly, in this approach, we skip downloading the created site backups to any local machine simply because it's better than nothing. It is recommended for sure to download site-backups if you can. To do so, use the bash scripts listed below:

 

(1) Preparation

First things first, you always need to have credentials when you do something important:

  1. Issue an API token of the site-admins account at https://id.atlassian.com/manage/api-tokens
  2. Encode your credentials "$EMAIL_ADDRESS:$API_TOKENwith base64 as (1-2) Example below. Or go with any online tool like Base64 Encode. Make sure not to include a line break at the end.
    • base64encode.png

 

(1-2) Example - How to encode your API token

$ echo -n "site-admin@example.com:123e4567-e89b-12d3-a456-426614174000" | base64
c2l0ZS1hZG1pbkBleGFtcGxlLmNvbToxMjNlNDU2Ny1lODliLTEyZDMtYTQ1Ni00MjY2MTQxNzQw
MDA=

 

Besides, for those using IP Allowlistinginclude the egress IP ranges introduced at IP addresses and domains for Atlassian cloud products #Outgoing Connections.

 

(2) Create a rule for Confluence site-backup

Although you can start with either Jira or Confluence, we'll look into how to configure a rule for Confluence here for convenience.

  1. Set up the schedule
    • Add the content of (2-1) below to Cron expression
    • Select "simply run the conditions and actions without providing issues" for When rule executes
    • schedule-cron.png
  2. Add an action - Send web requests
    • Webhook URL{{baseUrl}}/wiki/rest/obm/1.0/runbackup
    • Headers:
      • Authorization: Basic <The output we got from (1-2)>
    • Webhook body: refer to (2-2)
    • action-webhook.png
  3. Add another action - Log action
    1. To be able to keep track of what we got with the webhook request, specify {{webhookResponse}} in Log message
  4. Name the rule "Site backup - Confluence Cloud" and save the setting
  5. Do a test run with [Run rule] button
    • The log then will be available in the Audit log
    • audit-log.png
  6. Check the download link
    • confluence-backup.png

 

 

(2-1) Cron expression

Each backup should be performed after a 48-hour interval at least. We here are going to schedule the execution twice a week on Monday and Thursday. Also, make sure to avoid maintenance windows for your site. For example, suppose your company sits in EST, specify 9 am in UTC:

0 0 9 ? * 1,4

 

(2-2) Webhook body

{"cbAttachments":"true" }

 

(3) Create a rule for Jira site backup

I'll put the diff in settings as it's almost the same with Confluence.

  1. Schedule - same
  2. Action - Send web request
    • Webhook URL{{baseUrl}}/rest/backup/1/export/runbackup
    • Webhook body: refer to (3-1)
  3. Remaining procedures - same
    • jira-backup.png

 

(3-1) Cron expression

Same here, on Monday and Thursday at 9 am in UTC:

0 0 9 ? * 1,4

(3-2) Webhook body

{"cbAttachments":"true", "exportToCloud":"true"}

 

(4) Constraints

Other than noted above, there are several points you should be aware of:

  • Jira has an upper bound on rule executions per subscription plan. That means you will not get a backup if exceed the limit. So consider upgrading to Premium, especially if you already have a respectable number of existing global automation rules.
  • The backup link will be available for 14 days for Jira and 7 days for Confluence. The link then will be replaced by the subsequent one. That means you will only be able to retrieve the backup file until the next backup ends. So make sure to download the backup file as soon as you find any data loss; otherwise, the backup link will be replaced with the one taken after the data loss.

 

62 comments

Veera
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 19, 2020

Great job, @K_ Yamamoto!!! 

Taranjeet Singh
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 6, 2020

very useful information indeed. Thanks!

ITS_Elvadata March 12, 2020

is it possible to create a backup and download the file automatically via a Windows Server ?

Thanks

K_ Yamamoto
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 12, 2020

 

Yes, it is possible. Refer to atlassianlabs/automatic-cloud-backup and try the PowerScript or Python script.

ITS_Elvadata March 12, 2020

thx, i will try it :-)

Thomas Kringstad April 3, 2020

Thanks, looks like this works.. but I have one question...in the Log Action is the correct input {webhookResponse}} or {webhookResponse.body}} 

In the png you have written .body but not in the text...

Like Santiago Bernheim likes this
Deleted user April 3, 2020

Any idea how to change the limit of the backups triggered from 48 hours to let's say every hour?

Nivruti Vora April 21, 2020

@[deleted]I don't think it's possible(depending on what you want to backup). Check https://confluence.atlassian.com/jirakb/how-to-automate-backups-for-jira-cloud-applications-779160659.html

From the article: "You can perform data-only backups—excluding attachments, avatars, and logos—one after the other, without a delay. When you're creating a backup that includes attachments, avatars, and logos, though, you need to wait 48 hours from the when you last created a backup." It's a limitation!

K_ Yamamoto
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 21, 2020

@Thomas Kringstad - As the response is different between Jira and Confluence, I'd recommend you to use {{webhookResponse}} for both to avoid the confusion. It will include all the headers and body content.

K_ Yamamoto
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 21, 2020

@[deleted] - As Nivruti kindly suggested, there's no option to change the backup frequency limit at the moment. Consider to vote for these feature requests:

If you are working on something that requires frequent backups such as data migration, reach out to https://support.atlassian.com/contact/ so that Atlassian Support can temporarily increase the quota.

Deleted user April 21, 2020

@K_ Yamamoto - I have an open case with support, thank you!

Jeffrey O'Hara April 23, 2020

Configured the backups for JIRA Cloud instance using Automation Rules and it works great. 

However, I would like to download the backups locally since they're overwritten. 

Configured the Jira-backup.sh script from atlassianlabs/automatic-cloud-backup link above. However, I'm receiving the error "Basic auth with password is not allowed on this instance". I see this link to setup oath for JIRA Cloud https://developer.atlassian.com/cloud/jira/platform/jira-rest-api-oauth-authentication/?_ga=2.101848449.109137584.1587588476-1332438393.1568149174. How can I tell which authorization I'm using? I don't have SAML authentication configured. 

Christopher Harris April 26, 2020

Even with all these steps, it still seems that one has to manually download the backup file.  What exactly is being accomplished with the above steps?  Simply generating a backup file, this is achievable with one click in backup manager.  What we really need is automatic backups, end to end.  But what we really really need, is Jira to build in the ability to recovery deleted data at least for some period of time.

Like # people like this
K_ Yamamoto
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 26, 2020
K_ Yamamoto
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 26, 2020

@Christopher Harris - I understand your concern and do agree with you. The workaround introduced in this article is far from the ideal experience. Please go with other solutions like below:

 

what we really really need, is Jira to build in the ability to recovery deleted data at least for some period of time. 

As this feature is not yet implemented, I'd advise you to add yourself as a watcher to the feature request tickets below to receive any updates and add any comments you think are necessary and/or important to the feature request. I've also added a private comment linking this case to the feature request for our dev's review.

In addition, more information about how Atlassian implements New Features can be found in Implementation of New Features Policy.

Like Craig likes this
KP
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 4, 2020

Awesome, @K_ Yamamoto !

Like Dominik Spescha likes this
Darryl Lee September 18, 2020

Hi, I've setup the Automation Rule for backups and it has run twice now (and the logs show that it was successful). But each time I've gone to the Backup manager page to look (usually a day or so after the rule runs), I do not see a download link.

I thought that backups are supposed to be kept for 7 days.

I just manually ran the rule, and the "Create backup for cloud" button is grayed out, so I assume a backup is in process, but I see no progress bar, and I suspect I will have the same issue again of no download link when the backup is complete.

Any ideas?

[Ooof, I realized I forgot to output the {{webhookResponse}} to the Audit log. Perhaps that will have a URL for download/progress that I'm missing. Still odd about the link not showing up in the Backup manager page. This is for Jira, BTW.]

Darryl Lee - CCC September 24, 2020

Ah, I think the trick was that I forgot this in the Webhook body: 

"exportToCloud":"true"

INTERESTINGLY... the backup was created, and it is supposedly Server compatible, and I *DID NOT* have to delete my Next-Gen project (which isn't an option for us).

It was a pain to find the download though. I had to hit up this URL:

> https://{MYHOST}.atlassian.net/rest/backup/1/export/lastTaskId

Which provided the TaskId of the last backup I kicked off. So then I did:

> https://{MYHOST}.atlassian.net/rest/backup/1/export/getProgress?taskId={LASTTASKID}

Which showed this result

{
status: "Success",
description: "Cloud Export task",
message: "Completed export",
result: "export/download/?fileId=MYFILEID",
progress: 100,
exportType: "SERVER",
}


So then, the final download link is:

https://${INSTANCE}/plugins/servlet/export/download/?fileId=MYFILEID

And... proof being in the pudding, I was able to successfully import this backup file to a Server instance. Woot!

UPDATE
After the import, I couldn't login, which is a known issue after importing from Cloud. So, per the instructions here, I restarted with the atlassian.recovery.password property set.

For some reason, restarting with this property set triggered Jira to do an immediate reindex.

Unfortunately this reindex failed with 1 error:

2020-09-24 14:58:46,125-0700 JIRA-Bootstrap ERROR      [c.a.jira.upgrade.LicenseCheckingUpgradeService] An exception occurred when running to call a callable method, this shouldn't happen.

java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.atlassian.jira.index.IndexingFailureException: Indexing completed with 1 errors

Finally though, after another stop and restart, Jira did successfully start up and I was able to login with the recovery_admin account/password. But meh, no data. Ok, will continue to dig into it.

Like Majo Francis likes this
Zack Foster January 15, 2021

Thank you @K_ Yamamoto , this was extremely helpful!

A very minor CRON nit I found while implementing:

0 0 9 ? * 1,4

In the article, you describe this as 9AM UTC Monday and Thursday. The CRON numeric schedule actually runs Sunday-Saturday, so your expression actually equates to Sunday (1) and Wednesday (4) best I can tell.

Thank you again for the article.

Warm regards~

Steffen Müller February 3, 2021

Hey, I just created a CLI tool with Opsgenie integration to manage Atlassian cloud backups. It creates, downloads and manages (as in "removes old") your backups and is configured with an easy config file. If things go wrong, Opsgenie alerts are created (optional).

It's open source and I'd be happy if it helps you as well, take a look at https://bitbucket.org/addcraftio/atlascloud-backup

I wrote it to scratch my own itch because I wanted to have Opsgenie monitoring and backup rotation on our offsite Synology NAS -- and got carried away a little. The tool is used in production for all our addcraft cloud instances.

Hope this helps out a little, @Christopher Harris & @K_ Yamamoto!

Danny Grenzowski _Rewind_
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
March 9, 2021

Thanks for posting this guide, it's very helpful. I noticed there were some comments, such as @Christopher Harris, asking about a more comprehensive solution, including restore functionality for Jira Cloud.

 

My team is working on a fully automated backup-and-restore product for Jira Cloud - Rewind Backups. If you interested, please let me know! I'd love to chat with Jira customers who are looking for this type of solution.

Like Andrew Jastremski likes this
Ludvig Hede March 26, 2021

@Darryl Lee - CCCI'm reading your posts here regarding a download link with high interest. I'm curious how you automatically fetched the link and how you knew when the link was ready for download. How did you implement this?

Phillip C May 3, 2021

The download link for Jira doesn't show but the job runs successfully. Any reason why this would be?

However, Confluence shows the download link and works fine as well.

I've tested with and without the 

"exportToCloud":"true"

 Also, there is no web page for https://{MYHOST}.atlassian.net/rest/backup/1/export/lastTaskId

Doesnt load, doesn't do anything - (dev tools in browser showing web page status code 204)

K_ Yamamoto
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 4, 2021

UPDATE: Free Apps for Jira and Confluence

It's been a while since I've published this article. Here, I'm thrilled to announce that here we have free apps providing backup automation, available for Jira and Confluence.

Not like the approach introduced in this article, the apps are not relying on Jira Automation. So you can now automate the backups without having any Jira license. Besides, the other advantage of the apps is backup history. Although the apps don't keep the local copy of backup files, the download links will be available even after creating the subsequent backups.

Please note, the apps are in beta and not officially supported. I am looking forward to hearing your feedback.

Like # people like this
Ali Alnakeeb September 26, 2021

thanks, however, the basic authentication method is being deprecated, this article could be updated to latest status 

Like # people like this

Comment

Log in or Sign up to comment
TAGS
AUG Leaders

Atlassian Community Events