Introducing Atlassian CloudFormation Backup Machine

Atlassian Cloudformation Backup Machine

Atlassian Cloudformation Backup Machine (Backmac) is an AWS State Machine, Lambdas and EC2 node which allow automated backups of Cloudformation Stacks produced by Atlassian's Cloudformation templates.

The IT Operations team at Atlassian built this tool internally as a backup solution which also allows for copying backups to our DR region as required.

What does the Backup Machine do?

Backmac runs a Cloudwatch rule to trigger a Lambda that will look for all Cloudformation stacks in a region with the tag 'backmac_enabled'. These will be added to an SQS queue and the State Machine execution will be triggered.

The State Machine orchestrates a series of Lambdas and a dedicated high IOPS compute node and works through the following:

  • Mounts the EFS volume on the compute node
  • If needed, create or resize a backup EBS volume to copy the EFS content onto
  • Trigger an RDS Snapshot
  • Do a parallel rsync of shared home (on EFS) to the backup EBS volume
  • Snapshot the EBS volume
  • Copy the snapshots to dr region

The code

https://bitbucket.org/atlassian/aws-backup-machine/src/master/

Pre-deployment configuration

  1. You will need an AWS account, with either the AWS managed "AdministratorAccess" policy or the effective "Allow *" for the following services:
    1. EC2
    2. IAM
    3. CloudWatch
    4. S3
    5. SSM
    6. Lambda
    7. Step Functions
  2. Decide which AWS regions you want to create stacks in

Deployment

Backmac is deployed in a two step process. 

  1. run package.sh <region> from the root of the repo
  2. deploy the generated Cloudformation template .yaml file

The template creates:

  1. An IAM role with all the access Backmac needs to operate
  2. An EC2 node that runs the rsync process
  3. A bunch of Lambdas used by the State Machine
  4. A State Machine to orchestrate the Lambdas

Post-deployment configuration

Make sure at least one stack is tagged with 'backmac_enabled=true'

Backmac will run on the cron schedule entered during the creation of Backmac, and will back up any stack that is tagged as above.

Cleaning up old snapshots

Backmac will also create an additional Lambda to clean up any backups older than their 'backup_delete_after' date (which Backmac will set to 30 days after backup by default).

Support

Atlassian CloudFormation Backup Machine is not supported by Atlassian. It is provided as is.

More information

More detailed information about the Backup Machine, and migrating to AWS CloudFormation in general, is provided in this video.

1 comment

Comment

Log in or Sign up to comment
Community showcase
Published Wednesday in Jira

Make your Atlassian Cloud products more secure: our NEW admin security guide

Hey admins! I’m Dave, Principal Product Manager here at Atlassian working on our cloud platform and security products. Cloud security is a moving target. As you adopt more products, employees consta...

45 views 0 2
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you