Introducing Atlassian Labs DC Backmac - formerly Atlassian CloudFormation Backup Machine

[#1]

Atlassian DC Labs Backmac

NB: Atlassian CloudFormation Backup Machine has been renamed to Atlassian Labs DC Backmac.

Backmac is an AWS SAM app for performing backups of Atlassian Data Center applications (Jira, Confluence, and Crowd) running in AWS as CloudFormation stacks.

The IT Platform Engineering team at Atlassian built this tool internally as a backup solution which also allows for copying backups to our DR region as required.

What does the Backup Machine do?

Backmac utilizes an EventBridge schedule to trigger a Lambda that will look for all Cloudformation stacks in a region with the tag "backmac_enabled" set to "true"; any stacks found meeting this criteria will be added to an SQS queue and the State Machine execution will be triggered.

The State Machine orchestrates a series of Lambda functions and a dedicated EC2 compute node and works through the following tasks (high-level):

• Mount the EFS volume on the compute node
• Find (or if needed, create) and conditionally resize a backup EBS volume to copy the EFS content onto
• Trigger an RDS Snapshot
• Do a parallel rsync of shared home (on EFS) to the backup EBS volume
• Snapshot the EBS volume
• Copy the snapshots to the configured DR region

The code

https://bitbucket.org/atlassian/atl-labs-dc-backmac/src/main/

Pre-deployment configuration

1. You must have the AWS SAM CLI installed
2. Your environment must be configured with AWS credentials
3. The access keys or IAM role you're using must have enough permissions to upload Lambdas and deploy the CloudFormation stack via the SAM CLI. Generally, you'll need administrator level of access; either the AWS managed "AdministratorAccess" policy or the effective "Allow *" for at least the following services:
   1. CloudWatch (Rule and Logs)
   2. EC2
   3. IAM
   4. Lambda
   5. S3
   6. SQS
   7. SSM
   8. States
   9. Step Functions

Deploying via SAM

Assuming the prerequisites above have been satisfied, first-time deploys should be as simple as:

sam build
sam deploy --guided

The --guided flag will step through the deploy process, creating the SAM-managed deployment bucket, asking for values for the above parameters, permissions to create IAM roles, and whether to present the changeset for verification, and will save those values to the samconfig.toml file enabling future deploys without the --guided flag.

Updates are handled via the same commands (build and deploy); you can either re-specify the --guided flag to update any template values in the interactive prompts, or you can just update the samconfig.toml directly.

Post-deployment configuration

Make sure at least one stack is tagged with 'backmac_enabled=true'

Backmac will run on the EventBridge schedule entered during the creation of Backmac, and will back up any stack that is tagged as above.

Cleaning up old snapshots

Backmac will also create an additional Lambda (also executed on an EventBridge schedule) to clean up any backups older than their "backup_delete_after" date (which Backmac will set to 30 days after backup by default).

Support

Backmac is not supported by Atlassian; it is provided as is.

More information

More detailed information about the Backup Machine, and migrating to AWS CloudFormation in general, is provided in this video.

[edwin]

Thank you!