Live Stash backups via filesystem snapshotting

[Eric Dong]

Hi,

I understand that Atlassian recommends shutting off Stash before taking backups to ensure data consistency.

I was thinking about using filesystem snapshotting as part of a live backup policy. If the git repositories and [PostgreSQL] database all reside on the same ZFS volume, and if I take a ZFS snapshot of that volume, and would I be able to restore that snapshot, and would the restored data be consistent and usable by Stash?

Thanks,

Eric

[Boris Berenberg · Accepted solution]

Hey Eric,

That is the nature of filesystem snapshots, you don't know about the data consistency of open files. I would recommend using a script to perform a pgdump first, then you can perform a filesystem snapshot. I am not sure about the rest of the open files, but I think that you should be able to replace / rebuild them if they are corrupted.

Cheers,
Boris

[Eric Dong]

Hi Boris,

Thanks for your response. I wanted to take a single filesystem snapshot, including WAL logs, both for convenience and also to try to keep the database and git repos as close in sync as is reasonably possible. According to the PostgreSQL documentation and what I believe to be true about ZFS snapshots and git, and assuming fsync()s work all the way down to the disks, I believe that restoring such a snapshot should be like recovering from a power outage:

• The PostgreSQL database may lose in-flight changes, but should otherwise be restorable
• git repos may have some corruption due to in-flight changes; it's a pain to clean up but it's doable

We'll schedule our backups around a time of day with the least amount of activity to avoid such data loss.

Thanks,

Eric

[Boris Berenberg]

Hey Eric,

I agree with your assesment. I would recommend testing this in a dev environment first.

Cheers,
Boris