Best practice for pipeline server deployment

[David Ethell]

We are evaluating pipelines for our continuous deployments and I'm hitting something of a roadblock. Our current deployment strategy uses rsync to upload changes to our dev and staging servers. Unfortunately, rsync isn't a great solution with Bitbucket pipelines because the docker container does a fresh git pull of the repo branch. This means rsync will always copy all the files rather than just the changes from the last commit. I realize it will be slightly more efficient since it should still just copy diffs of the data, but it will still always consider all the files rather than commit changes.

Any ideas here about a deployment script? Our platform is CakePHP so it's all PHP files and web assets.

[Tom Bradshaw · Accepted solution]

Hey @David Ethell,

There is a flag you can pass to rsync to make it ignore the timestamps of the files and rely only on the checksum. The flag is --ignore-times which should make your rsync command look something like this:

rsync --ignore-times <folder1> <folder2>

This should prevent unnecessary copies.

However, whether this is a good idea depends on what you're trying to accomplish. Calculating the checksums can be an expensive operation which may lead to longer build times.

Cheers,

Tom

[David Ethell]

For some reason I was worried that checksum calculation would slow things down, but this has greatly improved the run times and now correctly only uploads changed files. Thanks!

[Philip Hodder]

I'm not sure how git-settimes gist works. But it may be running into issues due to timezone differences. Your local commits may be on a different timezone to Pipelines. Pipelines runs on UTC. Your idea here makes sense, but maybe timezones are making things problematic for you here? 

[David Ethell]

I tried using this git-settimes gist to set the checked out timestamps in a way that should allow rsync to only upload changes but so far it does not appear to help.