nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How to deploy added or modified files?

Thiery Laverdur1

My company is looking into using Bamboo for web applications and websites. If we need to make a small change the whole repo doesn't need to be rebuilt and deployd. Is there a way to create an artifact with only the added or modified files?

Also lets say that we delete files from the repo, how do we delete them from our environment as well?

Accepted answers

Guillaume Dupér1

OK here it is :

Step -1: Create a tag with the commit that went to production (the tag name should be prod-master-ANYTHINGELSE ) . Replace anything else by maybe the date or anything else you like. If you don't follow this , it's not a big deal, just change the script in step 2.

Step 1 ... configure bamboo to do a checkout of your GIT repository.

Step 2 ... get the diff of modified files from the latest production tag (assuming the tagname as "prod-master-" in it , else just change the line below accordingly

Get commit id for last tag

DIFF_COMMIT=`git tag -l | grep -i prod-${branch}- | tail -1`

Get file list between current commit and DIFF_COMMIT (you can change the value of ref if you want to put it more powerful)

latest_commit=`git rev-parse ${branch}`
ref=$latest_commit
DIFF_FILE_PATH="./modified.txt"
git log --name-only --pretty=format: --diff-filter=AM ${DIFF_COMMIT}..${ref} | grep -v '^$' | sort -u > ${DIFF_FILE_PATH}

# create file with a list of all files that got deleted, copied or renamed,
# the diff-filter only gets the deleted files, 
# the pretty=format: removes all commits lines (which means format the commit line having output as "") in this case it adds a blank line then we remove it with
# grep -v '^$' and we finish with a unique sorting.
# this is used to remove them on the deployment server
DELETED_FILE_PATH=./deleted.txt
git log --pretty=format: --name-only --diff-filter=D ${DIFF_COMMIT}..${ref} | grep -v '^$' | sort -u > ${DELETED_FILE_PATH}

	diff_files=( `cat ${DIFF_FILE_PATH}` )
tar -p -cf diff-files.tar --files-from=${DIFF_FILE_PATH} --ignore-failed-read

In step 2 , you got a tar file with all modified files, a modified.txt file and a deleted.txt file (you can even add an md5 file of the tar file in case you want to make sure there is no problem in the file transfer

Step 3 , tar all output files from step 2

 md5sum diff-files.tar > diff-files.tar.md5
tar -czf package.tgz diff-files.tar modified.txt deleted.txt diff-files.tar.md5

Step 4 Send package to deployment server and tag release using the naming convention in step 2

Step 5 on the deployment server , create a script that will delete the deleted files and untar package.tgz

tar -xzf package.tgz
md5sum diff-files.tar.md5

cd YOURDOCUMENTLOCATION

rm -f `cat /location/of/your/package/deleted.txt`
tar -xf /location/of/your/package/diff-files.tar

 
That's it ... You now have a really small package and all files that got deleted in git ,  got deleted on the deployment server as well.

If you need anything else, please let me know !

Guillaume

All comments

Guillaume Dupér1

This is what I have setup at my office , I am using git to get a diff of what changed between last production release (which gets tagged when deployed to production) and the git diff also has a lit of all deleted files which I just do a delete FIRST in production before copying the files.

To be more specific :

1- Always tag your production deployments

2- do a git diff ( I can find you the exact command here , but I don't know if you are using git) to get all files that have been modified but not deleted and spit that in a modified.txt file

3- do a git diff for all deleted files and spit that in the deleted.txt file

4-tar only the files which you pick up the list from the modified.txt file PLUS the deleted.txt file

5-deploy to production and tag

6-Once in production, extract tar file

7-Delete all files based on the list you got in the deleted.txt file

8-untar the tar file which will overwrite your changes and VOILA !

You get a small package and deleted the files you don't want.

There is something tricky about the ORDER of the delete and insert...

If in git , you have added a file, then deleted the file, then readded the file, then the file will be in the modified.txt AND the deleted.txt ...

If you do a delete BEFORE , then it will delete the file , then readd it after , no prob.

If in git you delete a file, readd it after and delete it again... they will be in both modified.txt and deleted.txt, then when it will try to add it to the tar , it will complain then you need to add an option to tar to make sure it doesn't crash when it tries to add a non existent file.

Then the script will the delete the file and it will not readd it because it won't be in the tar at all.

All that to say that you REALLY need to do your delete BEFORE you untar your file.

If you need more help , please let me know !

Thiery Laverdur1

Guillaume, thanks for the response. This is great. Could you share more details about the scripts that you use for each step? Thanks!

Guillaume Dupér1

OK here it is :

Step 1 ... configure bamboo to do a checkout of your GIT repository.

Step 2 ... get the diff of modified files from the latest production tag (assuming the tagname as "prod-master-" in it , else just change the line below accordingly

Get commit id for last tag

DIFF_COMMIT=`git tag -l | grep -i prod-${branch}- | tail -1`

Get file list between current commit and DIFF_COMMIT (you can change the value of ref if you want to put it more powerful)

latest_commit=`git rev-parse ${branch}`
ref=$latest_commit
DIFF_FILE_PATH="./modified.txt"
git log --name-only --pretty=format: --diff-filter=AM ${DIFF_COMMIT}..${ref} | grep -v '^$' | sort -u > ${DIFF_FILE_PATH}

# create file with a list of all files that got deleted, copied or renamed,
# the diff-filter only gets the deleted files, 
# the pretty=format: removes all commits lines (which means format the commit line having output as "") in this case it adds a blank line then we remove it with
# grep -v '^$' and we finish with a unique sorting.
# this is used to remove them on the deployment server
DELETED_FILE_PATH=./deleted.txt
git log --pretty=format: --name-only --diff-filter=D ${DIFF_COMMIT}..${ref} | grep -v '^$' | sort -u > ${DELETED_FILE_PATH}

	diff_files=( `cat ${DIFF_FILE_PATH}` )
tar -p -cf diff-files.tar --files-from=${DIFF_FILE_PATH} --ignore-failed-read

Step 3 , tar all output files from step 2

 md5sum diff-files.tar > diff-files.tar.md5
tar -czf package.tgz diff-files.tar modified.txt deleted.txt diff-files.tar.md5

Step 4 Send package to deployment server and tag release using the naming convention in step 2

Step 5 on the deployment server , create a script that will delete the deleted files and untar package.tgz

tar -xzf package.tgz
md5sum diff-files.tar.md5

cd YOURDOCUMENTLOCATION

rm -f `cat /location/of/your/package/deleted.txt`
tar -xf /location/of/your/package/diff-files.tar

 
That's it ... You now have a really small package and all files that got deleted in git ,  got deleted on the deployment server as well.

If you need anything else, please let me know !

Guillaume