how to push only the code changes

zzzro February 3, 2021

Hello,

I am new to bitbucket. We have succcesfully configired bitbucket pipeline to push a repository branch to AWS EC2 instance using Code Build and Code Deploy.

But, the issue is, when we run the pipeline, it creates a zip file of entire repository and replaces the entire content on EC2. We want to push only the changes we did (say a single php file). Zipping entire repostiroy with images, and media files is a overkill and doesn't allow us to promote only bugfix.

How can we do that - any help will be great.

Pls. find our pipeline yml file content below.

- step: script:

- apt-get update

- apt-get install -y zip - zip -r appProd.zip

 - pipe: atlassian/aws-code-deploy:0.2.5

variables:

AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION

AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY

APPLICATION_NAME: $APPLICATION_NAME

S3_BUCKET: $S3_BUCKET

COMMAND: "upload"

ZIP_FILE: "appProd.zip"

VERSION_LABEL: "app-web-prod-1.0.0"

- pipe: atlassian/aws-code-deploy:0.2.5

variables:

AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION

AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY

APPLICATION_NAME: $APPLICATION_NAME

DEPLOYMENT_GROUP: "xx_Www_Pppp"

S3_BUCKET: $S3_BUCKET

COMMAND: "deploy"

WAIT: "true"

VERSION_LABEL: "app-web-prod-1.0.0"

IGNORE_APPLICATION_STOP_FAILURES: "true"

FILE_EXISTS_BEHAVIOR: "OVERWRITE"

1 answer

0 votes
Halyna Berezovska
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
February 5, 2021

@zzzrothanks for using the pipe.

Looks like --file-exists-behviour OVERWRITE does not do actually what you want. It does not compare what files have changed:

First option

In aws docs I see that this param can be described as the following:

OVERWRITE: The version of the file from the application revision currently being deployed replaces the version already on the instance.

So overwrite file-exists-behaviour option does not actually define what files have changed - there is nothing in the doc about that.

First option for you is try to use RETAIN for --file-exists-behaviour. I am not sure it does what you need, it seems it will not also detect the changed files.  Doc is here .

 

Second option

But I'd like to propose one more option, that you could go with:

So if aws deploy does not support RETAIN option in the way you like, I propose you to zip earlier (at build step) only changed files at the build step and pass this file.

I found some examples how can you do it through zip and git tools:

https://askubuntu.com/questions/205421/is-it-possible-to-zip-changed-files-into-a-separate-archive-with-only-the-change

https://stackoverflow.com/questions/4126300/git-how-to-get-all-the-files-changed-and-new-files-in-a-folder-or-zip .

 

P.S. Besides that, I cannot understand part of your case: when you deploy to instance, is there a possibility you have other changes, aimed not for that instance you deploy to?

Just in case we want to give some recommendation for CD best practices:

If you make bugfix to ec2 code, it is better to branch  from that revision (tag, version, whatever) that you have on that ec2 instance.

 

Waiting for your feedback, we are ready to improve our pipes, if we find that we could support some interesting cases, that makes our code more flexible for all users.

 

Regards, Galyna

zzzro February 5, 2021

Thanks Galyna.

We looked into both the solution proposed, thanks for providing those. Unfortunately both will not meet our needs, but will help us for another use case.

Here is our situation.

Developer A, is working on website changes, that are in UAT branch, and is deployed in UAT EC2 instance only. (pipeline is set up to push the changes from UAT branch).

Developer B is checking out a copy from master branch, to make a bug fix, and would like to push it UAT branch and then push it to Master branch (Production EC2 instance thru pipeline) following that.  But Developer B does not want to promote changes made in UAT by Developer A to Master branch (there by to Prod EC2 instance).

Hope I made it clear this time, Is this doable?

ktomk
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
February 6, 2021

@zzzro if I may add a comment: if you intermix, it is better to always push the whole revision in AWS context as otherwise you would need to handle the state at a different place at once.

One exception to the rule I would do with git or rsync based deployments AFAIK this is not supported by AWS code deploy so means running you own (depending on your way of thinking this might be a benefit or not).

Halyna Berezovska
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
February 8, 2021

@ktomkthanks for your comment.

I think , we have similar pipe doing rsync operations - rsync-deploy. So if it is fine, @zzzro , you may try that https://bitbucket.org/atlassian/rsync-deploy/src/master/ .

Thanks for your case you explained in detail. I guess I understood this.

I have some comment: when you push  some changes at uat ec2 instance from uat branch, this is fine. But let's simulate such situation:

- developer A puts change 1 to UAT ec2 from uat branch.

- developer B puts change 2 from master to UAT and to prod. 

When you want to push master changes to UAT (but not the opposite, you want to have change 1,2 at uat and change 2 at master), you may have git conflict in UAT branch: change1 can conflict with change2 .

So here is the question of git conflict issues and it should be first resolved in git.

To resolve this, I think you may want first to merge those changes from master to UAT branch. Usually Bitbucket has such ability: if configured may do this merge, BUT if there are some  conflicts, it will create separate pr to uat branch resolving these conflicts.

When you manually resolve the conflicts, you may merge this pr to UAT branch.

And after merging you actually may deploy to ec2 uat using pipe whatever suits you more.

rsync indeed may work better - it does not just copy, it syncs files, so it pushed only changed and new files you create.

aws-code-deploy does not support syncing files. I think this is for ensuring that all needed code is pushed.

The pipe's choice depends here on your decision.

 

Following this, I guess, you can avoid uat branches changes  appeared in master branch and transitively in prod instance.

 

I hope I got your explanation well.

Looking forward to seeing your answer.

Cheers, Galyna

Halyna Berezovska
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
February 8, 2021

It is actually very important for such situation to sync from master, otherwise you may have issues in your branch every time. If you sync, the task gets simpler both for branching and deployment

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events