need CI/CD only specific subfolders instead of full repository

[Sivanandh]

i am new to CI/CD.

in my staging/UAT server, we are having static configuration files/folder with a different database configuration that is added in .gitignore.

but in repository configuration, we are having production configuration.

when I tried CI/CD its building, it builds a whole repository and deployed it in a staging server with production configuration.

my project folder structure is:

Root Folder:
      -> src (folder)
      -> CDN(css/js - folder)
      -> config (folder)

here I don't want to replace the config folder for every CI/CD.

Please help me how can I achieve this.

my staging deployment step:

- step:

     name: 'Deployment to Staging'

     deployment: staging

     script:

         - echo "Your deployment to staging script goes here..."

         - apt-get update

         - apt-get install -y zip

         - zip -r application1.zip .

         - pipe: atlassian/aws-code-deploy:0.2.5

         variables:

             AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION

             AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID

             AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY

             APPLICATION_NAME: $AWS_APPLICATION_NAME

             S3_BUCKET: $AWS_S3_BUCKET

             LOCAL_PATH: 'src'

             COMMAND: 'upload'

             ZIP_FILE: 'application1.zip'

             VERSION_LABEL: 'my-app-3.0.0'

         - pipe: atlassian/aws-code-deploy:0.2.5

           variables:

              AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION

              AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID

              AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY

              APPLICATION_NAME: $AWS_APPLICATION_NAME

              DEPLOYMENT_GROUP: $AWS_DEPLOYMENT_GROUP

              S3_BUCKET: $AWS_S3_BUCKET

              COMMAND: 'deploy'

              WAIT: 'true'

              VERSION_LABEL: 'my-app-3.0.0'

              IGNORE_APPLICATION_STOP_FAILURES: 'true'

[Theodora Boudale · Accepted solution]

Hi @Sivanandh,

Please allow me to give some context first. When you run a Pipelines build, then for each step of this build:
- a Docker container starts
- the repo is cloned in the directory /opt/atlassian/pipelines/agent/build, which is the base directory of the build
- then the commands in the script of this step (from the yml file) are executed

If your repo has the following directories at its root:
- src
- CDN
- config

then these directories will get cloned in the base directory of the build. During the build, you should be able to change directories with a cd command and execute commands in that specific directory.

For the specific issue you are reporting here, you mention that you don't want the whole repo deployed to the staging server.
Based on the part of the yaml file that you included in your post, I can see you are using a pipe to deploy, and you are deploying the file application1.zip.

This file is generated by the previous command in your script, which is

zip -r application1.zip .

The dot in that command specifies that the zip should include all the files/directories from the current directory.

If you don't want the config folder to be included in the zip, but only the other 2 folders, you can specify this in the command, e.g. like below:

zip -r application1.zip src CDN

Please feel free to let me know if this helps and if you have any questions.

Kind regards,
Theodora

[Sivanandh]

Hello @Theodora Boudale 

Tnx for your response, your suggestion works fine.

But I want to keep static config files and folders on my server

zip -r application1.zip src CDN

but the above command is removed all other files from the staging server.  I don't want to remove the config/vendor folder.

and after I added the above command, I have to create appspec.yml in src, CDN folder, am I correct? please correct me I am wrong.

can I keep the config & vendor folder in the root folder without any build process from CI/CD

can I implement ci/cd only for the src & CDN folder,? I want to keep all other folders in my root folder without ci/cd
is it possible to achieve this?

[Theodora Boudale]

Hi @Sivanandh,

Thank you for your reply.

First, allow me to say that I am not very familiar with CodeDeploy, so the help I can provide with details of a solution is limited. After doing some search, I found that it may be standard behavior from CodDeploy to delete all existing files and replace them with the deployment package:

• https://docs.aws.amazon.com/codedeploy/latest/userguide/deployments-rollback-and-redeploy.html

During the deployment process, CodeDeploy deletes all the files that are part of the previous successful deployment.

Considering that, some ideas to work around this issue:

1) Commit the config directory of the staging server to the repo, and include it in the zip you create for deployment.
Is that a possibility for you?

2) If you don't want to or cannot commit the staging server's config directory in the repo, then would the following be an option?

during the step 'Deployment to Staging'
- delete the config directory that is already there (since it's the production server's config)
- copy the staging server's config from EC2 (e.g. with scp if that's possible)
- then include it in the zip you create for deployment.

3) I also found the following thread in StackOverflow where a similar issue is discussed, and the answer there mentions a way to configure CodeDeploy so that files are retained:

https://stackoverflow.com/questions/48464830/aws-codedeploy-how-to-stop-it-from-deleting-files

Kind regards,
Theodora

[Sivanandh]

Understood. Thanks @Theodora Boudale .