Clear the bitbucket pipeline caches only if the package.json or (particular files have changed)

[Yogeshwar Chaudhari]

Background: 

I am currently working on a node application. I am considering caching "node_modules" that can be reused in between builds. We have a considerable number of projects in the org and hence saving even 5 secs will have a huge impact across the org over time.

What am I looking for:

It would be ideal to provide a way to clear the caches based on changes in specific files.
If those specific files have not changed, use the data from the cache otherwise clear cache -> download new node_modules-> cache again.

For example

pipe: clear-pipeline-caches
   depends: [package.json. package-lock.json]

[Patrik S]

Hello @Yogeshwar Chaudhari ,

Welcome to Atlassian Community!

I would suggest making use of the changeset condition in your pipeline. The changesets condition is defined at the step level and is used to execute a step only if one of the modified files matches the expression in includePaths, like the example below :

- step:
          name: step1
          script:
             - echo "my condition step"
          condition:
              changesets:
                 includePaths:
                   # only xml files directly under path1 directory
                   - "path1/*.xml"
                   # any changes in deeply nested directories under path2
                   - "path2/**"

If the commit has not changed one of that files, the step will be skipped and considered successful.

In a pull-request pipeline, all commits are taken into account, and if you provide an includePathlist of patterns, a step will be executed when at least one commit change matches one of the conditions.For other types of pipelines, only the last commit is considered. More details on this documentation.

So using the changeset condition, along with the pipe atlassian/bitbucket-clear-cache, you could create a step to clear the cache that is only executed when the package.json file is modified :

definitions: 
  steps:
    - step: &clear-cache-if-needed
        name: 'Clear cache if dependency changed'
        script:
          - pipe: atlassian/bitbucket-clear-cache:3.1.1
            variables:
             BITBUCKET_USERNAME: $BB_USERNAME
             BITBUCKET_APP_PASSWORD: $BB_APP_PASS
             CACHES: ["node"]
             DEBUG: "true"
        condition:
              changesets:
                 includePaths:
                   # only if package.json file has changed
                   - "package.json"
pipelines:
  default:
    - step: *clear-cache-if-needed
    - step:
        name: 'Normal Step'
        script:
          - echo "This is a normal step"
  branches:
    develop:
      - step: *clear-cache-if-needed
    main:
      - step: *clear-cache-if-needed

You can configure the atlassian/bitbucket-clear-cache to clear all the caches or just the ones listed in CACHES variable. You can refer to that pipe's documentation here for details on how it can be configured.

You're also welcome to read the below blogpost for further details :

• https://bitbucket.org/blog/automatically-refresh-caches-when-build-dependencies-are-updated

Hope that helps! Let me know if you have any question.

Thank you, @Yogeshwar Chaudhari !

Kind regards,

Patrik S