Understanding cache in monorepos - lerna

[shx24ztu]

I've got a monorepo which uses Lerna to publish to NPM.

- root

  - packages/componentB

  - packages/componentA

  - packages/all-components

I want to avoid building every single component everytime and thought 

can I not store the packages/compoent/dist in cache

but when i set the path: 

image: node:20

definitions:

  caches:

    build-cache:

      key:

       files:

         - lerna.json

       path: packages/**/dist

..

pipelines:

  pull-requests:

    "**": 

       - step:

         name: "Build divergent"

         caches:

           - build-cache

         script: 

           - node_modules/.bin/lerna run build --since=origin/main

         artifacts:

           - packages/**/dist

the dist/* get sqashed into a single folder without the filestructure so Iit cannot access any package/ * / dist  files

if I download the artifacts this is what it looks like, the build builds the divergent but then errors as it cannot find the rest that is inside the ** folder



Am i doing it wrong?

[Theodora Boudale]

Hi @shx24ztu and welcome to the community!

You can do either of the following:

1. Use as path in the smart cache definition only packages

2. Define three separate smart caches in your yml file, one with path packages/componentA/dist, one with path packages/componentB/dist, and another one with path packages/all-components/dist. Please make sure to add all of them in your step's definition.

I will discuss with my team the behavior when using ** in the cache path.

Kind regards,
Theodora

[shx24ztu]

HI @Theodora Boudale ,

thanks for your answer

do you mean to create a cache for each monorepo item ie:

componentA:
  key:
    files:

      - packages/componentA/package.json


componentB:
  key:
    files:

      - packages/componentB/package.json
...etc

and reference the cache in each different step:

pipelines:
  pull-request:
    "**"
       - step:
          name: "Build changed component"
          caches:
             - componentA
             - componentB
             ...
       

pipelines:
  branches:
    "main"
       - step:
          name: "Publish"
          caches:
             - componentA
             - componentB
             ...
       

at the moment we have about a dozen components, but this will grow a lot, so this ads a bit of a strain to our DRY principle, do you have a way of populating these arrays in any clever way?

[Theodora Boudale]

Hi @shx24ztu ,

do you mean to create a cache for each monorepo item [...]

Yes, this is what I meant with the suggestion suggestion.

do you have a way of populating these arrays in any clever way?

I don't see a way to do that at the moment, but I'm still waiting for feedbck from my team in case I am missing something. Please allow me some more time, and I will get back to you on this.

Kind regards,
Theodora