Understanding cache in monorepos - lerna

Question

I've got a monorepo which uses Lerna to publish to NPM.

- root

- packages/componentB

- packages/componentA

- packages/all-components

I want to avoid building every single component everytime and thought

can I not store the packages/compoent/dist in cache

but when i set the path:

image: node:20

definitions:

caches:

build-cache:

key:

files:

- lerna.json

path: packages/**/dist

..

pipelines:

pull-requests:

"**":

- step:

name: "Build divergent"

caches:

- build-cache

script:

- node_modules/.bin/lerna run build --since=origin/main

artifacts:

- packages/**/dist

the dist/* get sqashed into a single folder without the filestructure so Iit cannot access any package/ * / dist  files

if I download the artifacts this is what it looks like, the build builds the divergent but then errors as it cannot find the rest that is inside the ** folder

Am i doing it wrong?

Theodora Boudale · Answer

Hi @shx24ztu,

I'm afraid the directory structure cannot be preserved if you use a single cache definition with a wildcard in the path.

I went ahead and created a request to preserve the dir structure in our issue tracker:

* https://jira.atlassian.com/browse/BCLOUD-23246

I would suggest adding your vote to that request (by selecting the Vote for this issue link) to express your interest. You are more than welcome to leave feedback, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

Implementation of features is done as per our policy here and any updates will be posted in the feature request.

In the meantime, you can use multiple cache definitions, one for each directory.

Kind regards,
Theodora

Theodora Boudale · Answer

Hi @shx24ztu ,

do you mean to create a cache for each monorepo item [...]

Yes, this is what I meant with the suggestion suggestion.

do you have a way of populating these arrays in any clever way?

I don't see a way to do that at the moment, but I'm still waiting for feedbck from my team in case I am missing something. Please allow me some more time, and I will get back to you on this.

Kind regards,
Theodora

shx24ztu · Answer

HI @Theodora Boudale ,

thanks for your answer

do you mean to create a cache for each monorepo item ie:

componentA:
  key:
    files:

- packages/componentA/package.json

componentB:
  key:
    files:

- packages/componentB/package.json
...etc

and reference the cache in each different step:

pipelines:
  pull-request:
    "**"
       - step:
          name: "Build changed component"
          caches:
             - componentA
             - componentB
             ...

pipelines:
  branches:
    "main"
       - step:
          name: "Publish"
          caches:
             - componentA
             - componentB
             ...

at the moment we have about a dozen components, but this will grow a lot, so this ads a bit of a strain to our DRY principle, do you have a way of populating these arrays in any clever way?