Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Is Yarn supported for bitbucket-pipelines caching?

I've been using the node cache feature for some time now and have been using yarn to load my load modules with the following pipeline snippet:

pipelines: 
default:
-
step:
caches:
-
node
script:
- yarn install
.
.
.

Observing the performance over several pipelines I haven't noticed any reduction in load time. I'm aware that the cache may not trigger for every pipeline, but I wanted to know if using yarn instead of npm is currently supported or is planned to be supported by pipelines caching.

1 answer

1 accepted

1 vote
Answer accepted
Ana Retamal Atlassian Team Oct 10, 2017

Hi Shane, can you confirm if you're managing your dependencies correctly as explained at Manage dependencies with Yarn.

Cheers,

Ana 

I can't reply for Shane, but I am following the same process outlined in the doc you linked to, but am not seeing any benefits either. I believe I used to before but it's been a while since I noticed no benefits. At this time, my `yarn install`s take over half a minute despite the cache.

Incidentally, at the bottom I see `Cache "node": Skipping upload for previously downloaded cache`

Deleted user Nov 30, 2017

Once install.sh is downloaded and executed, it will check if Yarn is already installed, if it does it'll just continue to the next script. FYI the script has 6.71 KB in size, so there's not too much to download :)

 

Cheers.

I think these answers are missing the point.

This isn't about Bitbucket caching the yarn installation... it's about caching the modules that yarn installs... the dependencies of the project being built.

I routinely see my `yarn install`s taking ~3m... they should take less than a second if my node_modules directory was being cached properly

Like # people like this

I did swap to npm install just for pipeline builds for the reduction in build times, but still use yarn install for my dev environment/production. 

Like aglitchman likes this

After playing around a bit, I've found that the following currently works

pipelines:
  default:
    - step:
        name: Build
        image: node:6.11.4
        caches:
          - yarncustom
        script:
          - curl -o- -L https://yarnpkg.com/install.sh | bash -s -- --version 1.3.2
          - export PATH=$HOME/.yarn/bin:$PATH
          - yarn
          - yarn build
definitions:
  caches:
    yarncustom: /usr/local/share/.cache/yarn/v1

I got here by executing "yarn cache dir" on a Bitbucket Pipeline and then just setup a custom cache as per the documentation. I then executed my build subsequent times with "yarn --verbose". I was able to confirm that builds were using the cache by seeing if it needed to download the modules again.

Like # people like this

@[deleted] Thanks that worked for me too and double checked like you did. The cache download time is almost as slow as pulling from yarn but still a little boost.

Another point, the node docker image contains yarn so don't think there's a reason to install it via curl.. unless I'm missing something.

Yeah, in pipelines, I've found yarn sadly very slow, even with the cache workaround from @[deleted], and have switched back to npm install. Thanks for trying to help all.

I was just about to try this out myself, but yesterday the `yarn install` step started executing in 1 second.

Seams like they might have fixed the caching in the pipelines, can anyone else confirm this?

Using 'yarn' to install all dependencies used to take around 6mins a couple of months ago -> today it only took 40s. After that it took 1s so it seems like the caching is working. 

Hi @nakul_predixt and @sondrel could you share your config file. My yarn command still take like 60-90s to execute.

The normal `-node` cache doesn't work with yarn and triggers the whole installation. 

However a custome cache: ` nodecustom: ./node_modules` works well with yarn and it just takes about 1s. 

Like # people like this

@Oscar OrigenDefault pipeline below:

image: node:11

pipelines:
default:
- step:
caches:
- node
script:
- yarn
- yarn test
- yarn build
Like # people like this

`yarncustom: ./node_modules` solution worked for me.

But the first time you run it will be slow because it will be building the cache.

Subsequent deployments will be quick.

Here's what I needed to do in December 2019 to get Yarn to take as fast as <1s.

image: node:10.17.0

pipelines:
default:
- step:
name: Packages
caches:
- nodecustom
- yarn
script:
- yarn

definitions:
caches:
nodecustom: ./node_modules
yarn: /usr/local/share/.cache/yarn

Yes, nodecustom is necessary as per @Oscar Origen . Just "node" is not good enough. Don't ask why.

For yarn caching, you don't include the /v1 bit. At this time /v6 is being used; it's simpler if we just cache the version's parent directory to future-proof.

Like # people like this

Thanks @Attila Szeremi This should be the accepted answer.

@Attila Szeremi's solution appears to work unless you're using yarn workspaces. I've spent almost two work days trying to figure out how to cache dependencies when using yarn workspaces and have come up with nothing. I'd appreciate any ideas.

 

Also, for what it's worth, Bitbucket Pipelines takes ~13mins to run all my steps whereas AWS CodeBuild runs the same pipeline with no cache in under 5mins.

Like # people like this

I have the same troubles as @Justin McMahon i'm using yarn workspace and yarn install is still 1min not 1s.

Like # people like this

I managed to get the yarn to run in 1s when using yarn workspaces by creating additional caches for each workspace, like this:

 

image: node:lts

pipelines:
default:
- step:
name: Build APP
caches:
- nodeall
- nodeapp
- nodelogin
- nodeapi
- yarn
script:
- yarn
- yarn workspace app build
- step:
name: Build LOGIN
caches:
- nodeall
- nodeapp
- nodelogin
- nodeapi
- yarn
script:
- yarn
- yarn workspace login build
- step:
name: Build API
caches:
- nodeall
- nodeapp
- nodelogin
- nodeapi
- yarn
script:
- yarn
- yarn workspace api build


definitions:
caches:
nodeall: ./node_modules
nodeapp: ./packages/app/node_modules
nodelogin: ./packages/login/node_modules
nodeapi: ./packages/api/node_modules
yarn: /usr/local/share/.cache/yarn
Like Alexey Grinko likes this

This one finally helped in my case, too. Thank you!

I am not using yarn workspaces and still have the same issue.

I have had to setup custom node and yarn caches in order to get my yarn install step to complete in <1s.

definitions:
caches:
nodecustom: ./node_modules
yarncustom: /usr/local/share/.cache/yarn

 

I have had an open ticket with support around yarn caching, including a link to this thread and my solution, and got them to open a ticket to add native support for it.

https://jira.atlassian.com/browse/BCLOUD-20511

Like Asa Baylus likes this

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

📣 Calling Bitbucket Data Center customers to participate in research

Hi everyone, Are you Bitbucket DC customer? If so, we'd love to talk to you! Our team wants to dive deep to understand your long-term plans regarding Bitbucket DC and Atlassian Cloud. Do you plan...

117 views 2 4
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you