Hello, I'm building a docker image and inside the Dockefile I'm doing:
RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements.txt
While in the pipeline for the step I have something like:
- step: &build-app-image
name: Build app image
caches:
- docker
- pip
script:
- docker build . --file Dockerfile --tag ${IMAGE}
I've noticed that every time the `pip install` during the build is downloading the libraries all the time. So I believe the caching does not work.
I don't understand why. I've tried all kinds of things.
For example, doing a bind mount with rw permissions, but it then complains that `/root/.cache/pip` doesn't exist but it does because BitBucket pipeline is storying a pip cache and I've already checked that `ls /root/.cache/pip` returns something.
Can anyone step in to help please? In theory what I'm doing (without bind mount but with cache mount) should work perfectly fine.
I think the docker cache not working is exactly reported here as my builds always start from scratch and do not seem to cache any of the build steps.
Welcome to the community, @lcucurachi
For that cache to be meaningful, your Bitbucket runners must be able to access the host filesystem, and future runs should be on the same host. Using cloud runners, I think caching like that is doomed from the start.
You might be able to pull something off if you're using self-hosted runners, but I wouldn't advise using runners that keep any files between runs. Completely ephemeral runners are the current best practice.
If you're using a Python venv or similar, you build the venv, .zip it up, and use it as an artifact. Just download the .zip and decompress it during the build instead of using pip.
ref: https://support.atlassian.com/bitbucket-cloud/docs/runners/ ("Linux Docker-only")
Thanks, that makes sense. It sounds like my only option is to use an external cache using the `--cache-from` and `--cache-to` commands to overcome this issue. Not the best solution but I will try. And it doesn't help that the docker layer caching is broken.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
The caching feature of build layers has been deprecated in RuntimeV3 while in the previous version it doesn't work at all. I think this is quite sad since it's a very important feature.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.