Error getting caches of one step to another

Dipendra Chaudhary

I want to get `.terraform` directory as a cache in the next step so that I can run driftctl command. Here is my bitbucket-pipeline.yml

```

definitions:

  caches:

    terraform: .terraform

pipelines:

  branches:

    Dev:

      - step:

          name: Dev - Validate

          caches:

            - terraform

          script:

            - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID_DEV

            - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY_DEV

            - export AWS_REGION=$AWS_REGION_DEV

            - cp env/dev/backend.tf backend.tf

            - terraform init

            - terraform validate

      - step:

          name: Dev - Driftctl Scan

          image: cloudskiff/driftctl

          script:

            - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID_DEV

            - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY_DEV

            - export AWS_REGION=$AWS_REGION_DEV

            - cd .terraform

            - driftctl scan --from tfstate+s3://dev-tfstate/Terraform/terraform.tfstate

```

Since the previous step use different image, seems the it is not getting the cache. Here is the error I'm getting in second step:

```

cd .terraform

<span>/opt/atlassian/pipelines/agent/tmp/shellScript449014055886284312.sh: cd: line 13: can't cd to .terraform: No such file or directory</span>

```

Patrik S

Hello @Dipendra Chaudhary and welcome to the Community!

For a previously generated cache to be used by a step, you need to configure the step to do so by including the cache section, as you did in the first step: 

 - step:
          name: Step name
          caches:
            - <cache name>
         <rest of step definition>

This will need to be included in every step where you want to use cache. I see your second step does not have the cache section so this is the reason why it's not being used.

Also, if your end goal is to pass files from one step to another, I would recommend using pipelines artifacts instead, because caches will only be automatically refreshed after 7 days.

When you define a file/folder as an artifact in a given step, the artifact will be available in each future step and downloaded by default. Following is an example of defining the terraform folder as an artifact, and listing its content in the next step : 

 

pipelines:
  default:
      - step:
          script:
            - mkdir .terraform
            - touch .terraform/fileA.txt
            - ls -la
          artifacts: 
            - .terraform/** #include all files and subfolder
      - step:
          script:
            - cd .terraform
            - ls -la

Hope that helps! Let me know in case you have any questions.

Thank you, @Dipendra Chaudhary !

Patrik S