Sed in pipelines doesnt like the word SECRET

Oli Girling

Had an odd issue with sed and wanted to share it.

 

image: node:14.15.4

pipelines:
branches:
dev:
      - step:
script:
            - npm -g config set user root
            - npm i
            - npm i -g @cloudflare/wrangler
            - echo $CLOUDFLARE_API_TOKEN | wrangler config
            - mv wrangler-sample.toml wrangler.toml
            - sed -i "s/account_id = \"\"/account_id = \"${CLOUDFLARE_ACCOUNT_ID}\"/" wrangler.toml
            - sed -i "s/zone_id = \"\"/zone_id = \"${CLOUDFLARE_ZONE_ID}\"/" wrangler.toml
            - sed -i "s/RAPID_API_SECRET = \"\"/RAPID_API_SECRET = \"${RAPID_API_SECRET}\"/" wrangler.toml
            - sed -i "s/API_KEYS\", id = \"\"/API_KEYS\", id = \"${API_KEYS_NAMESPACE_ID}\"/" wrangler.toml
            - cat wrangler.toml
            - wrangler publish

 This line:

sed -i "s/RAPID_API_SECRET = \"\"/RAPID_API_SECRET = \"${RAPID_API_SECRET}\"/" wrangler.toml

Should replace RAPID_API_SECRET = "" with RAPID_API_SECRET = "${RAPID_API_SECRET}"

But instead it does this:

sed -i "s/$$$$RAPID_API_KEY = \"\"/$$$$RAPID_API_SECRET = \"${$$$$RAPID_API_SECRET}\"/" wrangler.toml 

Notice the $$$$ it adds

If i change it to RAPID_API_KEY it works no problem.

Ive tried a few things and its not the full word RAPID_API_SECRET, looks like its the word SECRET.

 

Theodora Boudale

Hi Oli,

I have attempted to reproduce this issue, but I'm afraid I haven't been able to.

What I did was create a test repo with a wrangler.toml file that included the line RAPID_API_SECRET = "".

I then created a bitbucket-pipelines.yml file using the same image you do (node:14.15.4), but I only included the cat and sed commands, like this:

image: node:14.15.4

pipelines:
  default:
    - step:
        script:
          - cat wrangler.toml
          - sed -i "s/RAPID_API_SECRET = \"\"/RAPID_API_SECRET = \"${RAPID_API_SECRET}\"/" wrangler.toml
          - cat wrangler.toml

This seems to succeed for me, and when the cat wrangler.toml command is executed after the sed command, I can see in the contents of the file that the line RAPID_API_SECRET = "" has been replaced with RAPID_API_SECRET = "$RAPID_API_SECRET".

I would like to ask a few questions so we can investigate this:

1. Do you see the following command you mentioned with the dollar signs in the Pipelines log file?

sed -i "s/$$$$RAPID_API_KEY = \"\"/$$$$RAPID_API_SECRET = \"${$$$$RAPID_API_SECRET}\"/" wrangler.toml

2. Can you open this Pipelines build, then in the Pipelines log select this sed command and expand it, take a screenshot just of that command and the content below it after you expand it, so we can see what you are seeing?

3. Is the line RAPID_API_SECRET = "" replaced in the wrangler.toml file, and if so with that? Can you take a screenshot from the output of the cat command, that shows only what this line has become after you executed the sed command?

4. If you have some time, one suggestion I can make is to debug this build locally with docker as per the instructions here 

Then check the output of the commands locally and and see if you get the same output and behaviour as in Pipelines. This is one way to narrow down whether the issue is specific to the build running in Pipelines or not.

Kind regards,
Theodora