apt-get does not work on pipelines

nyuuzyou May 27, 2021

Hello, community. After apt-get update && apt-get install -y shc git on pipelines, bash: git: command not found. What could be the problem?

yml:

image: debian

pipelines:
default:
- step:
name: 'Install shc'
script:
- apt-get update && apt-get install -y shc git
- step:
name: 'Running the DNSM build'
script:
- git clone https://bitbucket.org/httsmvkcom/donotscanme/
- cd donotscanme
- shc -U -r -f -v main.sh -o circleci.run
- step:
name: 'Checking DNSM'
script:
- ./circleci.run --version

 

1 answer

1 accepted

0 votes
Answer accepted
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
May 28, 2021

Hi @nyuuzyou and welcome to the community.

I assume that you get this error when the git clone command is executed?

Each step in your pipeline runs a separate Docker container. For every step in a Pipelines build, a Docker container starts, the repo is cloned in that container, then the commands of this step's script are executed, and when the script is finished the container gets deleted.

In the example bitbucket-pipelines.yml you posted above, a Docker container starts for the first step, the apt-get command gets executed and afterwards the container gets deleted.

When the second step runs, a new Docker container starts and there is no command to install git there. So, if the image doesn't already have git, the git clone command will fail.

I would suggest combining the first two steps into one, as it looks that you need git in the second step.

If certain tools are not available in the Docker image you use, and you need these tools in different steps, then they'll need to be installed in every step that uses them.

Another alternative would be to create your own Docker image with all the tools you need, and use that instead.

Please feel free to let me know if this helps and if you have any further questions!

Kind regards,
Theodora

nyuuzyou May 28, 2021

That helped me, the git clone command works, thanks. Now not installing shc with the same problem, everything runs in 1 container. On CircleCI the same problem, but in my regular docker container on my local machine it installs and works fine, what could be the problem?

image: debian

pipelines:
default:
- step:
name: 'Running the DNSM build'
script:
- apt-get update && apt-get install -y shc git
- git clone https://bitbucket.org/httsmvkcom/donotscanme/
- cd donotscanme
- shc -U -r -f -v main.sh -o circleci.run
(bash: command not found, because the compiler is not installed the build test also does not work)
- ./circleci.run --version
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
May 31, 2021

Hi @nyuuzyou ,

I believe that the failure of shc command is related to 2 issues:

1st. The version of shc installed

I ran a build with your yml file, and I added another "apt-get install shc" command, which gave me the following output:

Reading package lists...
Building dependency tree...
Reading state information...
shc is already the newest version (3.9.6-1).
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

Checking the releases of shc: https://github.com/neurobin/shc/releases/ I see that the latest version is 4.0.3.

I ran a Docker container with debian image locally on my machine, installed there the latest version of shc as per https://askubuntu.com/a/1208672, and then the shc command was successful. I'm not sure if the earlier version installed by "apt-get install shc" command supports all the options you have in your command.

I would suggest installing the latest version of shc as per the post I linked (I am not sure whether it is possible or how to get it with the "apt-get install" command).


2nd. I believe that the name of the script, main.sh, should be right after the -f option. So, I suggest moving the -v option before the -f one:

shc -U -r -v -f main.sh -o circleci.run


Regarding the ./circleci.run --version command:

I see in the yml you posted originally that you are using the circleci.run in a step other than the one where it is generated.

In order to do that, you'll first need to define circleci.run as an artifact (otherwise, files generated during one step are not shared with the rest of the steps):

So, the yml file in this case would look like that:

image: debian

pipelines:
default:
- step:
name: 'Running the DNSM build'
script:
- apt-get update && apt-get install -y shc git
- git clone https://bitbucket.org/httsmvkcom/donotscanme/
- cd donotscanme
- shc -U -r -v -f main.sh -o circleci.run
artifacts:
- donotscanme/circleci.run
- step:
name: 'Checking DNSM'
script:
- ./circleci.run --version

Please feel free to let me know how it goes.

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events