Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,457,140
Community Members
 
Community Events
176
Community Groups

Problem cloning source repository on self-hosted runner with dind

Edited

I work on a project where we have specific computers (configured specs) which pair with our products (hardware). My aim is to configure a self hosted runner so that we can run our test suite on a real product with the same computer and environment that our customers have. 

 

I've been able to set up a self hosted agent with this code

runs-on:
- "self.hosted"
- "ubuntu18.04"

But this gives me a docker container on my self host.

Q1)  is there any way to use bitbucket-pipelines to run the CI on the self hosted machine itself, rather than a docker container inside of the self hosted machine? 

My research has suggested the answer to this Q1 is no, so I then continued to try and develop using the docker container provided by bitbucket for self hosted runner.

The problem now, was that I did not have the environment needed in the docker container provided by bitbucket. Configuring the environment with bash script is doable, but not an attractive option due to increases in build time.

I then found the docker-in-docker (dind) option so that I can run my normal build environment inside the self-hosted runner environment. Essentially then involved adding  a `CLONE_IMAGE` argument to the docker command provided by bitbucket. 

We now reach my current problem in that I get the following error in bitbucket-pipelines:

```

# under build tab

GIT_LFS_SKIP_SMUDGE=1 GIT_SSL_NO_VERIFY=true retry 6 git clone --branch="feature/testing-infrastructure" https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
/tmp/7b85a726-7c76-55d5-b4f4-d7fe74abc8e0/tmp/cloneScript5351248756573077914.sh: line 13: retry: command not found
# under the docker tab
time="2022-07-13T13:38:41.939639712Z" level=info msg="Starting up"
time="2022-07-13T13:38:41.940255296Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2022-07-13T13:38:41.940338812Z" level=warning msg="Binding to IP address without --tlsverify is insecure and gives root access on this machine to everyone who has access to your network." host="tcp://0.0.0.0:2375"
time="2022-07-13T13:38:41.940347755Z" level=warning msg="Binding to an IP address, even on localhost, can also give access to scripts run in a browser. Be safe out there!" host="tcp://0.0.0.0:2375"
```
The logging output from the docker container on the self-hosted runner is:
```
Completing step with result Result{status=ERROR, error=Some(Error{key='runner.bitbucket-pipelines.clone-container-failure', message='We couldn't clone the repository. Try rerunning the pipeline.', arguments={}})}.
```
Q2) What is the problem? What is this retry comment that can't be found? I tried `sudo apt install retry` which worked, but this is not the program that is needed. 
(apologies for the formatting, it seems bitbucket-questions does not want to let me have any more code blocks (?))

 

1 answer

0 votes
Patrik S Atlassian Team Jul 14, 2022

Hello @ciaran_welsh ,

Welcome to Atlassian Community!

For your first question, it's indeed not currently possible to run your pipeline script directly in your Linux machine, just inside a docker container. We do have a feature request to allow un-conteinarized linux runners, which you can find in the following link : 

I would suggest you to add your vote there, since this helps both developers and product managers to understand the interest. Also, make sure you add yourself as a watcher in case you want to receive first-hand updates from that ticket. Please note that all features are implemented with this policy in mind.

As for your second question, my understanding is that you are starting a second container (dind) as part of your build to get an environment similar to what you would have running directly in linux. Is there any reason why you could not build a custom docker image with all the dependencies required for your project, and use this image in the steps of your build, instead of having to start a second container?

In pipelines, you can use any public images available in docker hub and other container registries, or build your own with all the specific dependencies your project requires, and close to the environment your customers have. If that's a possibility, you can find more information about using custom images in the following documentation :

However, if for some reason using a custom docker image is not applicable to your use-case, I'd like to ask you to share the yml file of your build to investigate further the error you reported.

Thank you, @ciaran_welsh .

Kind regards,

Patrik S

Hello Patrik, the non-containerised runner offers several advantages over the docker environment for us because we are testing our software with in-house made hardware (think performance and less complicated network setup). I've upvoted the issue and will ask my team to do the same.

It turns out that I already have a custom container that I can use, I was just trying to use it in the wrong way. I now have the runner working as it should,

Thanks for the help, and for getting back to me quickly, all the best,

Ciaran 

Like Patrik S likes this

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS

Atlassian Community Events