Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

Pipeline Hang During Checkout?

Andy Gikling April 25, 2021

I've got a build runner in AWS.  I have a yml file with nothing but echo statements in it.  My repo is running Git LFS for a few folders and the repo will grow quite a bit.

We want to use Bitbucket runners, badly.  We've had enormous success with other servers using Github Actions runners. 

On Github Actions, the git tree persists on the runner.  They basically reset the tree to a hash before building.  How does Bitbucket runners work?  Is there some formal design documentation somewhere?  I hope you don't clone all the code for every build.

I used the generated docker run command on my AWS VM.  Bitbucket immediately tells me it's online - cool.

I run a pipeline build and it just hangs during cloning.  All I get for output on the Bitbucket web ui is:

Runner matching labels:

- self.hostedRunner

name: fsg-aws-g3lsRunner

labels: self.hosted, linux

This is a step in the right direction from a few weeks ago.  The last time I tried this, all the pipeline UI would say is "Starting" or something like that.

My docker runner output is as as follow once I started the pipline run.

[2021-04-25 22:08:03,393] Updating runner state to "ONLINE".
[2021-04-25 22:08:12,906] Getting step StepId{accountUuid={<guid>}, repositoryUuid={<guid>}, pipelineUuid={<guid>}, stepUuid={<guid>}}.
[2021-04-25 22:08:12,913] Getting oauth token for step.
[2021-04-25 22:08:12,918] Getting environment variables for step.
[2021-04-25 22:08:13,503] Getting all artifacts for step.
[2021-04-25 22:08:13,868] Setting up directories.
[2021-04-25 22:08:13,870] Starting log uploader.
[2021-04-25 22:08:13,885] Removing container clone
[2021-04-25 22:08:13,915] Removing container build
[2021-04-25 22:08:13,933] Setting up step timeout: PT2H
[2021-04-25 22:08:14,247] Updating step progress to PULLING_IMAGES.
[2021-04-25 22:08:14,600] Pulling image docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-dvcs-tools:prod-stable.
[2021-04-25 22:08:14,892] Appending log line to main log.
[2021-04-25 22:08:17,009] Pulling image atlassian/default-image:latest.
[2021-04-25 22:08:33,393] Updating runner state to "ONLINE".
[2021-04-25 22:08:50,958] Updating step progress to CLONING.
[2021-04-25 22:08:51,114] Generating clone script.
[2021-04-25 22:08:51,139] Creating container clone.
[2021-04-25 22:08:51,140] Executing clone script in clone container.
[2021-04-25 22:08:56,056] Starting container.
[2021-04-25 22:08:56,370] Adding container log: /var/lib/docker/containers/<guid>-json.log
[2021-04-25 22:08:56,379] Waiting on container to exit.
[2021-04-25 22:08:56,384] Creating exec into container.
[2021-04-25 22:08:56,394] Starting exec into container and waiting for exec to exit.
[2021-04-25 22:09:03,393] Updating runner state to "ONLINE".
[2021-04-25 22:09:33,393] Updating runner state to "ONLINE".
[2021-04-25 22:10:03,393] Updating runner state to "ONLINE".

After that it just keeps pinging me with the ONLINE statement.

The Bitbucket web UI just sits there for what seems like hours then it times out.

My json.log file appears to be empty and the WORKING_DIRECTORY parameter used in the docker run call is clearly not getting filled with any cloned repo.

What's going on here?  Why is clone not working?  Is there a way to get more information out of the system?

Thanks,

~Andy

 

2 answers

Suggest an answer

Log in or Sign up to answer
Andy Gikling April 26, 2021

That issue does seem related.  In that discussion though, it appears the build does work sometimes for the user.  In my case, it never runs, it always hangs during checkout.

Paulo Antoniassi April 26, 2021

Today it's not working at all. I need to restart the runner between each build.

Andy Gikling May 5, 2021

@Paulo Antoniassi try it again.  It's working for me now.  Thanks.

0 votes
James Szklarz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 25, 2021

Hi,

We want to look into what could be causing this issue but we'll need some more information.

Could you please attach the uuids that you used to start the runner so that we can investigate further.

Kind regards,

James Szklarz

Andy Gikling May 5, 2021

James,

Good news.  Seems like we have progress here!

I made a new runner just now and now it starts and runs the build - no more hang on checkout.  The first line of the runner container output says:

Runner starting version: 1.110

This is the first time my hello-world build script has run to completes.  Nice.  It seems fixed, thank you.  

TAGS
AUG Leaders

Atlassian Community Events