Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

How to run the bitbucket runner behind proxy.

Hi, I tried to start the Bitbucket runner inside the server that has limited internet access. everything must pass through the proxy. After I tried to start the Bitbucket runner container, I've found an issue on start like this.

Wrapped by: com.netflix.hystrix.exception.HystrixRuntimeException: STARGATE_BITBUCKETCI_REST_SERVICE_PUT_REPOSITORY_RUNNER_STATE timed-out and no fallback available.

 

How do I resolve this problem?

1 answer

0 votes
Caroline R Atlassian Team Oct 13, 2021

Hi, @Chatree Kunjai! Welcome to the community! 

By the output you have shared, it seems that runner failed to reach Atlassian public gateway. In this case, could you please check if you have whitelisted the following IPs:

Please let me know how it goes and don't hesitate to ask for any further information if needed.

Kind regards,
Caroline 

I add the list of an IP Address to /etc/environment and /root/.docker/config.json and now it's working. Thank you for your suggestion.

Now docker of runner is running, but the runner status doesn't go to online. I check on the log and got by this.

 

[2021-10-14 07:38:04,830] An error occurred whilst updating runner state to "ONLINE".
java.util.concurrent.TimeoutException: null
at com.netflix.hystrix.AbstractCommand.handleTimeoutViaFallback(AbstractCommand.java:997)
at com.netflix.hystrix.AbstractCommand.access$500(AbstractCommand.java:60)
at com.netflix.hystrix.AbstractCommand$12.call(AbstractCommand.java:609)
at com.netflix.hystrix.AbstractCommand$12.call(AbstractCommand.java:601)
at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:140)
... 14 common frames omitted
Wrapped by: com.netflix.hystrix.exception.HystrixRuntimeException: STARGATE_BITBUCKETCI_REST_SERVICE_PUT_REPOSITORY_RUNNER_STATE timed-out and no fallback available.
Caroline R Atlassian Team Oct 14, 2021

Hi, @Chatree Kunjai

I'm glad to know the docker of runner is running now. As you mentioned that the runner status is not changing to online, I believe this is behind a proxy and this is not supported by Pipelines Runners yer, but we already have a feature request to implement this:

As you can see on the feature request, we have a workaround that can be performed:

Include the following configuration in `~/.docker/config.json` to force the docker daemon to add the proxy configuration while initiating the docker container.

  "proxies": {
    "default": {
      "httpProxy": "http://username:password@proxy2.domain.com",
      "httpsProxy": "http://username:password@proxy2.domain.com"
    }
  }

I've linked your question to this feature. Please consider adding yourself as a watcher, this way you get updated as we make progress with this. 

If you are not familiar with our Feature Request Policy, you can read more about it here: 

Please, feel free to share any additional questions or concerns regarding this subject. 

Kind regards,
Caroline

Hi, @Caroline R 

Thanks for your reply. I already add the json config for the docker. But still got the same result. I tried to use a workaround by add the environment variable to the runner docker below but still not working.

-e http_proxy=http://username:password@proxy2.domain.com -e https_proxy=http://username:password@proxy2.domain.com

I attach the full log of runner, hope It might be help you to investigate on that.

[2021-10-15 02:32:45,908] Updating runner state to "ONLINE".
[2021-10-15 02:33:15,909] Updating runner status to "ONLINE" and checking for new steps assigned to the runner after 0 seconds and then every 30 seconds.
[2021-10-15 02:33:15,914] Updating runner state to "ONLINE".
[2021-10-15 02:33:45,918] An error occurred whilst updating runner state to "ONLINE".
java.util.concurrent.TimeoutException: null
at com.netflix.hystrix.AbstractCommand.handleTimeoutViaFallback(AbstractCommand.java:997)
at com.netflix.hystrix.AbstractCommand.access$500(AbstractCommand.java:60)
at com.netflix.hystrix.AbstractCommand$12.call(AbstractCommand.java:609)
at com.netflix.hystrix.AbstractCommand$12.call(AbstractCommand.java:601)
at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:140)
... 14 common frames omitted
Wrapped by: com.netflix.hystrix.exception.HystrixRuntimeException: STARGATE_BITBUCKETCI_REST_SERVICE_PUT_REPOSITORY_RUNNER_STATE timed-out and no fallback available.
at com.netflix.hystrix.AbstractCommand$22.call(AbstractCommand.java:822)
at com.netflix.hystrix.AbstractCommand$22.call(AbstractCommand.java:807)
at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:140)
at rx.internal.operators.OnSubscribeDoOnEach$DoOnEachSubscriber.onError(OnSubscribeDoOnEach.java:87)
at rx.internal.operators.OnSubscribeDoOnEach$DoOnEachSubscriber.onError(OnSubscribeDoOnEach.java:87)
at com.netflix.hystrix.AbstractCommand$DeprecatedOnFallbackHookApplication$1.onError(AbstractCommand.java:1472)
at com.netflix.hystrix.AbstractCommand$FallbackHookApplication$1.onError(AbstractCommand.java:1397)
at rx.internal.operators.OnSubscribeDoOnEach$DoOnEachSubscriber.onError(OnSubscribeDoOnEach.java:87)
at rx.internal.operators.OnSubscribeThrow.call(OnSubscribeThrow.java:44)
at rx.internal.operators.OnSubscribeThrow.call(OnSubscribeThrow.java:28)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:142)
at rx.internal.operators.OnSubscribeDoOnEach$DoOnEachSubscriber.onError(OnSubscribeDoOnEach.java:87)
at rx.internal.operators.OnSubscribeDoOnEach$DoOnEachSubscriber.onError(OnSubscribeDoOnEach.java:87)
at com.netflix.hystrix.AbstractCommand$HystrixObservableTimeoutOperator$1.run(AbstractCommand.java:1142)
at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:41)
at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:37)
at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable.run(HystrixContextRunnable.java:57)
at com.netflix.hystrix.AbstractCommand$HystrixObservableTimeoutOperator$2.tick(AbstractCommand.java:1159)
at com.netflix.hystrix.util.HystrixTimer$1.run(HystrixTimer.java:99)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caroline R Atlassian Team Oct 20, 2021

Hi, @Chatree Kunjai

Thanks for getting back to us. It's a bit hard to say what happened without accessing the build in question, so I have created an internal ticket for you using the email of your community account.

You should have received an email with a link to the support ticket. Just in case you haven't received it, please feel free to let me know and I can post the ticket URL here. The ticket will be visible only to you and Atlassian staff, no one else can view its contents even if they have the URL.

Please feel free to let me know if you have any questions.

Kind regards,
Caroline

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

📣 Calling Bitbucket Data Center customers to participate in research

Hi everyone, Are you Bitbucket DC customer? If so, we'd love to talk to you! Our team wants to dive deep to understand your long-term plans regarding Bitbucket DC and Atlassian Cloud. Do you plan...

227 views 2 5
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you