Warning Ubuntu 20.04LTS users before upgrading to Bitbucket 8.0!

tdeleeuw May 17, 2022

Hi

I have been hit by an issue when upgrading to BitBucket 8.0 on my Ubuntu 20.04 server. The git version available in Ubuntu packages is 2.25 while BitBucket 8.0 requires 2.31+ (the log is hidden under <BITBUCKET_HOME>/mesh/log) While the "main log" only complains

WARN [spring-startup] o.s.w.c.s.XmlWebApplicationContext Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Failed to start bean 'meshController'; nested exception is com.atlassian.bitbucket.repository.RepositoryOfflineException: bitbucket.scm.repository.offline
ERROR [hz.hazelcast.event-1] c.h.s.i.e.impl.EventServiceImpl [YOURIP]:5701 [bitbucket] [3.12.12] hz.hazelcast.event-1 caught an exception while processing:com.hazelcast.spi.impl.eventservice.impl.LocalEventDispatcher@9955b2a
com.atlassian.bitbucket.repository.RepositoryOfflineException: bitbucket.scm.repository.offline
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.translateOffline(DefaultErrorTranslator.java:382)
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.translateIfKnownCause(DefaultErrorTranslator.java:278)
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.maybeTranslate(DefaultErrorTranslator.java:55)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.maybeTranslate(AbstractFutureResponseObserver.java:203)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.translateCause(AbstractFutureResponseObserver.java:292)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.get(AbstractFutureResponseObserver.java:229)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:145)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:133)
at com.atlassian.stash.internal.scm.git.mesh.GrpcManagementClient.updateConfig(GrpcManagementClient.java:275)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at jdk.internal.reflect.GeneratedMethodAccessor220.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at com.atlassian.stash.internal.mesh.ConfigureSidecarTask.configure(ConfigureSidecarTask.java:75)
at com.atlassian.stash.internal.mesh.DefaultMeshController.maybeConfigureSidecar(DefaultMeshController.java:255)
at com.atlassian.stash.internal.mesh.DefaultMeshController.access$700(DefaultMeshController.java:70)
at com.atlassian.stash.internal.mesh.DefaultMeshController$ControlPlaneStateListener.lambda$onUpdated$0(DefaultMeshController.java:287)
at java.base/java.util.Optional.ifPresent(Optional.java:183)
at com.atlassian.stash.internal.mesh.DefaultMeshController$ControlPlaneStateListener.onUpdated(DefaultMeshController.java:286)
at com.atlassian.stash.internal.mesh.DefaultMeshController$ControlPlaneStateListener.entryAdded(DefaultMeshController.java:273)
at com.hazelcast.map.impl.MapListenerAdaptors$1$1.onEvent(MapListenerAdaptors.java:67)
at com.hazelcast.map.impl.MapListenerAdaptors$1$1.onEvent(MapListenerAdaptors.java:64)
at com.hazelcast.map.impl.InternalMapListenerAdapter.onEvent(InternalMapListenerAdapter.java:56)
at com.hazelcast.map.impl.InternalMapListenerAdapter.onEvent(InternalMapListenerAdapter.java:35)
at com.hazelcast.map.impl.event.MapEventPublishingService.callListener(MapEventPublishingService.java:172)
at com.hazelcast.map.impl.event.MapEventPublishingService.dispatchEntryEventData(MapEventPublishingService.java:184)
at com.hazelcast.map.impl.event.MapEventPublishingService.dispatchEvent(MapEventPublishingService.java:83)
at com.hazelcast.map.impl.event.MapEventPublishingService.dispatchEvent(MapEventPublishingService.java:47)
at com.hazelcast.map.impl.MapService.dispatchEvent(MapService.java:105)
at com.hazelcast.map.impl.MapService.dispatchEvent(MapService.java:78)
at com.hazelcast.spi.impl.eventservice.impl.LocalEventDispatcher.run(LocalEventDispatcher.java:64)
at com.hazelcast.util.executor.StripedExecutor$Worker.process(StripedExecutor.java:244)
at com.hazelcast.util.executor.StripedExecutor$Worker.run(StripedExecutor.java:227)
... 15 frames trimmed
Caused by: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
at io.grpc.Status.asRuntimeException(Status.java:535)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:479)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.LastSeenClientInterceptor$LastSeenClientListener.onClose(LastSeenClientInterceptor.java:40)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.StatefulClientCallListener.onClose(StatefulClientCallListener.java:34)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.DeadlinePropagatingClientInterceptor$DeadlinePropagatingListener.onClose(DeadlinePropagatingClientInterceptor.java:156)
at com.atlassian.stash.internal.scm.git.mesh.ErrorHandlingClientInterceptor$ErrorHandlingCall$1.onClose(ErrorHandlingClientInterceptor.java:136)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.lang.Thread.run(Thread.java:829)
... 1 common frames omitted
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:7777
Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused
at io.netty.channel.unix.Errors.newConnectException0(Errors.java:155)
at io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128)
at io.netty.channel.unix.Socket.finishConnect(Socket.java:320)
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710)
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687)
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)

ERROR [spring-startup] c.a.j.s.w.s.JohnsonDispatcherServlet SpringMVC dispatcher [springMvc] could not be started

The solution is to install from  official GIT release (see https://git-scm.com/download/linux):

add-apt-repository ppa:git-core/ppa
apt update
apt install git

 

That will install a more recent git version that will be higher than 2.31. Bitbucket happily start after this. 

3 comments

tdeleeuw May 18, 2022

Side note, on https://confluence.atlassian.com/bitbucketserver/install-bitbucket-server-on-linux-868976991.html Atlassian says 

Do not upgrade Git to the latest version until you check the Supported Platforms page for which version of Git is currently supported.

Which is unfortunate, as using the stable git repo will upgrade it automatically as soon as a new stable version is available... So Using stable git from Ubuntu is not an option anymore (just providing security fixes), and using the official repo (providing security fixes and latest features) is "forbidden" by Atlassian. 

Any Atlassian representative to provide a viable solution?

Like # people like this
jcbsv June 8, 2022

For Debian bullseye (v11.3) you can install git v2.34.1 from backports:

apt install git/bullseye-backports

 which is sufficient to start Bitbucket v8.

 

Note: bullseye-backports have to be in /etc/sources.list:

deb http://deb.debian.org/debian bullseye-backports main
Adrien Ragot _Requirement Yogi_ September 20, 2022

Thank you @jcbsv , and thank you for providing the detailed solution. 

tdeleeuw October 19, 2022

If your system has been updated to latest GIT version (2.38.1 or even 2.38.0) that is solving  CVE-2022-39253 (Score 5.5) and CVE-2022-39260 (score 8.6!) on version 8.4.1 or even 8.5.0 that is currently the latest release,it prevents bitbucket to start, only solution is to manually install a version 2.37.4 or higher from the 2.37 branch (2.38.x is not supported even on the latest Bitbucket 8.5.0)

Hope Atlassian will support 2.38 soon and maybe in 8.5.1 they might have to release if they use StringSubstitutor of commons-text as they include a vulnerable version of commons-text 1.9.

Like # people like this
Celso Freitas October 31, 2022

same problem here, and solved following your instructions!!! thank u so much!

Comment

Log in or Sign up to comment
TAGS
AUG Leaders

Atlassian Community Events