Creating repository fails with 500

[Thomas Nielsen]

Creating a new repository on the server results in a 500 error.

Adding new projects works. I cannot add repositories to either old projects or the new one.

Pushing code works existing repositories work.

Bitbucket is running in a single-node on-prem kubernetes "cluster"

Logs in debug mode:

2020-07-20 15:44:31,259 INFO [hz.hazelcast.event-3 c.h.i.diagnostics.HealthMonitor Switching to log level [DEBUG2020-07-20 15:44:31,650 DEBUG [http-nio-7990-exec-8 thomas @XUAP0Sx944x208x0 5bjk5e 10.244.1.1 "GET /rest/api/latest/inbox/pull-requests/count HTTP/1.1" c.a.s.i.rest.inbox.InboxResource Retrieving pull request count for user Thomas2020-07-20 15:44:35,636 DEBUG [http-nio-7990-exec-5 thomas @XUAP0Sx944x212x0 5bjk5e 10.244.1.1 "POST /projects/SM/repos HTTP/1.1" c.a.s.i.s.git.command.CreateCommand Creating repository for SM/website (26)2020-07-20 15:44:35,889 DEBUG [http-nio-7990-exec-5 thomas @XUAP0Sx944x212x0 5bjk5e 10.244.1.1 "POST /projects/SM/repos HTTP/1.1" c.a.s.i.s.git.command.CreateCommand Configuring repository for SM/website (26)2020-07-20 15:44:35,906 DEBUG [http-nio-7990-exec-5 thomas @XUAP0Sx944x212x0 5bjk5e 10.244.1.1 "POST /projects/SM/repos HTTP/1.1" c.a.s.i.s.git.command.CreateCommand Cleaning up partially-created repository for SM/website (26)2020-07-20 15:44:36,135 ERROR [http-nio-7990-exec-5 thomas @XUAP0Sx944x212x0 5bjk5e 10.244.1.1 "POST /projects/SM/repos HTTP/1.1" c.a.s.i.w.HandlerExceptionResolverImpl Handler dispatch failed; nested exception is java.lang.Error: java.io.IOException: No such file or directoryorg.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.Error: java.io.IOException: No such file or directory at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1055) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.bitbucket.internal.ratelimit.servlet.filter.RateLimitFilter.doFilter(RateLimitFilter.java:75) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.stash.internal.web.auth.AuthorizationFailureInterceptor.doFilterInternal(AuthorizationFailureInterceptor.java:39) at com.atlassian.stash.internal.spring.security.StashAuthenticationFilter.doFilter(StashAuthenticationFilter.java:85) at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doInsideSpringSecurityChain(BeforeLoginPluginAuthenticationFilter.java:112) at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:75) at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94) at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94) at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.plugin.connect.plugin.auth.oauth2.DefaultSalAuthenticationFilter.doFilter(DefaultSalAuthenticationFilter.java:69) at com.atlassian.plugin.connect.plugin.auth.user.ThreeLeggedAuthFilter.doFilter(ThreeLeggedAuthFilter.java:109) at com.atlassian.jwt.internal.servlet.JwtAuthFilter.doFilter(JwtAuthFilter.java:37) at com.atlassian.analytics.client.filter.DefaultAnalyticsFilter.doFilter(DefaultAnalyticsFilter.java:26) at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:48) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doBeforeBeforeLoginFilters(BeforeLoginPluginAuthenticationFilter.java:90) at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:73) at com.atlassian.stash.internal.request.DefaultRequestManager.doAsRequest(DefaultRequestManager.java:84) at com.atlassian.stash.internal.hazelcast.ConfigurableWebFilter.doFilter(ConfigurableWebFilter.java:38) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.lang.Thread.run(Thread.java:748) ... 275 frames trimmedCaused by: java.lang.Error: java.io.IOException: No such file or directory at sun.nio.ch.FileKey.create(FileKey.java:46) at sun.nio.ch.SharedFileLockTable.<init>(FileLockTable.java:120) at sun.nio.ch.FileLockTable.newSharedFileLockTable(FileLockTable.java:47) at sun.nio.ch.FileChannelImpl.fileLockTable(FileChannelImpl.java:1039) at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1106) at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155) at com.atlassian.stash.internal.scm.git.DefaultGitRepositoryLayout.withLock(DefaultGitRepositoryLayout.java:169) at com.atlassian.stash.internal.scm.git.DefaultGitRepositoryLayout.writeRepositoryConfig(DefaultGitRepositoryLayout.java:141) at com.atlassian.stash.internal.scm.git.command.AbstractCreateCommand.configureRepository(AbstractCreateCommand.java:106) at com.atlassian.stash.internal.scm.git.command.AbstractCreateCommand.call(AbstractCreateCommand.java:66) at com.atlassian.stash.internal.scm.git.command.AbstractCreateCommand.call(AbstractCreateCommand.java:28) at com.atlassian.stash.internal.scm.PluginScmService.create(PluginScmService.java:199) at com.atlassian.stash.internal.repository.DefaultRepositoryService.createRepositoryInScm(DefaultRepositoryService.java:765) at com.atlassian.stash.internal.repository.DefaultRepositoryService.lambda$null$7(DefaultRepositoryService.java:807) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) at com.atlassian.stash.internal.repository.DefaultRepositoryService.lambda$createRepositoryWithRetry$8(DefaultRepositoryService.java:807) at io.atlassian.fugue.retry.RetrySupplier.get(RetrySupplier.java:100) at com.atlassian.stash.internal.repository.DefaultRepositoryService.createRepositoryWithRetry(DefaultRepositoryService.java:825) at com.atlassian.stash.internal.repository.DefaultRepositoryService.create(DefaultRepositoryService.java:196) at com.atlassian.stash.internal.repository.DefaultRepositoryService.create(DefaultRepositoryService.java:94) at com.atlassian.stash.internal.web.admin.RepositoryAdminController.createRepository(RepositoryAdminController.java:138) ... 32 common frames omittedCaused by: java.io.IOException: No such file or directory at sun.nio.ch.FileKey.init(Native Method) at sun.nio.ch.FileKey.create(FileKey.java:44) ... 52 common frames omitted

Validated write permissions from pod

bitbucket@bitbucket-64588d47fb-gwxvj:~/shared/data/repositories$ echo writable > foo
bitbucket@bitbucket-64588d47fb-gwxvj:~/shared/data/repositories$ cat foo
writable

I tried restarting the pod as well but that did not help.

Can someone help with my next steps for getting this working again ?

[Thomas Nielsen]

Further googling reveals that Bitbucket does not support CIFS.

Since I was only having it in kubernetes for the convinience, I just migrated to a hostpath volume instead of a cifs voume

[Thomas Nielsen]

https://confluence.atlassian.com/enterprise/bitbucket-data-center-faq-776663707.html