Get error message "java.io.IOException: No such file or directory" when creating a new repo

[Alan Yu]

We recently upgraded our Stash 3.7.0 to Bitbucket 6.2.0. But we are getting error when trying to create a new repo. Following is the log shows. Please advise.

2019-05-14 10:36:19,284 ERROR [ajp-nio-8009-exec-3] alan.yu @N3PL6Wx636x118052x0 1o56pa8 129.228.131.221 "POST /projects/ADS3/repos HTTP/1.1" c.a.s.i.w.HandlerExceptionResolverImpl Handler dispatch failed; nested exception is java.lang.Error: java.io.IOException: No such file or directory
org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.Error: java.io.IOException: No such file or directory
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1053)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.stash.internal.spring.security.StashAuthenticationFilter.doFilter(StashAuthenticationFilter.java:85)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doInsideSpringSecurityChain(BeforeLoginPluginAuthenticationFilter.java:112)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:75)
at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94)
at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.bitbucket.internal.scm.git.lfs.servlet.filter.GitLfsLockingFilter.doFilter(GitLfsLockingFilter.java:50)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94)
at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.plugin.connect.plugin.auth.oauth2.DefaultSalAuthenticationFilter.doFilter(DefaultSalAuthenticationFilter.java:69)
at com.atlassian.plugin.connect.plugin.auth.user.ThreeLeggedAuthFilter.doFilter(ThreeLeggedAuthFilter.java:109)
at com.atlassian.jwt.internal.servlet.JwtAuthFilter.doFilter(JwtAuthFilter.java:37)
at com.atlassian.analytics.client.filter.DefaultAnalyticsFilter.doFilter(DefaultAnalyticsFilter.java:38)
at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:39)
at com.atlassian.bitbucket.internal.scm.git.lfs.servlet.filter.GitLfsLockingFilter.doFilter(GitLfsLockingFilter.java:50)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doBeforeBeforeLoginFilters(BeforeLoginPluginAuthenticationFilter.java:90)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:73)
at com.atlassian.stash.internal.request.DefaultRequestManager.doAsRequest(DefaultRequestManager.java:87)
at com.atlassian.stash.internal.hazelcast.ConfigurableWebFilter.doFilter(ConfigurableWebFilter.java:38)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.lang.Thread.run(Thread.java:748)
... 267 frames trimmed
Caused by: java.lang.Error: java.io.IOException: No such file or directory
at sun.nio.ch.FileKey.create(FileKey.java:46)
at sun.nio.ch.SharedFileLockTable.<init>(FileLockTable.java:120)
at sun.nio.ch.FileLockTable.newSharedFileLockTable(FileLockTable.java:47)
at sun.nio.ch.FileChannelImpl.fileLockTable(FileChannelImpl.java:1039)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1106)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155)
at com.atlassian.stash.internal.scm.git.DefaultGitRepositoryLayout.withLock(DefaultGitRepositoryLayout.java:169)
at com.atlassian.stash.internal.scm.git.DefaultGitRepositoryLayout.writeRepositoryConfig(DefaultGitRepositoryLayout.java:141)
at com.atlassian.stash.internal.scm.git.command.AbstractCreateCommand.configureRepository(AbstractCreateCommand.java:106)
at com.atlassian.stash.internal.scm.git.command.AbstractCreateCommand.call(AbstractCreateCommand.java:66)
at com.atlassian.stash.internal.scm.git.command.AbstractCreateCommand.call(AbstractCreateCommand.java:28)
at com.atlassian.stash.internal.scm.PluginScmService.create(PluginScmService.java:197)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.createRepositoryInScm(DefaultRepositoryService.java:697)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.lambda$null$7(DefaultRepositoryService.java:739)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.lambda$createRepositoryWithRetry$8(DefaultRepositoryService.java:739)
at io.atlassian.fugue.retry.RetrySupplier.get(RetrySupplier.java:100)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.createRepositoryWithRetry(DefaultRepositoryService.java:757)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.create(DefaultRepositoryService.java:182)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.create(DefaultRepositoryService.java:86)
at com.atlassian.stash.internal.web.admin.RepositoryAdminController.createRepository(RepositoryAdminController.java:137)
... 32 common frames omitted
Caused by: java.io.IOException: No such file or directory
at sun.nio.ch.FileKey.init(Native Method)
at sun.nio.ch.FileKey.create(FileKey.java:44)
... 52 common frames omitted

[Alan Yu · Accepted solution]

It turned out to be the issue with our mounted file system being CIFS. At this moment CIFS is not supported in Bitbucket server or data center. The only file system mounting type supported is NFS with the following parameters:

lookupcache=pos,noatime,intr,rsize=32768,wsize=32768 0 0

We changed our mounted file system to be NFS with above parameters and then it worked.

[Andrew Goodbody]

I can't find this documented anywhere. Where did you get this information from? We have a CIFS mounted home that used to work on 5.13.0 but after upgrade to 6.6.1 are now getting 'No such file or directory' when trying to create a new repo. I am trying to work out if I absolutely have to migrate to NFS which has various pain points involved.

[Alan Yu]

We got the answer from Atlassian Support team.

[Andrew Goodbody]

Thanks. Moved to NFS and all working now. Using Windows as a NFS server so had to add nolock to the mount line to make it work. Also intr was deprecated a long time ago.

I am really not sure what they are doing so that the underlying protocol makes a difference.