Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,293,286
Community Members
 
Community Events
165
Community Groups

Lucene Index Directory over NFS

Hello,

Our company has an instance of Confluence Server under Kubernetes and has the home directory confluence in NFS and it is not possible to change it because the entire Kubernetes infrastructure works on NFS.

My question is this: What is the impact of working under NFS apart from performance problems? We are evaluating changing the product to Confluence Datacenter but we will have the same NFS problem so I need more information about the problems of working with indexing under NFS.

A greeting.

2 answers

1 accepted

0 votes
Answer accepted
Alexis Robert Community Leader Mar 21, 2019

Hi @Confluence , 

 

having the indexes on NFS shares is usually not recommended because of performance issues. The documentation from Atlassian clearly states it:

 

Using Lucene over a NFS mount will cause stability issues with the instance. Some of the more common symptoms are slow search speeds (as NFS mounts are typically slower than local) and running out of open files as detailed in the below KBs.

 

If you believe your NFS infrastructure is performant enough to handle lucene indexes, then you can always use it. You should however be aware of the risks involved.

 

Let me know if you have any questions, 

 

--Alexis

Hi @Alexis Robert ,

So the problems are at the level of operations on disk I/O and does not involve the loss of information. I am right?

In the case that we are on an NFS inside an SSD where the I/O is relatively better, we would not notice the slowness?

I do not understand why the Liferay application that also uses the indexed by Lucene does not have this problem with the NFS

Regards.

No, it's not just to do with speed, although that is a factor.  If you've provisioned a fast disk with NFS over it, it reduces the risk and gives you better performance, but you still should move to a supported file system.

There is a very low risk of data loss.  But a much greater risk of performance issues and down-time. I almost all cases, the worst case is the index becomes totally corrupt and that can easily be fixed by running a complete re-index of the system.  So when you do run into a problem, the impact is "downtime while we re-index".  But there is a small chance of bad data going back into the database as well.  You will find creeping index corruption as well.

You are also totally unsupported if you remain on NFS.  Any support call will quickly lead to "move to a supported file system, and try it again"

Hi

Has there be any development around this issue? We also use Confluence on Kubernetes and it is a big problem to find an option for the index mount as everything works over NFS. 

Is something planned for supporting NFS or any recommended way to go from here?

Regards,

Dominik 

Not on the Atlassian side.  There's nothing they can do - the problems are with NFS and Lucence, not what Atlassian stuff is doing with it.

They only thing Atlassian could do on their side would be to swap to another indexing system.  I do not know if there are plans to do that.

Okay yeah I thought as much. But maybe in order to get ready so that customers can deploy Atlassian-software in a cloud-native way that could be a priority. 

What happens to a cluster if I delete a node and the index of that node is lost? The node would go offline and if a new node joins the cluster it will sync again with the running nodes, is that correct? 

What happens if I delete all nodes and the index is completely lost? Can it be rebuilt on startup?

They already do that, just not for cloud systems they can't support.  See the ami files for AWS as an example.

Yes, dropping a node loses that node, and a new node will synchronise.

If you completely destroy the index, then you will get a lot of errors, but hitting admin -> index -> rebuild will recrate a new set.

Thanks for your answers :)

I've seen the support for AWS with templates etc. Unfortunately I'm not free to deploy to AWS at the moment.

Just thought I'd add, that now the index recovery is in, when you start up the pod the index will recover from the shared home snapshot, however an issue I'm noticing is that even though it appears restored, the popular content tab is only showing the popular questions and not the popular pages, Am speaking to support out it for a workaround if any.

Yep, your index is damaged, the "workaround" is "do not put your index on an unsuitable file system".  Move it to a fast local disk.

As said, its on fast local storage, but local storage gets wiped on pod rest

 

art. unfortunately our kub system isnt built on Block but Vast, maybe one day 

You need to move the storage to a fast local perstistent disk, and stop wiping it with your "pod restart".

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Confluence

Confluence: Where work and wellness meet

Feeling overwhelmed by the demands of work and life? With a 25% increase in the prevalence of anxiety and depression worldwide during the pandemic, for most of us, it’s a resounding yes . 🙋‍♀️ ...

732 views 5 21
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you