We've been in production on Lucene over NFS for about 4 years now. Though we've had performance issues related to NFS (similar to those mentioned on this thread), we've only seen some reliability issues. Index writing I/O timeout exceptions are the primary issue. We've addressed these by implementing retry logic. This kept transactional consistency and avoided corruption. I can't recall specifically where these timeouts occurred, but I do remember that on our version of Lucene at the time (3.0.2) the timeout was not configurable, and defaulted to 5 seconds. Had it been configurable, we could have reduced how frequently we needed to rollback and retry.
On another point, NFS performance can be greatly enhanced with servers that have extra memory for mapping the index files. We found after initial warmup and loading of Lucene that queries performance very well since most of the data needed to execute them is cached locally. Also, keep in mind that local disk is not a free ride either. This takes you down data replication. You end up repeating indexing once per replica. You also may have to move the indices around as you add/remove/restart nodes. We are moving to this architecture with a new product, so I am just now starting to understand the trade-offs. Hope that helps. -John On 10/2/12 8:01 AM, "Jong Kim" <jong.luc...@gmail.com> wrote: >Thank you all for reply. > >So it soudns like it is a known fact that the performance would suffer >rather significantly when the index files are accessed over NFS. But how >about reliability and robustness (which seems even more important)? Isn't >there any increased possibility for intermittent errors such as index file >corruption (due to cache inconsistency, difference in delete semantics, >etc.) when using NFS? Has anyone run into such trouble? Or is it strictly >just a performance issue? > >/Jong >On Tue, Oct 2, 2012 at 5:17 AM, Paul Libbrecht <p...@hoplahup.net> wrote: > >> My experience in the Lucene 1.x times were a factor of at least four in >> writing to NFS and about two when reading from there. I'd discourage >>this >> as much as possible! >> >> (rsync is way more your friend for transporting and replication à la >>solr >> should also be considered) >> >> paul >> >> >> Le 2 oct. 2012 à 11:10, Ian Lea a écrit : >> >> > You'll certainly need to factor in the performance of NFS versus local >> disks. >> > >> > My experience is that smallish low activity indexes work just fine on >> > NFS, but large high activity indexes are not so good, particularly if >> > you have a lot of modifications to the index. >> > >> > You may want to install a custom IndexDeletionPolicy. See the >> > javadocs for details with specific reference to NFS. >> > >> > >> > -- >> > Ian. >> > >> > On Tue, Oct 2, 2012 at 3:21 AM, Vitaly Funstein <vfunst...@gmail.com> >> wrote: >> >> How tolerant is your project of decreased search and indexing >> performance? >> >> You could probably write a simple test that compares search and write >> >> speeds of local and NFS-mounted indexes and make the decision based >>on >> the >> >> results. >> >> >> >> On Mon, Oct 1, 2012 at 3:06 PM, Jong Kim <jong.luc...@gmail.com> >>wrote: >> >> >> >>> Hi, >> >>> >> >>> According to the Lucene In Action (Second Edition), the section >>2.11.2 >> >>> "Accessing an index over a remote file system" explains that there >>are >> >>> issues related to accessing a Lucene index across remote file system >> >>> including NFS. >> >>> >> >>> I'm particuarly interested in NFS compatibility, and wondering if >> there has >> >>> been any work done to solve or mitigate this problem. Has this issue >> been >> >>> addressed? If not, are there some reliable work-arounds that make >>this >> >>> possible at the expense of some sacrifice in other areas? >> >>> >> >>> Any information would be greatly appreciated, since my project >>heavily >> >>> depends on the feasibility of this. >> >>> >> >>> Thanks >> >>> /Jong >> >>> >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> > For additional commands, e-mail: java-user-h...@lucene.apache.org >> > >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
