> (a) Accessing index files over NFS from a "single" physical process on a > single computer is safe and can be made to work.
To add: This means writing only. Reading is fine from as many threads as you like - and using MMapDirectory for best performance. The problem with writing is that locking of index does not work with NFS (see my other mail). So use one server (ideally one that has the file system locally) and write your index. On the other servers which are using the same filesystem (which is local to the writer), you can use NFS to access to the writing server. Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Jong Kim [mailto:jong.luc...@gmail.com] > Sent: Tuesday, October 02, 2012 5:20 PM > To: java-user@lucene.apache.org > Subject: Re: Lucene index on NFS > > OK, so it sounds like I'm hearing that > > (a) Accessing index files over NFS from a "single" physical process on a > single > computer is safe and can be made to work. > > (b) Accessing index files over NFS from "multiple" processes/machines might be > problematic > > (c) In all cases, the performance would be lesser over NFS Thanks /Jong On > Tue, > Oct 2, 2012 at 9:36 AM, Tommaso Teofili > <tommaso.teof...@gmail.com>wrote: > > > Ok, that saves you from concurrency issue, but in my experience is > > just much slower than local file system, so still NFS can be used but > > with some tradeoff on performance. > > > > My 2 cents, > > Tommaso > > > > 2012/10/2 Jong Kim <jong.luc...@gmail.com> > > > > > The setup is I have a home-grown server process that has exclusive > > > access to the index files. All reads and writes are done through > > > this server. No other process is reading the same index files > > > whether it's local or over NFS. > > > /Jong > > > On Tue, Oct 2, 2012 at 8:56 AM, Ian Lea <ian....@gmail.com> wrote: > > > > > > > I agree that reliability/corruption is not an issue. > > > > > > > > I would also put it that performance is likely to suffer, but > > > > that's not certain. A fast disk mounted over NFS can be quicker > > > > than a slow local disk. And how much do you care about > > > > performance? Maybe it would be fast enough over NFS to make the > > > > ease of deployment balance out lesser speed. > > > > > > > > What's the setup here? Will you be writing to an index on local > > > > disk of server A and reading it, over NFS, from server B (and C > > > > and ...) or what? > > > > > > > > -- > > > > Ian. > > > > > > > > > > > > On Tue, Oct 2, 2012 at 1:45 PM, Paul Libbrecht <p...@hoplahup.net> > > > wrote: > > > > > I doubt NFS is an unreliable file-system. > > > > > Lucene uses normal random access to files and this has no reason > > > > > to > > be > > > > unreliable unless bad things such as network drops happen (in > > > > which > > case > > > > you'd get direct failures or timeouts rather than corruption). > > > > I've > > seen > > > > fairly large infrastructures being based on NFS and corruption is > > > something > > > > I've never heard about. > > > > > > > > > > Note: no concurrent access to a lucene index, right? > > > > > > > > > > Paul > > > > > > > > > > > > > > > Le 2 oct. 2012 à 14:01, Jong Kim a écrit : > > > > > > > > > >> Thank you all for reply. > > > > >> > > > > >> So it soudns like it is a known fact that the performance would > > suffer > > > > >> rather significantly when the index files are accessed over > > > > >> NFS. But > > > how > > > > >> about reliability and robustness (which seems even more important)? > > > > Isn't > > > > >> there any increased possibility for intermittent errors such as > > index > > > > file > > > > >> corruption (due to cache inconsistency, difference in delete > > > semantics, > > > > >> etc.) when using NFS? Has anyone run into such trouble? Or is > > > > >> it > > > > strictly > > > > >> just a performance issue? > > > > >> > > > > >> /Jong > > > > >> On Tue, Oct 2, 2012 at 5:17 AM, Paul Libbrecht > > > > >> <p...@hoplahup.net> > > > > wrote: > > > > >> > > > > >>> My experience in the Lucene 1.x times were a factor of at > > > > >>> least > > four > > > in > > > > >>> writing to NFS and about two when reading from there. I'd > > discourage > > > > this > > > > >>> as much as possible! > > > > >>> > > > > >>> (rsync is way more your friend for transporting and > > > > >>> replication à > > la > > > > solr > > > > >>> should also be considered) > > > > >>> > > > > >>> paul > > > > >>> > > > > >>> > > > > >>> Le 2 oct. 2012 à 11:10, Ian Lea a écrit : > > > > >>> > > > > >>>> You'll certainly need to factor in the performance of NFS > > > > >>>> versus > > > local > > > > >>> disks. > > > > >>>> > > > > >>>> My experience is that smallish low activity indexes work just > > > > >>>> fine > > > on > > > > >>>> NFS, but large high activity indexes are not so good, > > > > >>>> particularly > > > if > > > > >>>> you have a lot of modifications to the index. > > > > >>>> > > > > >>>> You may want to install a custom IndexDeletionPolicy. See > > > > >>>> the javadocs for details with specific reference to NFS. > > > > >>>> > > > > >>>> > > > > >>>> -- > > > > >>>> Ian. > > > > >>>> > > > > >>>> On Tue, Oct 2, 2012 at 3:21 AM, Vitaly Funstein < > > > vfunst...@gmail.com> > > > > >>> wrote: > > > > >>>>> How tolerant is your project of decreased search and > > > > >>>>> indexing > > > > >>> performance? > > > > >>>>> You could probably write a simple test that compares search > > > > >>>>> and > > > write > > > > >>>>> speeds of local and NFS-mounted indexes and make the > > > > >>>>> decision > > based > > > > on > > > > >>> the > > > > >>>>> results. > > > > >>>>> > > > > >>>>> On Mon, Oct 1, 2012 at 3:06 PM, Jong Kim > > > > >>>>> <jong.luc...@gmail.com> > > > > wrote: > > > > >>>>> > > > > >>>>>> Hi, > > > > >>>>>> > > > > >>>>>> According to the Lucene In Action (Second Edition), the > > > > >>>>>> section > > > > 2.11.2 > > > > >>>>>> "Accessing an index over a remote file system" explains > > > > >>>>>> that > > there > > > > are > > > > >>>>>> issues related to accessing a Lucene index across remote > > > > >>>>>> file > > > system > > > > >>>>>> including NFS. > > > > >>>>>> > > > > >>>>>> I'm particuarly interested in NFS compatibility, and > > > > >>>>>> wondering > > if > > > > >>> there has > > > > >>>>>> been any work done to solve or mitigate this problem. Has > > > > >>>>>> this > > > issue > > > > >>> been > > > > >>>>>> addressed? If not, are there some reliable work-arounds > > > > >>>>>> that > > make > > > > this > > > > >>>>>> possible at the expense of some sacrifice in other areas? > > > > >>>>>> > > > > >>>>>> Any information would be greatly appreciated, since my > > > > >>>>>> project > > > > heavily > > > > >>>>>> depends on the feasibility of this. > > > > >>>>>> > > > > >>>>>> Thanks > > > > >>>>>> /Jong > > > > >>>>>> > > > > >>>> > > > > >>>> > > > -------------------------------------------------------------------- > > > - > > > > >>>> To unsubscribe, e-mail: > > > > >>>> java-user-unsubscr...@lucene.apache.org > > > > >>>> For additional commands, e-mail: > > > > >>>> java-user-h...@lucene.apache.org > > > > >>>> > > > > >>> > > > > >>> > > > > >>> > > --------------------------------------------------------------------- > > > > >>> To unsubscribe, e-mail: > > > > >>> java-user-unsubscr...@lucene.apache.org > > > > >>> For additional commands, e-mail: > > > > >>> java-user-h...@lucene.apache.org > > > > >>> > > > > >>> > > > > > > > > > > > > > > > ---------------------------------------------------------------- > > > > > ----- To unsubscribe, e-mail: > > > > > java-user-unsubscr...@lucene.apache.org > > > > > For additional commands, e-mail: > > > > > java-user-h...@lucene.apache.org > > > > > > > > > > > > > ------------------------------------------------------------------ > > > > --- To unsubscribe, e-mail: > > > > java-user-unsubscr...@lucene.apache.org > > > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
