On Jan 17, 2007, at 1:16 PM, Michael McCandless wrote:
This is the solution I have in mind for LUCENE-710: change the IndexFileDeleter so that instead of always immediately deleting the last commit when a new commit happens, allow some time before doing so. This way readers have a chance to refresh. The actual time would be settable by the developer. So if you set it to 6 hours, then, a commit would remain usable for at least 6 hours after it had been obsoleted by a new commit. This means if you can ensure your readers refresh within 6 hours of a new commit happening, then the writer will never delete an "in-use" commit.
I've been mulling this over. If you set the interval to 6 hours, and there's a lot of churn (e.g. if you optimize frequently), you'll end up with a lot of wasted disk space. On the flip side, the user has to set up some sort of trigger for refreshing the IndexReaders anyway. It's still not user-friendly by default, and we'd be polluting the API with a hateful workaround.
The real problem is NFS. For background, see <http:// nfs.sourceforge.net/#section_d>, item D2, which deals with NFS and "delete on last close".
Now I wonder. Version 4 of the NFS protocol introduces state, so it's possible to implement file locking. Can we lock a segments file, then have IndexFileDeleter detect which segments are locked that way? And if that's the case, can we detect whether the locking mechanism is failing and throw an exception if someone tries to use an earlier version of NFS?
I'd be cool with making it impossible to put an index on an NFS volume prior to version 4. That puts the blame where it belongs.
Marvin Humphrey Rectangular Research http://www.rectangular.com/ --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
