Lucene 2.9.0 Near Real Time Indexing and Service Crashes/restarts

2010-01-12 Thread jchang
Lucene 2.9.0 has near real time indexing, writing to a RAMDir which gets flushed to disk when you do a search. Does anybody know how this works out with service restarts (both orderly shutdown and a crash)? If the service goes down while indexed items are in RAMDir but not on disk, are they lost

Lucene 2.9.0 Near Real Time Indexing and lock timeouts

2010-01-12 Thread jchang
Hello, I am using Lucene 2.4.0 and am getting org.apache.lucene.store.LockObtainFailedException's when I have a backed up queue of items to index (with multiple concurrent writers). Of course, if I throttle all my writer threads to 1, I don't get the exception, but I'm hoping to write faster th

Re: Lucene 2.9.0 Near Real Time Indexing and Service Crashes/restarts

2010-01-13 Thread jchang
iving on top > which can wrap them up in their own business logic as they saw fit... as > long as it was ok to have all the applications in the same JVM, of > course). > > -jake > > >> >> Otis >> -- >> Sematext -- http://sematext.com/ -- Solr - Lucene -

Re: Lucene 2.9.0 Near Real Time Indexing and Service Crashes/restarts

2010-01-13 Thread jchang
Actually, unless IW.commit is called, all changes after the last commit will be lost (because the segment infos file will not have been written). On Tue, Jan 12, 2010 at 3:37 PM, Jason Rutherglen wrote: > Greetin's John, > > 2.9 and 3.0 don't use a RAMDir... Deletes are held in RAM however so >

Re: Lucene 2.9.0 Near Real Time Indexing and lock timeouts

2010-01-14 Thread jchang
With only 10 concurrent consumers, I do get lock problems. However, I am calling commit() at the end of each addition. Could I expect better concurrency without timeouts if I did not commit as often? -- View this message in context: http://old.nabble.com/Lucene-2.9.0-Near-Real-Time-Indexing-a

NRT and IndexSearcher performance

2010-01-19 Thread jchang
The javadocs for IndexSearcher in Lucene 3.0.0 read: "For performance reasons it is recommended to open only one IndexSearcher and use it for all of your searches." However, to use NRT, it seems I have to do this for every search, which contradicts the advice above: IndexSearcher myIndexSear

FileNotFoundException for write.lock

2010-01-23 Thread jchang
When I try to start my service and construct an IndexWriter, I get this: java.io.FileNotFoundException: no segments* file found in org.apache.lucene.store.NIOFSDirectory@/home/jchang/IdeaProjects/index-service_trunk/target/testindexA/index/indexablemaildata: files: [write.lock] It is odd. The

Re: FileNotFoundException for write.lock

2010-01-23 Thread jchang
By the way: this happens with a brand new directory with no files at all in it. jchang wrote: > > When I try to start my service and construct an IndexWriter, I get this: > > java.io.FileNotFoundException: no segments* file found in > org.apache.lucene.store.NIOFSDirecto