Re: Unable to improve performance

2009-04-01 Thread Toke Eskildsen
On Fri, 2009-03-27 at 12:07 +0100, Paul Taylor wrote: [2Gb index, 7 million documents(?)] > I ran the test a number of times with 30 threads, and max memory of > 3500mb I was processing 10,000 records in about 43 seconds ( 233 > queries/second) , the index was stored on a solid state drive runn

Re: Unable to improve performance

2009-03-27 Thread Paul Taylor
Michael McCandless wrote: Are you opening your IndexReader with readOnly=true? If not, you're likely hitting contention on the "isDeleted" method. When you run with a "normal" directory, either on a traditional hard drive or SSD device, do you use NIOFSDirectory? That removes contention, but,

Re: Unable to improve performance

2009-03-27 Thread Simon Willnauer
ReadOnly option was introduce with 2.4 from javadoc: "...as of 2.4, it's possible to open a read-only IndexReader using one of the static open methods that accepts the boolean readOnly parameter." http://lucene.apache.org/java/2_4_1/api/core/org/apache/lucene/index/IndexReader.html#open(org.apache

Re: Unable to improve performance

2009-03-27 Thread Michael McCandless
Alas, it's new as of 2.4. Can you upgrade? Mike On Fri, Mar 27, 2009 at 11:55 AM, wrote: >> > How can I open it "readonly"? >> >> See the javadocs for IndexReader. > > I did it already for 2.3 - cannot find readonly > > > - >

RE: Unable to improve performance

2009-03-27 Thread spring
> > How can I open it "readonly"? > > See the javadocs for IndexReader. I did it already for 2.3 - cannot find readonly - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-use

Re: Unable to improve performance

2009-03-27 Thread Ian Lea
>> Are you opening your IndexReader with readOnly=true?  If not, you're >> likely hitting contention on the "isDeleted" method. > > How can I open it "readonly"? See the javadocs for IndexReader. -- Ian. - To unsubscribe, e-mail

RE: Unable to improve performance

2009-03-27 Thread spring
> Are you opening your IndexReader with readOnly=true? If not, you're > likely hitting contention on the "isDeleted" method. How can I open it "readonly"? - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For ad

Re: Unable to improve performance

2009-03-27 Thread Michael McCandless
Also, see here for other ideas that may help: http://wiki.apache.org/lucene-java/ImproveSearchingSpeed I just updated that page with readOnly IndexReader & NIOFSDirectory. Mike On Fri, Mar 27, 2009 at 7:07 AM, Paul Taylor wrote: > Hi > > I am trying to run the performance tests against luc

Re: Unable to improve performance

2009-03-27 Thread Michael McCandless
Are you opening your IndexReader with readOnly=true? If not, you're likely hitting contention on the "isDeleted" method. When you run with a "normal" directory, either on a traditional hard drive or SSD device, do you use NIOFSDirectory? That removes contention, but, it only works on non-Windows