Curious... what kind of search performance are you getting for an index this size...
-----Original Message----- From: Omar Didi [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 30, 2005 3:15 PM To: java-user@lucene.apache.org Subject: RE: searcher question my java heap is between 128 and 1024 MB, I have 2GB of RAM and about 10 million documents in the index which is broken down to 6 indexes. I am using a multi-searcher to query the index. I am using lucene1.4.3. -----Original Message----- From: Doug Cutting [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 30, 2005 1:59 PM To: java-user@lucene.apache.org Subject: Re: searcher question Omar Didi wrote: > I am having a large index (100GB) and when i run the following code : > > String indexLocation = servlet.getServletContext().getInitParameter( "com.lucene.index" ); > logger.log( Level.INFO, "got the index location from: " + indexLocation ); > searcher = new IndexSearcher(indexLocation); > logger.log( Level.INFO, "we created an instance of SearchIndex" ); > > I never get to see the last message "we created an instance of SearchIndex" and I get > java.lang.OutOfMemoryError: Java heap space. How big is your java heap? How much RAM do you have on the machine? How many documents are in the index? What version of Lucene? You might try calling IndexWriter.setTermIndexInterval(512) and re-optimizing your index. You might need to add and/or delete a document for this to have an effect if the index is already optimized. This method is only in the latest sources, available from subversion. It should dramatically reduce the amount of memory required to open the index. There are other changes in the latest sources that will also reduce memory requirements, so you may not even need to use IndexWriter.setTermIndexInterval(). Doug --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
