What is the size of heap u r allocating for your app ? -----Original Message----- From: Harini Raghavan [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 01, 2007 2:29 PM To: java-user@lucene.apache.org Subject: Searching with too many clauses + Out of Memory
Hi Everyone, I am using Compass 1.1 M2 which supports Lucene 2.2 to store & search huge amount of company, executive and employment data. There are some usecases where I need to search for executives/employments on the result set of company search. But when I try to create a compass query to search for executives for over 1 lac company ids, it runs out of memory as the query is huge. Here is the exception stack trace: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:342) at org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:435) at org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:428) at org.apache.lucene.index.MultiTermDocs.read(MultiReader.java:393) at org.apache.lucene.search.TermScorer.next(TermScorer.java:106) at org.apache.lucene.util.ScorerDocQueue.topNextAndAdjustElsePop( ScorerDocQueue.ja va:116) at org.apache.lucene.search.DisjunctionSumScorer.advanceAfterCurrent(Disjunctio nSu mScorer.java:175) at org.apache.lucene.search.DisjunctionSumScorer.next( DisjunctionSumScorer.java:14 6) at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:327) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:124) at org.apache.lucene.search.MultiSearcher.search(MultiSearcher.java:232) at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:74) at org.apache.lucene.search.Hits.<init>(Hits.java:61) at org.apache.lucene.search.Searcher.search(Searcher.java:55) at org.compass.core.lucene.engine.transaction.ReadCommittedTransaction.findByQu ery( ReadCommittedTransaction.java:469) at org.compass.core.lucene.engine.transaction.ReadCommittedTransaction.doFind(R ead CommittedTransaction.java:426) at org.compass.core.lucene.engine.transaction.AbstractTransaction.find(Abstract Tra nsaction.java:91) at org.compass.core.lucene.engine.LuceneSearchEngine.find( LuceneSearchEngine.java: 379) at org.compass.core.lucene.engine.LuceneSearchEngineQuery.hits(LuceneSearchEngi neQ uery.java:151) at org.compass.core.impl.DefaultCompassQuery.hits(DefaultCompassQuery.java:133) at org.compass.core.support.search.CompassSearchHelper.performSearch(CompassSea rch Helper.java:144) at org.compass.core.support.search.CompassSearchHelper$1.doInCompass(CompassSea rch Helper.java:89) at org.compass.core.CompassTemplate.execute(CompassTemplate.java:137) at org.compass.core.support.search.CompassSearchHelper.search(CompassSearchHelp er. java:86) It looks like this error is actually in the lucene code. It would be great if anyone in this group has an idea about this kind of usecase and has some suggestions. Thanks, Harini --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]