Hi Everyone, I am using Compass 1.1 M2 which supports Lucene 2.2 to store & search huge amount of company, executive and employment data. There are some usecases where I need to search for executives/employments on the result set of company search. But when I try to create a compass query to search for executives for over 1 lac company ids, it runs out of memory as the query is huge. Here is the exception stack trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:342) at org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:435) at org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:428) at org.apache.lucene.index.MultiTermDocs.read(MultiReader.java:393) at org.apache.lucene.search.TermScorer.next(TermScorer.java:106) at org.apache.lucene.util.ScorerDocQueue.topNextAndAdjustElsePop( ScorerDocQueue.ja va:116) at org.apache.lucene.search.DisjunctionSumScorer.advanceAfterCurrent(DisjunctionSu mScorer.java:175) at org.apache.lucene.search.DisjunctionSumScorer.next( DisjunctionSumScorer.java:14 6) at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:327) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:124) at org.apache.lucene.search.MultiSearcher.search(MultiSearcher.java:232) at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:74) at org.apache.lucene.search.Hits.<init>(Hits.java:61) at org.apache.lucene.search.Searcher.search(Searcher.java:55) at org.compass.core.lucene.engine.transaction.ReadCommittedTransaction.findByQuery( ReadCommittedTransaction.java:469) at org.compass.core.lucene.engine.transaction.ReadCommittedTransaction.doFind(Read CommittedTransaction.java:426) at org.compass.core.lucene.engine.transaction.AbstractTransaction.find(AbstractTra nsaction.java:91) at org.compass.core.lucene.engine.LuceneSearchEngine.find( LuceneSearchEngine.java: 379) at org.compass.core.lucene.engine.LuceneSearchEngineQuery.hits(LuceneSearchEngineQ uery.java:151) at org.compass.core.impl.DefaultCompassQuery.hits(DefaultCompassQuery.java:133) at org.compass.core.support.search.CompassSearchHelper.performSearch(CompassSearch Helper.java:144) at org.compass.core.support.search.CompassSearchHelper$1.doInCompass(CompassSearch Helper.java:89) at org.compass.core.CompassTemplate.execute(CompassTemplate.java:137) at org.compass.core.support.search.CompassSearchHelper.search(CompassSearchHelper. java:86) It looks like this error is actually in the lucene code. It would be great if anyone in this group has an idea about this kind of usecase and has some suggestions. Thanks, Harini
