Thanks Fuad. But why does just sorting provide an OOM. I executed the query without adding the sort clause it executed perfectly. In fact I even tried remove the maxrows=10 and executed. it came out fine. Queries with bigger results seems to come out fine too. But why just sort of that too just 10 rows?? -Sundar
> Date: Tue, 22 Jul 2008 12:24:35 -0700> From: [EMAIL PROTECTED]> To: > solr-user@lucene.apache.org> Subject: RE: Out of memory on Solr sorting> > > org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:403)> > > - this piece of code do not request Array[100M] (as I seen with > Lucene), > it asks only few bytes / Kb for a field...> > > Probably 128 - 512 is not > enough; it is also advisable to use equal sizes> -Xms1024M -Xmx1024M> (it > minimizes GC frequency, and itensures that 1024M is available at startup)> > > OOM happens also with fragmented memory, when application requests big > > contigues fragment and GC is unable to optimize; looks like your > > application requests a little and memory is not available...> > > Quoting > sundar shankar <[EMAIL PROTECTED]>:> > >> >> >> >> From: [EMAIL PROTECTED]> > >> To: solr-user@lucene.apache.org> >> Subject: Out of memory on Solr > sorting> >> Date: Tue, 22 Jul 2008 19:11:02 +0000> >>> >>> >> Hi,> >> Sorry > again fellos. I am not sure whats happening. The day with > >> solr is bad > for me I guess. EZMLM didnt let me send any mails this > >> morning. Asked me > to confirm subscription and when I did, it said I > >> was already a member. > Now my mails are all coming out bad. Sorry > >> for troubling y'all this bad. > I hope this mail comes out right.> >> >> > Hi,> > We are developing a product > in a agile manner and the current > > implementation has a data of size just > about a 800 megs in dev.> > The memory allocated to solr on dev (Dual core > Linux box) is 128-512.> >> > My config> > =========> >> > <!-- autocommit > pending docs if certain criteria are met> > <autoCommit>> > > <maxDocs>10000</maxDocs>> > <maxTime>1000</maxTime>> > </autoCommit>> > -->> > >> > <filterCache> > class="solr.LRUCache"> > size="512"> > > initialSize="512"> > autowarmCount="256"/>> >> > <queryResultCache> > > class="solr.LRUCache"> > size="512"> > initialSize="512"> > > autowarmCount="256"/>> >> > <documentCache> > class="solr.LRUCache"> > > size="512"> > initialSize="512"> > autowarmCount="0"/>> >> > > <enableLazyFieldLoading>true</enableLazyFieldLoading>> >> >> > My Field> > > =======> >> > <fieldType name="autocomplete" class="solr.TextField">> > > <analyzer type="index">> > <tokenizer class="solr.KeywordTokenizerFactory"/>> > > <filter class="solr.LowerCaseFilterFactory" />> > <filter > class="solr.PatternReplaceFilterFactory" > > pattern="([^a-z0-9])" > replacement="" replace="all" />> > <filter > class="solr.EdgeNGramFilterFactory" > > maxGramSize="100" minGramSize="1" />> > > </analyzer>> > <analyzer type="query">> > <tokenizer > class="solr.KeywordTokenizerFactory"/>> > <filter > class="solr.LowerCaseFilterFactory" />> > <filter > class="solr.PatternReplaceFilterFactory" > > pattern="([^a-z0-9])" > replacement="" replace="all" />> > <filter > class="solr.PatternReplaceFilterFactory" > > pattern="^(.{20})(.*)?" > replacement="$1" replace="all" />> > </analyzer>> > </fieldType>> >> >> > > Problem> > ======> >> > I execute a query that returns 24 rows of result. I > pick 10 out of > > it. I have no problem when I execute this.> > But When I > do sort it by a String field that is fetched from this > > result. I get an > OOM. I am able to execute several> > other queries with no problem. Just > having a sort asc clause added > > to the query throws an OOM. Why is that.> > > What should I have ideally done. My config on QA is pretty similar > > to > the dev box and probably has more data than on dev.> > It didnt throw any OOM > during the integration test. The Autocomplete > > is a new field we added > recently.> >> > Another point is that the indexing is done with a field of > type string> > <field name="XXX" type="string" indexed="true" stored="true" > > > termVectors="true"/>> >> > and the autocomplete field is a copy field.> >> > > The sorting is done based on string field.> >> > Please do lemme know what > mistake am I doing?> >> > Regards> > Sundar> >> > P.S: The stack trace of the > exception is> >> >> > Caused by: > org.apache.solr.client.solrj.SolrServerException: Error > > executing query> > > at > > > org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:86)> > > at > > > org.apache.solr.client.solrj.impl.BaseSolrServer.query(BaseSolrServer.java:101)> > > at > > > com.apollo.sisaw.solr.service.AbstractSolrSearchService.makeSolrQuery(AbstractSolrSearchService.java:193)> > > ... 105 more> > Caused by: org.apache.solr.common.SolrException: Java heap > space > > java.lang.OutOfMemoryError: Java heap space> > at > > > org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:403)> > > at > org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)> > > at > > > org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:352)> > > at > > > org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSortedHitQueue.java:416)> > > at > > > org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:207)> > > at > org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)> > > at > > > org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:168)> > > at > > > org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.java:56)> > > at > > > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:907)> > > at > > > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)> > > at > > > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)> > > at > > > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)> > > at > > > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:156)> > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:128)> > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1025)> > at > > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)> > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272)> > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)> > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)> > > at > > > org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)> > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)> > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)> > > at > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)> > > at > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)> > > at > > > org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:175)> > > at > > > org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:74)> > > at > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)> > > at > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)> > > at > > > org.jboss.web.tomcat.tc5.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:156)> > > at > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)> > > at > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)> > > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)> > >> > _________________________________________________________________> > > Wish to Marry Now? Click Here to Register FREE> > > http://www.shaadi.com/registration/user/index.php?ptnr=mhottag> > > _________________________________________________________________ Missed your favourite programme? Stop surfing TV channels and start planning your weekend TV viewing with our comprehensive TV Listing http://entertainment.in.msn.com/TV/TVListing.aspx