Can anyone put some light on this? On Tue, Jun 12, 2018 at 12:32 AM, Ratnadeep Rakshit <ratnad...@qedrix.com> wrote:
> Here's the stack trace : > > 538 ERROR - 2018-06-07 09:07:36.030; [ x:addressbook] > org.apache.solr.common.SolrException; null:java.lang.RuntimeException: > java.lang.OutOfMemory > > Error: Java heap space > > 539 at org.apache.solr.servlet.HttpSolrCall.sendError( > HttpSolrCall.java:607) > > 540 at org.apache.solr.servlet.HttpSolrCall.call( > HttpSolrCall.java:475) > > 541 at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:257) > > 542 at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:208) > > 543 at org.eclipse.jetty.servlet.ServletHandler$CachedChain. > doFilter(ServletHandler.java:1652) > > 544 at org.eclipse.jetty.servlet.ServletHandler.doHandle( > ServletHandler.java:585) > > 545 at org.eclipse.jetty.server.handler.ScopedHandler.handle( > ScopedHandler.java:143) > > 546 at org.eclipse.jetty.security.SecurityHandler.handle( > SecurityHandler.java:577) > > 547 at org.eclipse.jetty.server.session.SessionHandler. > doHandle(SessionHandler.java:223) > > 548 at org.eclipse.jetty.server.handler.ContextHandler. > doHandle(ContextHandler.java:1127) > > 549 at org.eclipse.jetty.servlet.ServletHandler.doScope( > ServletHandler.java:515) > > 550 at org.eclipse.jetty.server.session.SessionHandler. > doScope(SessionHandler.java:185) > > 551 at org.eclipse.jetty.server.handler.ContextHandler. > doScope(ContextHandler.java:1061) > > 552 at org.eclipse.jetty.server.handler.ScopedHandler.handle( > ScopedHandler.java:141) > > 553 at org.eclipse.jetty.server.handler. > ContextHandlerCollection.handle(ContextHandlerCollection.java:215) > > 554 at org.eclipse.jetty.server.handler.HandlerCollection. > handle(HandlerCollection.java:110) > > 555 at org.eclipse.jetty.server.handler.HandlerWrapper.handle( > HandlerWrapper.java:97) > > 556 at org.eclipse.jetty.server.Server.handle(Server.java:499) > > 557 at org.eclipse.jetty.server.HttpChannel.handle( > HttpChannel.java:310) > > 558 at org.eclipse.jetty.server.HttpConnection.onFillable( > HttpConnection.java:257) > > 559 at org.eclipse.jetty.io.AbstractConnection$2.run( > AbstractConnection.java:540) > > 560 at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( > QueuedThreadPool.java:635) > > 561 at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run( > QueuedThreadPool.java:555) > > 562 at java.lang.Thread.run(Thread.java:745) > > 563 Caused by: java.lang.OutOfMemoryError: Java heap space > > 564 at org.apache.lucene.util.packed. > Packed64.<init>(Packed64.java:73) > > 565 at org.apache.lucene.util.packed.PackedInts.getMutable( > PackedInts.java:1009) > > 566 at org.apache.lucene.util.packed.PackedInts.getMutable( > PackedInts.java:976) > > 567 at org.apache.lucene.util.packed. > GrowableWriter.ensureCapacity(GrowableWriter.java:80) > > 568 at org.apache.lucene.util.packed.GrowableWriter.set( > GrowableWriter.java:88) > > 569 at org.apache.lucene.util.packed.AbstractPagedMutable.set( > AbstractPagedMutable.java:101) > > 570 at org.apache.lucene.util.fst. > NodeHash.addNew(NodeHash.java:152) > > 571 at org.apache.lucene.util.fst. > NodeHash.rehash(NodeHash.java:169) > > 572 at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java: > 133) > > 573 at org.apache.lucene.util.fst.Builder.compileNode(Builder. > java:215) > > 574 at org.apache.lucene.util.fst.Builder.freezeTail(Builder. > java:310) > > 575 at org.apache.lucene.util.fst. > Builder.add(Builder.java:417) > > 576 at org.apache.lucene.search.suggest.analyzing. > AnalyzingSuggester.build(AnalyzingSuggester.java:565) > > 577 at org.apache.lucene.search.suggest.Lookup.build(Lookup. > java:193) > > 578 at org.apache.solr.spelling.suggest.SolrSuggester.build( > SolrSuggester.java:176) > > 576 at org.apache.lucene.search.suggest.analyzing. > AnalyzingSuggester.build(AnalyzingSuggester.java:565) > > 577 at org.apache.lucene.search.suggest.Lookup.build(Lookup. > java:193) > > 578 at org.apache.solr.spelling.suggest.SolrSuggester.build( > SolrSuggester.java:176) > > 579 at org.apache.solr.handler.component.SuggestComponent. > prepare(SuggestComponent.java:179) > > 580 at org.apache.solr.handler.component.SearchHandler. > handleRequestBody(SearchHandler.java:246) > > 581 at org.apache.solr.handler.RequestHandlerBase. > handleRequest(RequestHandlerBase.java:155) > > 582 at org.apache.solr.core.SolrCore. > execute(SolrCore.java:2102) > > 583 at org.apache.solr.servlet.HttpSolrCall.execute( > HttpSolrCall.java:654) > > 584 at org.apache.solr.servlet.HttpSolrCall.call( > HttpSolrCall.java:460) > > 585 at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:257) > > 586 at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:208) > > 587 at org.eclipse.jetty.servlet.ServletHandler$CachedChain. > doFilter(ServletHandler.java:1652) > > 588 at org.eclipse.jetty.servlet.ServletHandler.doHandle( > ServletHandler.java:585) > > 589 at org.eclipse.jetty.server.handler.ScopedHandler.handle( > ScopedHandler.java:143) > > 590 at org.eclipse.jetty.security.SecurityHandler.handle( > SecurityHandler.java:577) > > 591 at org.eclipse.jetty.server.session.SessionHandler. > doHandle(SessionHandler.java:223) > > 592 at org.eclipse.jetty.server.handler.ContextHandler. > doHandle(ContextHandler.java:1127) > > 593 at org.eclipse.jetty.servlet.ServletHandler.doScope( > ServletHandler.java:515) > > 594 at org.eclipse.jetty.server.session.SessionHandler. > doScope(SessionHandler.java:185) > > 595 at org.eclipse.jetty.server.handler.ContextHandler. > doScope(ContextHandler.java:1061) > > 596 > > 597 WARN - 2018-06-07 09:07:36.053; [ x:addressbook] > org.eclipse.jetty.servlet.ServletHandler; Error for > /solr/addressbook/suggest > > 598 java.lang.OutOfMemoryError: Java heap space > > 599 at org.apache.lucene.util.packed. > Packed64.<init>(Packed64.java:73) > > 600 at org.apache.lucene.util.packed.PackedInts.getMutable( > PackedInts.java:1009) > > 601 at org.apache.lucene.util.packed.PackedInts.getMutable( > PackedInts.java:976) > > 602 at org.apache.lucene.util.packed. > GrowableWriter.ensureCapacity(GrowableWriter.java:80) > > 603 at org.apache.lucene.util.packed.GrowableWriter.set( > GrowableWriter.java:88) > > 604 at org.apache.lucene.util.packed.AbstractPagedMutable.set( > AbstractPagedMutable.java:101) > > 605 at org.apache.lucene.util.fst. > NodeHash.addNew(NodeHash.java:152) > > 606 at org.apache.lucene.util.fst. > NodeHash.rehash(NodeHash.java:169) > > 607 at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java: > 133) > > 608 at org.apache.lucene.util.fst.Builder.compileNode(Builder. > java:215) > > 609 at org.apache.lucene.util.fst.Builder.freezeTail(Builder. > java:310) > > 610 at org.apache.lucene.util.fst. > Builder.add(Builder.java:417) > > 611 at org.apache.lucene.search.suggest.analyzing. > AnalyzingSuggester.build(AnalyzingSuggester.java:565) > > 612 at org.apache.lucene.search.suggest.Lookup.build(Lookup. > java:193) > > 613 at org.apache.solr.spelling.suggest.SolrSuggester.build( > SolrSuggester.java:176) > > 614 at org.apache.solr.handler.component.SuggestComponent. > prepare(SuggestComponent.java:179) > > 615 at org.apache.solr.handler.component.SearchHandler. > handleRequestBody(SearchHandler.java:246) > > 616 at org.apache.solr.handler.RequestHandlerBase. > handleRequest(RequestHandlerBase.java:155) > > 617 at org.apache.solr.core.SolrCore. > execute(SolrCore.java:2102) > > 618 at org.apache.solr.servlet.HttpSolrCall.execute( > HttpSolrCall.java:654) > > 619 at org.apache.solr.servlet.HttpSolrCall.call( > HttpSolrCall.java:460) > > 620 at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:257) > > 621 at org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:208) > > 622 at org.eclipse.jetty.servlet.ServletHandler$CachedChain. > doFilter(ServletHandler.java:1652) > > 623 at org.eclipse.jetty.servlet.ServletHandler.doHandle( > ServletHandler.java:585) > > 624 at org.eclipse.jetty.server.handler.ScopedHandler.handle( > ScopedHandler.java:143) > > 625 at org.eclipse.jetty.security.SecurityHandler.handle( > SecurityHandler.java:577) > > 626 at org.eclipse.jetty.server.session.SessionHandler. > doHandle(SessionHandler.java:223) > > 627 at org.eclipse.jetty.server.handler.ContextHandler. > doHandle(ContextHandler.java:1127) > > 628 at org.eclipse.jetty.servlet.ServletHandler.doScope( > ServletHandler.java:515) > > 629 at org.eclipse.jetty.server.session.SessionHandler. > doScope(SessionHandler.java:185) > > 630 at org.eclipse.jetty.server.handler.ContextHandler. > doScope(ContextHandler.java:1061) > > > > > On Mon, Jun 11, 2018 at 11:34 PM, Christopher Schultz < > ch...@christopherschultz.net> wrote: > >> Ratnadeep, >> >> On 6/11/18 12:25 PM, Ratnadeep Rakshit wrote: >> > I am using the Solr Suggester component in Solr 5.5 with a lot of >> address >> > data. My Machine has allotted 20Gb RAM for solr and the machine has 32GB >> > RAM in total. >> > >> > I have an address book core with the following vitals - >> > >> > "numDocs"=153242074 >> > "segmentCount"=34 >> > "size"=30.29 GB >> > >> > My solrconfig.xml looks something like this - >> > >> > <searchComponent name="suggest" class="solr.SuggestComponent"> >> > <lst name="suggester"> >> > <str name="name">mySuggester1</str> >> > <str name="lookupImpl">FuzzyLookupFactory</str> >> > <str name="storeDir">suggester_fuzzy_dir</str> >> > >> > <!-- Substitute these for the two above for another "flavor" >> > <str name="lookupImpl">AnalyzingInfixLookupFactory</str> >> > <str name=?indexPath?>suggester_infix_dir</str> >> > --> >> > >> > <str name="dictionaryImpl">DocumentDictionaryFactory</str> >> > <str name="field">site_address</str> >> > <str name="suggestAnalyzerFieldType">suggestType</str> >> > <str name="payloadField">property_metadata</str> >> > <str name="buildOnStartup">false</str> >> > <str name="buildOnCommit">false</str> >> > </lst> >> > <lst name="suggester"> >> > <str name="name">mySuggester2</str> >> > <str name="lookupImpl">AnalyzingInfixLookupFactory</str> >> > <str name="indexPath">suggester_infix_dir</str> >> > >> > <str name="dictionaryImpl">DocumentDictionaryFactory</str> >> > <str name="field">site_address_other</str> >> > <str name="suggestAnalyzerFieldType">suggestType</str> >> > <str name="payloadField">property_metadata</str> >> > <str name="buildOnStartup">false</str> >> > <str name="buildOnCommit">false</str> >> > </lst> >> > </searchComponent> >> > >> > The handler is defined like so - >> > >> > <requestHandler name="/suggest" class="solr.SearchHandler" >> startup="lazy" > >> > <lst name="defaults"> >> > <str name="suggest">true</str> >> > <str name="suggest.count">10</str> >> > <str name="suggest.dictionary">mySuggester1</str> >> > <str name="suggest.dictionary">mySuggester2</str> >> > <str name="suggest.collate">false</str> >> > <str name="echoParams">explicit</str> >> > </lst> >> > <arr name="components"> >> > <str>suggest</str> >> > </arr> >> > </requestHandler> >> > >> > *Problem Statement* >> > >> > Every time I try to build the suggest index using the suggest.build=true >> > url parameter, I end up with an OutOfMemory error. I have no clue how I >> can >> > make this work with the current setup. Can anyone explain why this is >> > happening? And how can I fix this issue? >> > *StackOverflow:* >> > https://stackoverflow.com/questions/50802122/solr-suggest- >> component-and-outofmemory-error >> > >> >> Can you explain the nature of the OOM? Not all OOMs are due to heap >> exhaustion... >> >> -chris >> >> >> >
