On Tue, May 2, 2017 at 10:22 AM, Damien Kamerman <dami...@gmail.com> wrote: > Hi all, > > I have a Solr v6.4.2 collection with 12 shards and 2 replicas. Each replica > uses about 14GB disk usage. I'm using Solaris 11 and I see the 'Page cache' > grow by about 7GB for each suggester replica I build. The suggester index > itself is very small. The 'Page cache' memory is freed when the node is > stopped. > > I guess the Suggester component is mmap'ing the entire Lucene index into > memory and holding it? Is this expected behavior? Is there a workaround? >
Yes, this is expected. The suggester opens the index using Lucene's MMapDirectory which ends up memory mapping the index. But the memory mapped pages can be shared across everyone using the same index which basically means that the replica's usual index searcher can also use these pages. If you were not building the suggester index, even then the replica's search index would have been mmapped but perhaps only on demand instead of all at once. > I use this command to build the suggester for just the replica > 'target1_shard1_replica1': > curl " > http://localhost:8983/solr/collection1/suggest?suggest.dictionary=mySuggester&suggest.build=true&shards=localhost:8983/solr/target1_shard1_replica1 > " > > BTW: Without the 'shards' param the distributed request will randomly hit > half the replicas. Yes, this is a problem. I recently opened https://issues.apache.org/jira/browse/SOLR-10532 but unfortunately I don't have the time to fix it soon. Patches are always welcome. > > From my solrconfig.xml: > <searchComponent name="suggest" class="solr.SuggestComponent"> > <lst name="suggester"> > <str name="name">mySuggester</str> > <str name="lookupImpl">AnalyzingInfixLookupFactory</str> > <str name="indexPath">mySuggester</str> > <str name="dictionaryImpl">DocumentDictionaryFactory</str> > <str name="field">mySuggest</str> > <str name="contextField">x</str> > <str name="suggestAnalyzerFieldType">suggestTypeLc</str> > <str name="buildOnStartup">false</str> > </lst> > </searchComponent> > > Cheers, > Damien. -- Regards, Shalin Shekhar Mangar.
