Agree, Pushkar. I had docValues for sorting / faceting fields from begining (since I setup Solr 6.0). So good on that side. I am going to analyze the queries to find any potential issue. Two questions which I am puzzling with
a) Should the below JVM parameter be included for Prod to get heap dump "-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/to/the/dump" b) Currently OOM script just kills the Solr instance. Shouldn't it be enhanced to wait and restart Solr instance Thanks, Susheel On Tue, Oct 25, 2016 at 7:35 PM, Pushkar Raste <pushkar.ra...@gmail.com> wrote: > You should look into using docValues. docValues are stored off heap and > hence you would be better off than just bumping up the heap. > > Don't enable docValues on existing fields unless you plan to reindex data > from scratch. > > On Oct 25, 2016 3:04 PM, "Susheel Kumar" <susheel2...@gmail.com> wrote: > > > Thanks, Toke. Analyzing GC logs helped to determine that it was a sudden > > death. The peaks in last 20 mins... See http://tinypic.com/r/n2zonb/9 > > > > Will look into the queries more closer and also adjusting the cache > sizing. > > > > > > Thanks, > > Susheel > > > > On Tue, Oct 25, 2016 at 3:37 AM, Toke Eskildsen <t...@statsbiblioteket.dk> > > wrote: > > > > > On Mon, 2016-10-24 at 18:27 -0400, Susheel Kumar wrote: > > > > I am seeing OOM script killed solr (solr 6.0.0) on couple of our VM's > > > > today. So far our solr cluster has been running fine but suddenly > > > > today many of the VM's Solr instance got killed. > > > > > > As you have the GC-logs, you should be able to determine if it was a > > > slow death (e.g. caches gradually being filled) or a sudden one (e.g. > > > grouping or faceting on a large new non-DocValued field). > > > > > > Try plotting the GC logs with time on the x-axis and free memory after > > > GC on the y-axis. It it happens to be a sudden death, the last lines in > > > solr.log might hold a clue after all. > > > > > > - Toke Eskildsen, State and University Library, Denmark > > > > > >
