Hi, We had the problem again a few days ago.
I have noticed that each time the problem occurs the old generation of the heap suddenly grows. Its size is generally between 0,5 et 1,5Gb on 3Gg limit. In 4 minutes the old generation grows to 3Gb and never goes down as consecutive GC reclaims 0 bytes. I have a heap dump for a previous problem. Near 1.7 Gb is of memory is an array of 6 BoostQuery objects including themself a huge array of PhraseQuery$PhraseWeigth I also noticed that a few queries (nearly 10 per minute) use the parameter " debug=track&debug=timing". I have asked the customer not enable debug in production. Can query debug cause so high memory usage ? Regards. Dominique Le lun. 18 mai 2020 à 09:42, Dominique Bejean <dominique.bej...@eolya.fr> a écrit : > Hi Shawn, > > In fact, I was using logs from a core at WARN log level so with only slow > queries (>500ms). > > I just checked in a core at INFO log level with all queries (we set the > log level top INFO for one core after the previous crash) and there is no > more queries with these two facets when the problem starts. There are > nearly 150 queries per minute faceting with the 750K unique terms fields > during the 3 hours before the problem occurs and no increase during the few > minutes before and when the problem starts. > > I can't see anything specific in logs at the time the problem start. > > Regards > > Dominique > > > > > Le lun. 18 mai 2020 à 03:28, Shawn Heisey <apa...@elyograg.org> a écrit : > >> On 5/17/2020 4:18 PM, Dominique Bejean wrote: >> > I was not thinking that queries using facet with fields with high number >> > of unique value but with low hits count can be the origin of this >> problem. >> >> Performance for most things does not depend on numFound (hit count) or >> the rows parameter. The number of terms in the field and the total >> number of documents in the index matters a lot more. >> >> If you do facets or grouping on a field with 750K unique terms, it's >> going to be very slow and require a LOT of memory. I would not be >> surprised to see it require more than 4GB. These features are designed >> to work best with fields that have a relatively small number of possible >> values. >> >> Thanks, >> Shawn >> >
