That’s the upper limit of a filter cache entry (maxDoc/8). For low numbers of hits, more space-efficient structures are used. Specifically a list of doc IDs is kept. So say you have an fq clause that marks 10 doc. The filterCache entry is closer to 40 bytes + sizeof(query object) etc.
Still, it’s what you have to be prepared for. filterCache is local to the core. So if you have 8 replicas they’d each have 128M docs or so and each filterCache entry would be bounded by about 128M/8 Checking the filterCache via the admin UI is a way to find current usage, but be sure it’s full. The memory is allocated as needed, not up front. All that said, you’re certainly right, the filterCache can certainly lead to OOMs. What I try to emphasize to people is that they cannot allocate huge filterCaches without considering memory implications... Best, Erick > On Feb 17, 2020, at 4:51 AM, Vadim Ivanov <vadim.iva...@spb.ntk-intourist.ru> > wrote: > > You can easily check amount of RAM used by core filterCache in Admin UI: > Choose core - Plugins/Stats - Cache - filterCache > It shows useful information on configuration, statistics and current RAM > usage by filter cache, > as well as some examples of current filtercaches in RAM > Core, for ex, with 10 mln docs uses 1.3 MB of Ram for every filterCache > > >> -----Original Message----- >> From: Hongxu Ma [mailto:inte...@outlook.com] >> Sent: Monday, February 17, 2020 12:13 PM >> To: solr-user@lucene.apache.org >> Subject: A question about solr filter cache >> >> Hi >> I want to know the internal of solr filter cache, especially its memory > usage. >> >> I googled some pages: >> https://teaspoon-consulting.com/articles/solr-cache-tuning.html >> https://lucene.472066.n3.nabble.com/Solr-Filter-Cache-Size-td4120912.html >> (Erick Erickson's answer) >> >> All of them said its structure is: fq => a bitmap (total doc number bits), > but I >> think it's not so simple, reason: >> Given total doc number is 1 billion, each filter cache entry will use > nearly >> 1GB(1000000000/8 bit), it's too big and very easy to make solr OOM (I have > a >> 1 billion doc cluster, looks it works well) >> >> And I also checked solr node, but cannot find the details (only saw using >> DocSets structure) >> >> So far, I guess: >> >> * degenerate into an doc id array/list when the bitmap is sparse >> * using some compressed bitmap, e.g. roaring bitmaps >> >> which one is correct? or another answer, thanks you very much! > >
