Re: Loading data to FieldValueCache

bq: There will be no updates to my index. So, no worries about ageing out or garbage collection This is irrelevant to aging out filterCache entries, this is purely query time. bq: Each having 64 GB of RAM, out of which I am allocating 45 GB to Solr. It's usually a mistake to give Solr so much ra

Re: Loading data to FieldValueCache

On Fri, Dec 26, 2014 at 12:26 PM, Erick Erickson wrote: > I don't know the complete algorithm, but if the number of docs that > satisfy the fq is "small enough", > then just the internal Lucene doc IDs are stored rather than a bitset. If smaller than maxDoc/64 ids are collected, a sorted int set

Re: Loading data to FieldValueCache

Erick, I am trying to do a premature optimization. *There will be no updates to my index. So, no worries about ageing out or garbage collection.* Let me get my understanding correctly; when we talk about filterCache, it just stores the document IDs in the cache right? And my setup is as follows.

Re: Loading data to FieldValueCache

Manohar: Please approach this cautiously. You state that you have "hundreds of states". Every 100 states will use roughly 1.2G of your filter cache. Just for this field. Plus it'll fill up the cache and they may soon be aged out anyway. Can you really afford the space? Is it really a problem that

Re: Loading data to FieldValueCache

Thanks Toke for the explanation, I will experiment with f.state.facet.method=enum Thanks, Manohar On Fri, Dec 26, 2014 at 4:09 PM, Toke Eskildsen wrote: > Manohar Sripada [manohar...@gmail.com] wrote: > > I have 100 million documents in my index. The maxDoc here is the maximum > > Documents in

RE: Loading data to FieldValueCache

Manohar Sripada [manohar...@gmail.com] wrote: > I have 100 million documents in my index. The maxDoc here is the maximum > Documents in each shard, right? How is it determined that each entry will > occupy maxDoc/8 approximately. Assuming that it is random whether a document is part of the result

Re: Loading data to FieldValueCache

I have 100 million documents in my index. The maxDoc here is the maximum Documents in each shard, right? How is it determined that each entry will occupy maxDoc/8 approximately. If I have to add facet.method=enum every time in the query, how should I specify for each field separately? Like in the

Re: Loading data to FieldValueCache

Inline. On Tue, Dec 23, 2014 at 11:12 PM, Manohar Sripada wrote: > Okay. Let me try like this, as mine is a read-only index. I will have some > queries in firstSearcher event listener > 1) q=*:*&facet=true&facet.method=enum&facet.field=state --> To load all > the state related unique values to

Re: Loading data to FieldValueCache

Okay. Let me try like this, as mine is a read-only index. I will have some queries in firstSearcher event listener 1) q=*:*&facet=true&facet.method=enum&facet.field=state --> To load all the state related unique values to filterCache. > Will it use filterCache when I sent a query with filter,

Re: Loading data to FieldValueCache

By and large, don't use the enum method unless there are _very_ few unique values. It forms a filter (size roughly mixDoc/8 bytes) for _every_ unique value in the field, i.e. if you have 10,000 unique values it'll try to form 10,000 filterCache entries. Let the system do this for you automatically

Re: Loading data to FieldValueCache

Thanks Erick and Toke, Also, I read here that, filterCache can also be used for faceting with facet.method=enum. So, I am bit confused here on which one to use for faceting. One more thing here is I have different types of facets. (For exampl

Re: Loading data to FieldValueCache

Or just not worry about it. The cache will be filled up automatically as you query for facets etc., the benefit to trying to fill it up as Toke outlines is just that the first few user queries that call for faceting will be somewhat faster. But after the first few user queries have gone through, it

RE: Loading data to FieldValueCache

Manohar Sripada [manohar...@gmail.com] wrote: > From the wiki, it states that > http://wiki.apache.org/solr/SolrCaching#fieldValueCache is mostly used for > faceting. > Can someone please throw some light on how to load data to this cache. Like > on what solrquery option does this consider the dat