Re: Is the act of *caching* an fq very expensive? (seems to cost 4 seconds in my example)

2014-06-03 Thread Yonik Seeley
On Tue, Jun 3, 2014 at 9:48 PM, Brett Hoerner wrote: > Yonik, I'm familiar with your blog posts -- and thanks very much for them. > :) Though I'm not sure what you're trying to show me with the q=*:* part? I > was of course using q=*:* in my queries, but I assume you mean to leave off > the text:l

Re: Is the act of *caching* an fq very expensive? (seems to cost 4 seconds in my example)

2014-06-03 Thread Brett Hoerner
Yonik, I'm familiar with your blog posts -- and thanks very much for them. :) Though I'm not sure what you're trying to show me with the q=*:* part? I was of course using q=*:* in my queries, but I assume you mean to leave off the text:lol bit? I've done some Cluster changes, so these are my basel

Re: Is the act of *caching* an fq very expensive? (seems to cost 4 seconds in my example)

2014-06-03 Thread Yonik Seeley
On Tue, Jun 3, 2014 at 5:19 PM, Yonik Seeley wrote: > So try: > q=*:* > fq=created_at_tdid:[1400544000 TO 1400630400] vs So try: q=*:* fq={!cache=false}created_at_tdid:[1400544000 TO 1400630400] -Yonik http://heliosearch.org - facet functions, subfacets, off-heap filters&fieldcache

Re: Is the act of *caching* an fq very expensive? (seems to cost 4 seconds in my example)

2014-06-03 Thread Yonik Seeley
On Tue, Jun 3, 2014 at 4:44 PM, Brett Hoerner wrote: > If I run a query like this, > > fq=text:lol > fq=created_at_tdid:[1400544000 TO 1400630400] > > It takes about 6 seconds. Following queries take only 50ms or less, as > expected because my fqs are cached. > > However, if I change the query to

Re: Is the act of *caching* an fq very expensive? (seems to cost 4 seconds in my example)

2014-06-03 Thread Brett Hoerner
This is seemingly where it checks whether to use cache or not, the extra work is really just a get (miss) and a put: https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L1216 I suppose it's possible the put is taking 4 seconds, but th

Re: Is the act of *caching* an fq very expensive? (seems to cost 4 seconds in my example)

2014-06-03 Thread Brett Hoerner
In this case, I have >400 million documents, so I understand it taking a while. That said, I'm still not sure I understand why it would take *more* time. In your example above, wouldn't it have to create an 11.92MB bitset even if I *don't* cache the bitset? It seems the mere act of storing the wor

Re: Is the act of *caching* an fq very expensive? (seems to cost 4 seconds in my example)

2014-06-03 Thread Shawn Heisey
On 6/3/2014 2:44 PM, Brett Hoerner wrote: > If I run a query like this, > > fq=text:lol > fq=created_at_tdid:[1400544000 TO 1400630400] > > It takes about 6 seconds. Following queries take only 50ms or less, as > expected because my fqs are cached. > > However, if I change the query to not cache my

Is the act of *caching* an fq very expensive? (seems to cost 4 seconds in my example)

2014-06-03 Thread Brett Hoerner
If I run a query like this, fq=text:lol fq=created_at_tdid:[1400544000 TO 1400630400] It takes about 6 seconds. Following queries take only 50ms or less, as expected because my fqs are cached. However, if I change the query to not cache my big range query: fq=text:lol fq={!cache=false}created_a