Re: post filtering for boolean filter queries

2013-12-07 Thread Dmitry Kan
On Thu, Dec 5, 2013 at 4:49 PM, Yonik Seeley yo...@heliosearch.com wrote: On Thu, Dec 5, 2013 at 7:39 AM, Dmitry Kan solrexp...@gmail.com wrote: Thanks Erick! To be sure we are using cost 101 and no cache. It seems to affect on searches as we expected. Basically with cache on we see

Re: post filtering for boolean filter queries

2013-12-07 Thread Dmitry Kan
bq. How slow is around commit points really slow? You could at least lessen the pain here by committing less often if you can stand the latency They are shamelessly slow, like 60-70 seconds. While normal searches are within 1-3 seconds range. And, yes. your idea is right and what we are

Re: post filtering for boolean filter queries

2013-12-05 Thread Dmitry Kan
Thanks Erick! To be sure we are using cost 101 and no cache. It seems to affect on searches as we expected. Basically with cache on we see more fat spikes around commit points, as cache is getting flushed (we don't rerun too many entries from old cache). But when the post-filtering is involved,

Re: post filtering for boolean filter queries

2013-12-05 Thread Erick Erickson
bq: To be sure we are using cost 101 and no cache The guy who wrote the code is really good, but I'm paranoid too so I use 101. Based on the number of off-by-one errors I've coded :)... How slow is around commit points really slow? You could at least lessen the pain here by committing less often

Re: post filtering for boolean filter queries

2013-12-05 Thread Yonik Seeley
On Thu, Dec 5, 2013 at 7:39 AM, Dmitry Kan solrexp...@gmail.com wrote: Thanks Erick! To be sure we are using cost 101 and no cache. It seems to affect on searches as we expected. Basically with cache on we see more fat spikes around commit points, as cache is getting flushed (we don't rerun

Re: post filtering for boolean filter queries

2013-12-04 Thread Dmitry Kan
Thanks Yonik. For our use case, we would like to skip caching only one particular filter cache, yet apply a high cost for it to make sure it executes last of all filter queries. So this means, the rest of the fqs will execute and cache as usual. On Tue, Dec 3, 2013 at 9:58 PM, Yonik Seeley

Re: post filtering for boolean filter queries

2013-12-04 Thread Erick Erickson
OK, so cache=false and cost=100 should do it, see: http://searchhub.org/2012/02/22/custom-security-filtering-in-solr/ Best, Erick On Wed, Dec 4, 2013 at 5:56 AM, Dmitry Kan solrexp...@gmail.com wrote: Thanks Yonik. For our use case, we would like to skip caching only one particular filter

Re: post filtering for boolean filter queries

2013-12-03 Thread Dmitry Kan
ok, we were able to confirm the behavior regarding not caching the filter query. It works as expected. It does not cache with {!cache=false}. We are still looking into clarifying the cost assignment: i.e. whether it works as expected for long boolean filter queries. On Tue, Dec 3, 2013 at 8:55

Re: post filtering for boolean filter queries

2013-12-03 Thread Yonik Seeley
On Tue, Dec 3, 2013 at 4:45 AM, Dmitry Kan solrexp...@gmail.com wrote: ok, we were able to confirm the behavior regarding not caching the filter query. It works as expected. It does not cache with {!cache=false}. We are still looking into clarifying the cost assignment: i.e. whether it works

Re: post filtering for boolean filter queries

2013-12-03 Thread Michael Sokolov
On 12/03/2013 01:55 AM, Dmitry Kan wrote: Hello! We have been experimenting with post filtering lately. Our setup is a filter having long boolean query; drawing the example from the Dublin's Stump the Chump: fq=UserId:(user1 OR user2 OR...OR user1000) The underlining issue impacting

post filtering for boolean filter queries

2013-12-02 Thread Dmitry Kan
Hello! We have been experimenting with post filtering lately. Our setup is a filter having long boolean query; drawing the example from the Dublin's Stump the Chump: fq=UserId:(user1 OR user2 OR...OR user1000) The underlining issue impacting performance is that the combination of user ids in