bq: To be sure we are using cost 101 and no cache The guy who wrote the code is really good, but I'm paranoid too so I use 101. Based on the number of off-by-one errors I've coded :)...
How slow is "around commit points really slow"? You could at least lessen the pain here by committing less often if you can stand the latency.... But otherwise you've pretty much nailed your options. One approach is to give users _predictable_ responses, not necessarily the best average. Often users are more disturbed by getting (numbers from thin air) 2 second responses occasionally spiking to 20 seconds with an average of 3 seconds than getting all responses between 4 and 6 seconds with an average of 5. FWIW, Erick On Thu, Dec 5, 2013 at 7:39 AM, Dmitry Kan <solrexp...@gmail.com> wrote: > Thanks Erick! > To be sure we are using cost 101 and no cache. It seems to affect on > searches as we expected. > > Basically with cache on we see more "fat" spikes around commit points, as > cache is getting flushed (we don't rerun too many entries from old cache). > But when the post-filtering is involved, those spikes are thinner, but the > rest of the queries take about 2 seconds longer (our queries are pretty > heavy duty stuff). > > So the post-filtering gives an option of making trade-offs between query > times for all users during normal execution and query times during commits. > To rephrase we have 2 options: > > 1. Make all searches somewhat slower for all users and avoid really slow > searches around commit points: post-filtering option > > OR > > 2. Make majority of searches really fast, but around commit points really > slow: normal with cache option > > Dmitry > > > On Wed, Dec 4, 2013 at 3:34 PM, Erick Erickson <erickerick...@gmail.com > >wrote: > > > OK, so cache=false and cost=100 should do it, see: > > http://searchhub.org/2012/02/22/custom-security-filtering-in-solr/ > > > > Best, > > Erick > > > > > > On Wed, Dec 4, 2013 at 5:56 AM, Dmitry Kan <solrexp...@gmail.com> wrote: > > > > > Thanks Yonik. > > > > > > For our use case, we would like to skip caching only one particular > > filter > > > cache, yet apply a high cost for it to make sure it executes last of > all > > > filter queries. > > > > > > So this means, the rest of the fqs will execute and cache as usual. > > > > > > > > > > > > > > > On Tue, Dec 3, 2013 at 9:58 PM, Yonik Seeley <yo...@heliosearch.com> > > > wrote: > > > > > > > On Tue, Dec 3, 2013 at 4:45 AM, Dmitry Kan <solrexp...@gmail.com> > > wrote: > > > > > ok, we were able to confirm the behavior regarding not caching the > > > filter > > > > > query. It works as expected. It does not cache with {!cache=false}. > > > > > > > > > > We are still looking into clarifying the cost assignment: i.e. > > whether > > > it > > > > > works as expected for long boolean filter queries. > > > > > > > > Yes, filters should be ordered by cost (cheapest first) whenever you > > > > use {!cache=false} > > > > > > > > -Yonik > > > > http://heliosearch.com -- making solr shine > > > > > > > > > > > > > > > > -- > > > Dmitry > > > Blog: http://dmitrykan.blogspot.com > > > Twitter: twitter.com/dmitrykan > > > > > > > > > -- > Dmitry > Blog: http://dmitrykan.blogspot.com > Twitter: twitter.com/dmitrykan >
