Pardon my typo. I meant 1000ms in my last mail. Thanks, -Vijay
On Mon, Mar 10, 2014 at 4:22 PM, Vijay Kokatnur <kokatnur.vi...@gmail.com>wrote: > Thanks Erick. The links you provided are invaluable. > > Here are our commit settings. Since we have NRT search, softCommit is set > to 1000s which explains why cache is constantly invalidated. > > <autoCommit> > <maxTime>600000</maxTime> > <openSearcher>false</openSearcher> > </autoCommit> > > <autoSoftCommit> > <maxTime>1000</maxTime> > </autoSoftCommit> > > > With constant cache invalidation it becomes almost impossible to get > better response times. Is the only to solve this is to fine tune > softCommit settings? > > > > On Fri, Mar 7, 2014 at 6:17 PM, Erick Erickson <erickerick...@gmail.com>wrote: > >> OK, something is not right here. What are >> your autocommit settings? What you pasted >> above looks like you're looking at a searcher that >> has _just_ opened, which would mean either >> 1> you just had a hard commit with openSearcher=false happen >> or >> 2> you just had a soft commit happen >> >> In either case, the cache is thrown out. That said, if you >> have autowarming for the cache set up you should be >> seeing some hits eventually. >> >> The top part is the _current_ searcher. The cumulative_* >> is all the cache results since the application started. >> >> A couple of blogs: >> >> >> http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ >> >> http://searchhub.org/2012/02/23/date-math-now-and-filter-queries/ >> >> I'm going to guess that you have soft commits or hard commits >> with openSearcher=true set to a very short interval and are >> having your filter caches invalidated very frequently, and that is >> misleading you, but that's just a guess. >> >> Best, >> Erick >> >> >> >> On Thu, Mar 6, 2014 at 9:32 PM, Vijay Kokatnur <kokatnur.vi...@gmail.com> >> wrote: >> > My initial approach was to use filter cache static fields. However when >> > filter query is used, every query after the first has the same response >> > time as the first. For instance, when cache is enabled in the query >> under >> > review, response time shoots up to 4-5secs and stays there. >> > >> > We are using default filter cache settings provided with 4.5.0 >> > distribution. >> > >> > Current Filter Cache stats : >> > >> > lookups:0 >> > hits:0 >> > hitratio:0 >> > inserts:0 >> > evictions:0 >> > size:0 >> > warmupTime:0 >> > cumulative_lookups:17135 >> > cumulative_hits:2465 >> > cumulative_hitratio:0.14 >> > cumulative_inserts:14670 >> > cumulative_evictions:0 >> > >> > I did not find what cumulative_* fields mean >> > here<http://wiki.apache.org/solr/SolrAdminStats> , >> > but it looks like nothing is being cached with fq as hit ratio is 0. >> > >> > Any idea whats happening? >> > >> > >> > >> > On Thu, Mar 6, 2014 at 2:41 PM, Ahmet Arslan <iori...@yahoo.com> wrote: >> > >> >> Hoss, >> >> >> >> Thanks for the correction. I missed the /DAY part and thought as it was >> >> StartDate:[NOW TO NOW+1YEAR] >> >> >> >> Ahmet >> >> >> >> >> >> On Friday, March 7, 2014 12:33 AM, Chris Hostetter < >> >> hossman_luc...@fucit.org> wrote: >> >> >> >> : That did the trick Ahmet. The first response was around 200ms, but >> the >> >> : subsequent queries were around 2-5ms. >> >> >> >> Are you really sure you want "cache=false" on all of those filters? >> >> >> >> While the "ClientID:4" query may by something that cahnges >> significantly >> >> enough in every query to not be useful to cache, i suspect you'd find a >> >> lot of value in going ahead and caching those Status:Booked and >> >> StartDate:[NOW/DAY TO NOW/DAY+1YEAR] clauses ... the first query to hit >> >> them might be "slower" but ever query after that should be fairly fast >> -- >> >> and if you really need them to *always* be fast, configure them as >> static >> >> newSeracher warming queries (or make sure you have autowarming on. >> >> >> >> It also look like you forgot the "StartDate:" part of your range query >> in >> >> your last test... >> >> >> >> : &fq={!cache=false cost=50}[NOW/DAY TO NOW/DAY+1YEAR] >> >> >> >> And one finally comment just to make sure it doesn't slip throug hthe >> >> cracks.... >> >> >> >> >> >> : > > Since your range query has NOW in it, it won't be cached >> >> meaningfully. >> >> >> >> this is not applicable. the use of "NOW" in a range query doesn't mean >> >> that it can't be cached -- the problem is anytime you use really >> precise >> >> dates (or numeric values) that *change* in every query. >> >> >> >> if your range query uses "NOW" as a lower/upper end point, then it >> calls >> >> in that "really precise dates" situation -- but for this user, who is >> >> specifically rounding his dates to hte nearest day, that advice isn't >> >> really applicable -- the date range queries can be cached & reused for >> an >> >> entire day. >> >> >> >> >> >> >> >> -Hoss >> >> http://www.lucidworks.com/ >> >> >> >> >> > >
