We see similar results, again we softCommit every 1s (trying to get as NRT as we can), and we very rarely get any hits in our caches. As an unscheduled test last week, we did shutdown indexing and noticed about 80% hit rate in caches (and average query time dropped from ~1s to 100ms!) so I think we are in the same position as you.
I appreciate with such a frequent soft commit that the caches get invalidated, but I was expecting cache warming to help though it doesn't appear to be. We *don't* currently run a warming query, my impression of NRT was that it was better to not do that as otherwise you spend more time warming the searcher and caches, and by the time you've done all that, the searcher is invalidated anyway! On 30 June 2013 01:58, Tim Vaillancourt <t...@elementspace.com> wrote: > That's a good idea, I'll try that next week. > > Thanks! > > Tim > > > On 29/06/13 12:39 PM, Erick Erickson wrote: > >> Tim: >> >> Yeah, this doesn't make much sense to me either since, >> as you say, you should be seeing some metrics upon >> occasion. But do note that the underlying cache only gets >> filled when getting documents to return in query results, >> since there's no autowarming going on it may come and >> go. >> >> But you can test this pretty quickly by lengthening your >> autocommit interval or just not indexing anything >> for a while, then run a bunch of queries and look at your >> cache stats. That'll at least tell you whether it works at all. >> You'll have to have hard commits turned off (or openSearcher >> set to 'false') for that check too. >> >> Best >> Erick >> >> >> On Sat, Jun 29, 2013 at 2:48 PM, Vaillancourt, Tim<tvaillanco...@ea.com>* >> *wrote: >> >> Yes, we are softCommit'ing every 1000ms, but that should be enough time >>> to >>> see metrics though, right? For example, I still get non-cumulative >>> metrics >>> from the other caches (which are also throw away). I've also curl/sampled >>> enough that I probably should have seen a value by now. >>> >>> If anyone else can reproduce this on 4.3.1 I will feel less crazy :). >>> >>> Cheers, >>> >>> Tim >>> >>> -----Original Message----- >>> From: Erick Erickson >>> [mailto:erickerickson@gmail.**com<erickerick...@gmail.com> >>> ] >>> Sent: Saturday, June 29, 2013 10:13 AM >>> To: solr-user@lucene.apache.org >>> Subject: Re: documentCache not used in 4.3.1? >>> >>> It's especially weird that the hit ratio is so high and you're not seeing >>> anything in the cache. Are you perhaps soft committing frequently? Soft >>> commits throw away all the top-level caches including documentCache I >>> think.... >>> >>> Erick >>> >>> >>> On Fri, Jun 28, 2013 at 7:23 PM, Tim >>> Vaillancourt<tim@elementspace.**com<t...@elementspace.com> >>> >>>> wrote: >>>> Thanks Otis, >>>> >>>> Yeah I realized after sending my e-mail that doc cache does not warm, >>>> however I'm still lost on why there are no other metrics. >>>> >>>> Thanks! >>>> >>>> Tim >>>> >>>> >>>> On 28 June 2013 16:22, Otis >>>> Gospodnetic<otis.gospodnetic@**gmail.com<otis.gospodne...@gmail.com> >>>> > >>>> wrote: >>>> >>>> Hi Tim, >>>>> >>>>> Not sure about the zeros in 4.3.1, but in SPM we see all these >>>>> numbers are non-0, though I haven't had the chance to confirm with >>>>> >>>> Solr 4.3.1. >>> >>>> Note that you can't really autowarm document cache... >>>>> >>>>> Otis >>>>> -- >>>>> Solr& ElasticSearch Support -- http://sematext.com/ Performance >>>>> >>>>> Monitoring -- http://sematext.com/spm >>>>> >>>>> >>>>> >>>>> On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt >>>>> <t...@elementspace.com> >>>>> wrote: >>>>> >>>>>> Hey guys, >>>>>> >>>>>> This has to be a stupid question/I must be doing something wrong, >>>>>> but >>>>>> >>>>> after >>>>> >>>>>> frequent load testing with documentCache enabled under Solr 4.3.1 >>>>>> with autoWarmCount=150, I'm noticing that my documentCache metrics >>>>>> are >>>>>> >>>>> always >>>> >>>>> zero for non-cumlative. >>>>>> >>>>>> At first I thought my commit rate is fast enough I just never see >>>>>> the non-cumlative result, but after 100s of samples I still always >>>>>> get zero values. >>>>>> >>>>>> Here is the current output of my documentCache from Solr's admin >>>>>> for 1 >>>>>> >>>>> core: >>>>> >>>>>> " >>>>>> >>>>>> - documentCache< >>>>>> >>>>> http://localhost:8983/solr/#/**channels_shard1_replica2/** >>>> plugins/cache?en<http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?en> >>>> try=documentCache >>>> >>>>> - class:org.apache.solr.search.**LRUCache >>>>>> - version:1.0 >>>>>> - description:LRU Cache(maxSize=512, initialSize=512, >>>>>> autowarmCount=150, regenerator=null) >>>>>> - src:$URL: https:/ >>>>>> /svn.apache.org/repos/asf/**lucene/dev/branches/lucene_** >>>>>> solr_4_3/<http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/> >>>>>> solr/core/src/java/org/apache/**solr/search/LRUCache.java< >>>>>> >>>>> https://svn.apache.org/repos/**asf/lucene/dev/branches/** >>>> lucene_solr_4_3/s<https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/s> >>>> olr/core/src/java/org/apache/**solr/search/LRUCache.java >>>> >>>>> $ >>>>>> - stats: >>>>>> - lookups:0 >>>>>> - hits:0 >>>>>> - hitratio:0.00 >>>>>> - inserts:0 >>>>>> - evictions:0 >>>>>> - size:0 >>>>>> - warmupTime:0 >>>>>> - cumulative_lookups:65198986 >>>>>> - cumulative_hits:63075669 >>>>>> - cumulative_hitratio:0.96 >>>>>> - cumulative_inserts:2123317 >>>>>> - cumulative_evictions:1010262 >>>>>> " >>>>>> >>>>>> The cumulative values seem to rise, suggesting doc cache is >>>>>> working, >>>>>> >>>>> but >>>> >>>>> at >>>>> >>>>>> the same time it seems I never see non-cumlative metrics, most >>>>>> >>>>> importantly >>>>> >>>>>> warmupTime. >>>>>> >>>>>> Am I doing something wrong, is this normal/by-design, or is there >>>>>> an >>>>>> >>>>> issue >>>>> >>>>>> here? >>>>>> >>>>>> Thanks for helping with my silly question! Have a good weekend, >>>>>> >>>>>> Tim >>>>>> >>>>>
