On Sat, Jun 13, 2009 at 1:36 AM, Ensdorf Ken <ensd...@zoominfo.com> wrote:
> I ran into this very issue recently as we are using a "freshness" filter > for our data that can be 6//12/18 months etc. I discovered that even though > we were only indexing with day-level granularity, we were specifying the > query by computing a date down to the second and thus virutally every filter > was unique. It's amazing how something this simple could bring solr to it's > knees on a large data set. By simply changing the filter to > "date:[NOW-18MONTHS TO NOW]" or equivalent, the problem vanishes. > Since you are indexing with day-level granularity, you should query too with the same granularity. For example, "date:[NOW/DAY-18MONTHS TO NOW/DAY]". The '/' operator is used for rounding off in DateMath syntax ( http://lucene.apache.org/solr/api/org/apache/solr/util/DateMathParser.html). Perhaps this is something we should document more clearly, we recently had high CPU issues with one of our webapps due to the same issue. > > It does bring up an interestion question though - how is "NOW" treated wrt > to the cache key? Does solr translate it to a date first? If so, how does > it determine the granularity? If not, is there any mechanism to flush the > cache when the corresponding result set changes? > > The date math syntax is translated to a date before a search is performed. NOW is always granular upto seconds (maybe milliseconds, not sure). -- Regards, Shalin Shekhar Mangar.
