Re: Query vs Filter Query Usage

2011-08-25 Thread Erick Erickson
The pitfalls of filter queries is also their strength. The results will be cached and re-used if possible. This will take some memory, of course. Depending upon how big your index is, this could be quite a lot. Yet another time/space tradeoff But yeah, use filter queries until you have OOMs,

Re: Query vs Filter Query Usage

2011-08-25 Thread Joshua Harness
Erick - Thanks for the insight. Does the filter cache just cache the internal document id's of the result set, correct (as opposed to the document)? If so, am I correct in the following math: 10,000,000 document index Internal Document id is 32 bit unsigned int Max Memory Used by a single cache

RE: Query vs Filter Query Usage

2011-08-25 Thread Michael Ryan
10,000,000 document index Internal Document id is 32 bit unsigned int Max Memory Used by a single cache slot in the filter cache = 32 bits x 10,000,000 docs = 320,000,000 bits or 38 MB I think it depends on where exactly the result set was generated. I believe the result set will usually be

Re: Query vs Filter Query Usage

2011-08-25 Thread Yonik Seeley
On Thu, Aug 25, 2011 at 5:19 PM, Michael Ryan mr...@moreover.com wrote: 10,000,000 document index Internal Document id is 32 bit unsigned int Max Memory Used by a single cache slot in the filter cache = 32 bits x 10,000,000 docs = 320,000,000 bits or 38 MB I think it depends on where exactly

Re: Query vs Filter Query Usage

2011-08-25 Thread Lance Norskog
The point of filter queries is that they are applied very early in the searching algorithm, and thus cut the amount of work later on. Some complex queries take a lot of time and so this pre-trimming helps a lot. On Thu, Aug 25, 2011 at 2:37 PM, Yonik Seeley yo...@lucidimagination.com wrote: On

Query vs Filter Query Usage

2011-08-24 Thread Joshua Harness
All - I apologize if this question has been asked before - I couldn't seem to find a straightforward answer by researching it on google and stackoverflow. I am trying to understand when I should use filter queries vs plain vanilla queries. Here's what I understand: * Filter queries can be

Re: Query vs Filter Query Usage

2011-08-24 Thread Shawn Heisey
On 8/24/2011 2:02 PM, Joshua Harness wrote: I've done some basic query performance testing on my SOLR instance, which allows users to search via a faceted search interface. As such, document relevancy is less important to me since I am performing exact match searching. Comparing using

Re: Query vs Filter Query Usage

2011-08-24 Thread Joshua Harness
Shawn - Thanks for your reply. Given that my application is mainly used as faceted search, would the following types of queries make sense or are there other pitfalls to consider? *q=*:*fq=someField:someValuefq=anotherField:anotherValue* Thanks! Josh On Wed, Aug 24, 2011 at 4:48 PM,

Re: Query vs Filter Query Usage

2011-08-24 Thread Shawn Heisey
On 8/24/2011 6:07 PM, Joshua Harness wrote: Shawn - Thanks for your reply. Given that my application is mainly used as faceted search, would the following types of queries make sense or are there other pitfalls to consider? q=*:*fq=someField:someValuefq=anotherField:anotherValue I'm no