Good luck, and let us know what the results are. About dropping the cache.. That
shouldn't be a problem, it should just be computed when your
component is called
the first time, so starting the server (or opening a new searcher)
should re-compute
it. Your filters shouldn't be very big, just maxDocs/8 bytes long...
By the way, there exist "user caches" (see solrconfig.xml) that you can use
for whatever you want, so you could consider stashing your filters in there. The
neat thing is that they get notified whenever the searcher is opened (?) and you
can regenerate the data held there. MIght be more convenient than mucking with
filter query caching...
Best
Erick
On Fri, Aug 19, 2011 at 2:58 AM, Tomas Zerolo
<tomas.zer...@axelspringer.de> wrote:
> On Thu, Aug 18, 2011 at 02:32:48PM -0400, Erick Erickson wrote:
>> Hmmm, I'm still not getting it...
>>
>> You have one or more lists. These lists change once a month or so. Are
>> you trying
>> to include or exclude the documents in these lists?
>
> In our specific case to include *only* the documents having a value of
> an attribute ("author") in this list (the user decides at query time
> which of those lists to use). But we do expect the problem to become
> more general over time...
>
>> And do the authors you
>> want
>> to include or exclude change on a per-query basis or would you be all set if
>> you
>> just had a filter that applied to all the authors on a particular list?
>
> No. ATM there are two "fixed" lists (in the sense that they are updated
> like "monthly". One problem: the document basis itself is huge (in the
> abouts of 3.5 million). Re-indexing is a painful exercise taking days,
> so we tend not to do it too often ;-)
>
>> But I *think* what you want is a SearchComponent that implements your
>> Filter. You can see various examples of how to add components to a seach
>> handler in the solrconfig.xml file.
>
> Thanks a lot for the pointer. Rushing to read on it.
>
>> WARNING: Haven't done this myself, so I'm partly guessing here.
>
> Hey: I asked for pointers and you're giving me some, so I'm a happy
> man now :-)
>
>> Although here's a hint that someone else has used this approach:
>> http://www.mail-archive.com/solr-user@lucene.apache.org/msg54240.html
>
> Thanks again
>
>> And you'll want to insure that the Filter is cached so you don't have to
>> compute
>> it more than once.
>
> Yes, I hope that will be the trick giving us the needed boost. Somehow
> we'll have to figure out how to drop the cache when a new version of the
> list arrives (without killing everyone in the building).
>
> I'll sure report back.
>
> Regards
> -- tomás
>
