Thanks Pablo. I'll be flying to New Orleans tomorrow for ApacheCon and would love the opportunity to talk with others about architectures others are using.
Todd On 11/4/08, PabloS <[EMAIL PROTECTED]> wrote: > > Sure Todd, > > the idea basically consist in the following: > > - Subclassing FIeldSortedHitQueue and calling support with an empty > SortField array: this disables caching because the comparators are retrieved > during construction > - Creating a new SortComparatorSource that creates the sort comparators > similarly to FieldCacheImpl but without performing any caching > - Creating a new HitCollector that behaves just like TopFieldDocCollector > but uses my custom FieldSortedHitQueue. The tricky part was creating a > TopFieldDoc instance (which has default visibility constructor. I managed to > do it creating an empty TopFieldDocCollector, asking the TopFieldDocs and > modifiying that instance. > - Overriding the Searcher's search(Weight, Filter, int, Sort) so it uses my > HitCollector (I use the Hits class, which internally calls that method) > > That way, you completely override field caching and you'll suffer the > performance consequences. What I did was making the SortComparatorSource a > spring bean and adding a caching interceptor to it. You an plug your > favorite cache library in, in my case ehcache. > > I hope you find this useful. > Regards. > > > tbenge wrote: >> >> Pablo, >> >> Would you mind adding a little more detail about how you're working >> around the problem? >> >> I'm still evaluating our different options so am interested in what you >> did. >> >> Todd >> >> On Mon, Nov 3, 2008 at 2:37 PM, PabloS <[EMAIL PROTECTED]> wrote: >>> >>> Thanks hossman, but I've already 'solved' the problem without the need to >>> patch lucene. I had to code a bit around Lucene's visibility restrictions >>> but I've managed to completely skip the field caching mechanism and add >>> ehcache to it. >>> >>> At the moment it seems to be working quite well, although not as fast as >>> it >>> was when lucene performed the caching. >>> >>> Thanks a lot for the info anyway. >>> Regards. >>> >>> hossman wrote: >>>> >>>> >>>> : I'm having a similar problem with my application, although we are >>>> using >>>> : lucene 2.3.2. The problem we have is that we are required to sort on >>>> most of >>>> : the fields (20 at least). Is there any way of changing the cache being >>>> used? >>>> >>>> there is a patch in Jira that takes a completley different approach >>>> towards dealing with sorting by using the *stored* values of the >>>> documents >>>> that match... >>>> >>>> https://issues.apache.org/jira/browse/LUCENE-769 >>>> >>>> ...in theory it's "better" for use cases where: >>>> >>>> 1) you are confident you are going to get back a "small" result set in >>>> proportion to the total size of your index. >>>> 2) you need to support sorting on "lots" of fields and don't have >>>> enough >>>> ram for all of the FieldCaches of those fields >>>> >>>> So far the only person to test it out (and post comments) is the patch >>>> submitter, but if other people report success it might be worth adding >>>> as >>>> a contrib. >>>> >>>> (Disclaimer: it looks like the patch was updated after my last >>>> review/comments. i have not read, nor claim any opinion about the >>>> currently attached patch) >>>> >>>> >>>> >>>> >>>> -Hoss >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>> >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/OutOfMemory-Problems-Lucene-2.4---Tomcat-tp20236834p20311386.html >>> Sent from the Lucene - Java Users mailing list archive at Nabble.com. >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>> For additional commands, e-mail: [EMAIL PROTECTED] >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> >> > > -- > View this message in context: > http://www.nabble.com/OutOfMemory-Problems-Lucene-2.4---Tomcat-tp20236834p20326783.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Sent from Gmail for mobile | mobile.google.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
