Sure, you can  write a custom function, see:
https://cwiki.apache.org/confluence/display/solr/Function+Queries

And you can invoke your custom function since sorting by function is supported.

But my point remains. To be performant, you'll have to cache the
results. Which is what's happening already.
If you do something clever that tries to purge old values that you're
sorting by, then you'll probably run into
performance issues. At least that's my guess.

I think this will be a dead-end for you, but would love to be proved
wrong about that....

Best
Erick

On Wed, Jun 25, 2014 at 4:34 AM, Sandeep Khanzode
<sandeep_khanz...@yahoo.com.invalid> wrote:
> Hi,
>
> Thanks for your reply.
> Actually, I am evaluating both approaches.
>
> With the sort being performed on a field indexed in Lucene itself, my concern 
> is with the FieldCache. Very quickly, for multiple clients executing in 
> parallel, it bumps up to 8-10GB. This is for 4-5 different Sort fields using 
> an index corpus of 50M documents. The problem is not so much the memory 
> consumption, as mush as controlling it. If the max heap argument for the JVM 
> is scaled back to 2-3GB, then all clients throw an OOM. How should the 
> FieldCache scale based on the amount of available max memory to the JVM or 
> can it be selectively turned off, or implement a LRU type of algorithm to 
> purge old entries?
>
> Secondly, the the DB approach, yes, it will not perform. However, I just 
> wanted to know whether such a custom sort function exists that allows one to 
> write their own sort on a field that is not indexed by Lucene.
>
> Thanks again,
>
> -----------------------
> Thanks n Regards,
> Sandeep Ramesh Khanzode
>
>
> On Wednesday, June 25, 2014 1:21 AM, Erick Erickson <erickerick...@gmail.com> 
> wrote:
>
>
>
> I'm a little confused here. Sure, sorting on a number of fields will
> increase memory, the basic idea here is that you need to cache all the
> sort values (plus support structures) for performance reasons.
>
> If you create your own custom sort that goes out to a DB and gets the
> doc, you have to be prepared for
> q=*:*&sort=custom_function
> Which means you'll have to fetch the value for each and every document
> in the index. If this is a DB call, it will NOT perform.
>
> In order to be performant, you'll need to cache the values. Which is
> what is being done _for_ you by the FieldCache.
>
> So I think this is really a false path, or an "XY" problem. Why do you
> think you need to do this?
>
> Best,
> Erick
>
>
> On Tue, Jun 24, 2014 at 10:31 AM, Sandeep Khanzode
> <sandeep_khanz...@yahoo.com.invalid> wrote:
>> Hi,
>>
>> I am trying to implement a sort order for search results in Lucene 4.7.2.
>>
>> If I want to use data for ordering that is not stored in Lucene as Fields, 
>> is there any way this can be done?
>> Basically, I would have certain data that is associated logically to a 
>> document but stored elsewhere, like a DB. Can I create a Custom Sort 
>> function on the lines of a FieldComparator to sort based on this data by 
>> plugging it inside the sort function?
>>
>> I have tested the performance of the Sort function for String and numeric 
>> types, and as mentioned in some blog, it seems that the numeric type is much 
>> faster compared to the string type. However, if I sort on a number of fields 
>> from multiple clients, the memory footprint, due to the FieldCache.DEFAULT 
>> impl, increases approximately 5-6 times. If I run this on a machine which 
>> does not have this capacity, will I get a OOM or will there be intense 
>> thrashing for the memory?
>>
>>
>> -----------------------
>> Thanks n Regards,
>> Sandeep Ramesh Khanzode
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Reply via email to