LUCENE-6766 index sorting and custom SortField

Hi, LUCENE-6766 allows to define index sorting on IndexWriterConfig instead of defining a SortingMergePolicy. However, the new index sorting only supports some types of sort fields, and the old SortingMergePolicy, which didn't have this limitation, has been removed. What should do projects depend

Re: LUCENE-6766 index sorting and custom SortField

Unfortunately, as of LUCENE-6766, index sorting only supports simple sort types. This was needed because Lucene needs to be able to easily serialize and de-serialize the sort order into the index. Can you compute your sort criteria and index it as a doc values field and then sort by that? Or, pa

Re: LUCENE-6766 index sorting and custom SortField

Hi, We are using a custom SortField to sort Cassandra primary keys. The sort criteria is based on the marshalled values of each of the columns in the primary key

Re: LUCENE-6766 index sorting and custom SortField

Hmm I see. Yeah, it seems like the only way forward is to explore SortField (and its subclasses) handling their own serialization, maybe via SPI (what we use for codecs), though that sounds somewhat heavy. Maybe open an issue for discussion? Mike McCandless http://blog.mikemccandless.com On Mo

Re: LUCENE-6766 index sorting and custom SortField

Maybe another way would be to create a custom SegmentInfoFormat that handles the serialization of this custom SortField. That would put the burden on the user to handle backward compatibility, but on the other hand it would not require SortFields to handle their own serialization? It would not work

Re: LUCENE-6766 index sorting and custom SortField

I like that idea! Mike McCandless http://blog.mikemccandless.com On Tue, Aug 16, 2016 at 4:45 AM, Adrien Grand wrote: > Maybe another way would be to create a custom SegmentInfoFormat that > handles the serialization of this custom SortField. That would put the > burden on the user to handle b

Re: LUCENE-6766 index sorting and custom SortField

Do you mean creating a custom SegmentInfoFormat per each custom SortField? 2016-08-16 10:20 GMT+01:00 Michael McCandless : > I like that idea! > > Mike McCandless > > http://blog.mikemccandless.com > > On Tue, Aug 16, 2016 at 4:45 AM, Adrien Grand wrote: > >> Maybe another way would be to create

Re: LUCENE-6766 index sorting and custom SortField

It would be a custom Codec implementation, where your codec could then make its own SegmentInfoFormat that knew how to serialize the custom sort field you had set on IndexWriterConfig. Mike McCandless http://blog.mikemccandless.com On Tue, Aug 16, 2016 at 2:51 PM, Andres de la Peña wrote: > Do

Re: LUCENE-6766 index sorting and custom SortField

I see. Each project not using the supported types should create its own SegmentInfoFormat able to serialize the involved sort fields. This should be provided by a custom FilterCodec, registered with the SPI mechanism. Also the checkings about supported SortFields should be skipped in some way. I t

Re: LUCENE-6766 index sorting and custom SortField

Le mar. 16 août 2016 à 22:21, Andres de la Peña a écrit : > I think that autoserializable SortFields should be considerably easier to > use, keeping the purpose of LUCENE-6766: to ease the usage of index > sorting, thus not being an expert feature. > I don't think a feature can be non-expert and

Re: LUCENE-6766 index sorting and custom SortField

Index time sorting was always an experimental feature ... it's free to change, be removed, be restricted, etc. We should probably just keep it simple and require users to compile their sort needs into an indexed (as doc values) sort key. Mike McCandless http://blog.mikemccandless.com On Wed, Au