[
https://issues.apache.org/jira/browse/SOLR-5463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe updated SOLR-5463:
-----------------------------
Attachment: SOLR-5463__straw_man__MissingStringLastComparatorSource.patch
{quote} One limitation i uncovered however is that
MissingStringLastComparatorSource (which ironically is used by both
sortMissingLast and sortMissingFirst) throws UsupOpEx for a large number of
it's methods, which makes it impossible to use to filter out docs that are
"before" the searchAfter totem. It should be fixable, but i've punted on this
for now.
{quote}
I investigated this, and added an implementation for the only method that was
needed ({{TermOrdValComparator_SML.compareDocToValues(int doc, Comparable
docValue)}}). The attached patch adds this implementation and uncomments the
elements in {{schema-sorts.xml}} that Hoss commented out to get tests to pass,
and assumes Hoss's most recent strawman patch has already been applied. (I
also tried converting the class to extend {{FieldComparator<BytesRef>}} instead
of the current {{FieldComparator<Comparable>}}, but that causes
{{TestDistributedGrouping}} to fail with a message about {{Integer}} not being
castable to {{BytesRef}} - that's why I introduced an {{instanceof BytesRef}}
check to convert the incoming value to {{BytesRef}} if necessary, via
{{toString()}}.)
NOTE: this patch is *not* required by Hoss's latest (non-strawman) patch - when
I uncomment the commented-out elements in {{schema-sorts.xml}} that triggered
test failures in the strawman, all tests still pass, *without* this patch. So
I'm attaching the patch to this issue just for anybody who wants to try out the
strawman implementation.
I believe my patch is not needed in the latest non-strawman implementation
because {{TermOrdValComparator_SML}} delegates to inner class
{{AnyOrdComparator}} for per-segment sorting, and {{AnyOrdComparator}} has a
functional implementation of {{compareDocToValues()}}; unlike the strawman
implementation, the latest version invokes sorts on a per-segment basis, via
{{PaginationCollector}}.
> Provide cursor/token based "searchAfter" support that works with arbitrary
> sorting (ie: "deep paging")
> ------------------------------------------------------------------------------------------------------
>
> Key: SOLR-5463
> URL: https://issues.apache.org/jira/browse/SOLR-5463
> Project: Solr
> Issue Type: New Feature
> Reporter: Hoss Man
> Assignee: Hoss Man
> Attachments: SOLR-5463.patch, SOLR-5463__straw_man.patch,
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch,
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch,
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch,
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch,
> SOLR-5463__straw_man.patch,
> SOLR-5463__straw_man__MissingStringLastComparatorSource.patch
>
>
> I'd like to revist a solution to the problem of "deep paging" in Solr,
> leveraging an HTTP based API similar to how IndexSearcher.searchAfter works
> at the lucene level: require the clients to provide back a token indicating
> the sort values of the last document seen on the previous "page". This is
> similar to the "cursor" model I've seen in several other REST APIs that
> support "pagnation" over a large sets of results (notable the twitter API and
> it's "since_id" param) except that we'll want something that works with
> arbitrary multi-level sort critera that can be either ascending or descending.
> SOLR-1726 laid some initial ground work here and was commited quite a while
> ago, but the key bit of argument parsing to leverage it was commented out due
> to some problems (see comments in that issue). It's also somewhat out of
> date at this point: at the time it was commited, IndexSearcher only supported
> searchAfter for simple scores, not arbitrary field sorts; and the params
> added in SOLR-1726 suffer from this limitation as well.
> ---
> I think it would make sense to start fresh with a new issue with a focus on
> ensuring that we have deep paging which:
> * supports arbitrary field sorts in addition to sorting by score
> * works in distributed mode
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
