[
https://issues.apache.org/jira/browse/SOLR-5463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841715#comment-13841715
]
Hoss Man commented on SOLR-5463:
--------------------------------
bq. All the new tests pass for me with the latest patch.
Right, i may not have been clear before but the problem that confuses me is:
Even with queryResultCaching enabled in the tests, they pass -- and do not
exhibit the same problems i'm seeing when manually using searchAfter with the
example configs.
bq. Also, if it were the case that the queryResultCache were ignoring the
searchAfter param, then pages 2 and 3 would return the same results as the
first page, which is not the case...
When {{searchAfter=\*}} is used, the search itself is no differnet then a
regular query, it can be cached, or re-used from an existing cached query --
the only thing special that happens is that the DeepPagingComponent knows it
needs to compute the "nextSearchAfter" totem. Once you request "page #2" (with
a searchAfter other then "\*", then a PostFilter is used, and you can see from
the cache stats that it's not considered a hit on the initial query -- so far
so good, but as you mentioned "page #3" is where we start to see the same data
as page #2 returned because it's a "cache hit".
It would not suprise me at all if there is a bug in my DeepPagingComponent that
was causing caching to be used when it shouldn't be (even though i think i did
everything right in the PostFilter) ... what suprises me is how easy it is to
reproduce manually, but how hard it is to reproduce in the automated tests.
> Provide cursor/token based "searchAfter" support that works with arbitrary
> sorting (ie: "deep paging")
> ------------------------------------------------------------------------------------------------------
>
> Key: SOLR-5463
> URL: https://issues.apache.org/jira/browse/SOLR-5463
> Project: Solr
> Issue Type: New Feature
> Reporter: Hoss Man
> Assignee: Hoss Man
> Attachments: SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch,
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch,
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch
>
>
> I'd like to revist a solution to the problem of "deep paging" in Solr,
> leveraging an HTTP based API similar to how IndexSearcher.searchAfter works
> at the lucene level: require the clients to provide back a token indicating
> the sort values of the last document seen on the previous "page". This is
> similar to the "cursor" model I've seen in several other REST APIs that
> support "pagnation" over a large sets of results (notable the twitter API and
> it's "since_id" param) except that we'll want something that works with
> arbitrary multi-level sort critera that can be either ascending or descending.
> SOLR-1726 laid some initial ground work here and was commited quite a while
> ago, but the key bit of argument parsing to leverage it was commented out due
> to some problems (see comments in that issue). It's also somewhat out of
> date at this point: at the time it was commited, IndexSearcher only supported
> searchAfter for simple scores, not arbitrary field sorts; and the params
> added in SOLR-1726 suffer from this limitation as well.
> ---
> I think it would make sense to start fresh with a new issue with a focus on
> ensuring that we have deep paging which:
> * supports arbitrary field sorts in addition to sorting by score
> * works in distributed mode
--
This message was sent by Atlassian JIRA
(v6.1#6144)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
