[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180102#comment-17180102
]
John Gallagher commented on SOLR-14413:
---------------------------------------
Hi folks, sorry I stepped away from this for a bit, and thanks [~mdrob] , and
[~epugh] for the comments, and [~bvd] for contributing!
To Mike's earlier point: when partialResults is true, you cannot be sure
whether there are more results or not. You could summarize it this way:
|| ||partialResults not present|| partialResults: true||
|more results?|nextCursorMark != cursorMark|unknown|
After figuring out how to properly generate documentation, I took a stab at
adding two notes to the documentation:
1. In timeAllowed documentation,
common-query-parameters.html#timeallowed-parameter
!image-2020-08-18-16-56-59-178.png|width=508,height=116!
2. In the Constraints when using Cursors documentation,
pagination-of-results.html#constraints-when-using-cursors
!image-2020-08-18-16-56-41-736.png|width=410,height=250!
Please let me know what you think of these additions.
I was also able to reproduce [~bvd]'s case where nextCursorMark was null (the
field actually wasn't present in the response at all). I tracked it down to
[this code in
SearchHandler.java|[https://github.com/slackhappy/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L349]].
When the Search times out very early, an exception is thrown before responses
can be handled, which skips the calculation of the next cursor mark. In that
case, the response is null, and the SearchHandler creates an empty response to
return (numFound 0, docs:[]).
I added code to also return the original cursorMark in that case, since we
haven't progressed (it is the value you would want to pass to a subsequent
search).
I added a basic test to CursorPagingTest that checks that the parameters can be
used in conjunction, and that the nextCursorMark returned is a valid one that
can be used in subsequent requests. Perhaps that test can be combined with
[~bvd]'s. I struggled a bit to come up with a test that would produce reliable
results when using timeAllowed. I borrowed a bit from
ExitableDirectoryReaderTest's use of a DelayingSearchComponent, which helped
somewhat.
The updated patch file is SOLR-14413-jg-update1.patch, and I have updated the
[corresponding PR|[https://github.com/apache/lucene-solr/pull/1436]].
John
> allow timeAllowed and cursorMark parameters
> -------------------------------------------
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Components: search
> Reporter: John Gallagher
> Priority: Minor
> Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch,
> SOLR-14413.patch, image-2020-08-18-16-56-41-736.png,
> image-2020-08-18-16-56-59-178.png, timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
