[ https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180102#comment-17180102 ]
John Gallagher commented on SOLR-14413: --------------------------------------- Hi folks, sorry I stepped away from this for a bit, and thanks [~mdrob] , and [~epugh] for the comments, and [~bvd] for contributing! To Mike's earlier point: when partialResults is true, you cannot be sure whether there are more results or not. You could summarize it this way: || ||partialResults not present|| partialResults: true|| |more results?|nextCursorMark != cursorMark|unknown| After figuring out how to properly generate documentation, I took a stab at adding two notes to the documentation: 1. In timeAllowed documentation, common-query-parameters.html#timeallowed-parameter !image-2020-08-18-16-56-59-178.png|width=508,height=116! 2. In the Constraints when using Cursors documentation, pagination-of-results.html#constraints-when-using-cursors !image-2020-08-18-16-56-41-736.png|width=410,height=250! Please let me know what you think of these additions. I was also able to reproduce [~bvd]'s case where nextCursorMark was null (the field actually wasn't present in the response at all). I tracked it down to [this code in SearchHandler.java|[https://github.com/slackhappy/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L349]]. When the Search times out very early, an exception is thrown before responses can be handled, which skips the calculation of the next cursor mark. In that case, the response is null, and the SearchHandler creates an empty response to return (numFound 0, docs:[]). I added code to also return the original cursorMark in that case, since we haven't progressed (it is the value you would want to pass to a subsequent search). I added a basic test to CursorPagingTest that checks that the parameters can be used in conjunction, and that the nextCursorMark returned is a valid one that can be used in subsequent requests. Perhaps that test can be combined with [~bvd]'s. I struggled a bit to come up with a test that would produce reliable results when using timeAllowed. I borrowed a bit from ExitableDirectoryReaderTest's use of a DelayingSearchComponent, which helped somewhat. The updated patch file is SOLR-14413-jg-update1.patch, and I have updated the [corresponding PR|[https://github.com/apache/lucene-solr/pull/1436]]. John > allow timeAllowed and cursorMark parameters > ------------------------------------------- > > Key: SOLR-14413 > URL: https://issues.apache.org/jira/browse/SOLR-14413 > Project: Solr > Issue Type: Improvement > Components: search > Reporter: John Gallagher > Priority: Minor > Attachments: SOLR-14413-bram.patch, SOLR-14413-jg-update1.patch, > SOLR-14413.patch, image-2020-08-18-16-56-41-736.png, > image-2020-08-18-16-56-59-178.png, timeallowed_cursormarks_results.txt > > Time Spent: 10m > Remaining Estimate: 0h > > Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and > timeAllowed parameters were not allowed in combination ("Can not search using > both cursorMark and timeAllowed") > , from [QueryComponent.java|#L359]]: > > {code:java} > > if (null != rb.getCursorMark() && 0 < timeAllowed) { > // fundamentally incompatible > throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not > search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " + > CommonParams.TIME_ALLOWED); > } {code} > While theoretically impure to use them in combination, it is often desirable > to support cursormarks-style deep paging and attempt to protect Solr nodes > from runaway queries using timeAllowed, in the hopes that most of the time, > the query completes in the allotted time, and there is no conflict. > > However if the query takes too long, it may be preferable to end the query > and protect the Solr node and provide the user with a somewhat inaccurate > sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is > frequently used to prevent runaway load. In fact, cursorMark and > shards.tolerant are allowed in combination, so any argument in favor of > purity would be a bit muddied in my opinion. > > This was discussed once in the mailing list that I can find: > [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3c5591740b.4080...@elyograg.org%3E] > It did not look like there was strong support for preventing the combination. > > I have tested cursorMark and timeAllowed combination together, and even when > partial results are returned because the timeAllowed is exceeded, the > cursorMark response value is still valid and reasonable. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org