[
https://issues.apache.org/jira/browse/SOLR-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090169#comment-17090169
]
John Gallagher commented on SOLR-14413:
---------------------------------------
Hi [~mdrob]. I've attached a session that I ran where I created 100 docs,
restarted solr, and used a short timeAllowed=1 to get partial results. In my
tests, the nextCursorMark works as you would want: it returns a nextCursorMark
value corresponding to how far it got in the partial results (even of it
returns 0 results, it will just return that cursor value again)
The session has the commands I ran and the comments, but the TL;DR is here:
I created 100 docs, from id:100 to id:199. I then restarted solr so that it
would run my query cold and slowly:
{code:java}
# try to run the query and show all the rows:
[johng solr (SOLR-14413)]$ curl -sq
'localhost:8983/solr/test_timeallowed_cursormark/query?q=*:*&cursorMark=*&timeAllowed=1&sort=id+asc&rows=100'
{
"responseHeader":{
"zkConnected":true,
"partialResults":true,
"status":0,
"QTime":7,
"params":{
"q":"*:*",
"cursorMark":"*",
"timeAllowed":"1",
"sort":"id asc",
"rows":"100"}},
"response":{"numFound":45,"start":0,"docs":[
{
"id":"100",
"_version_":1664726406826819584},
{
"id":"101",
"_version_":1664726406874005504},
{
"id":"102",
"_version_":1664726406902317056},
{
"id":"103",
"_version_":1664726406926434304},
{
"id":"104",
"_version_":1664726406952648704},
{
"id":"105",
"_version_":1664726406977814528},
{
"id":"106",
"_version_":1664726407000883200},
{
"id":"107",
"_version_":1664726407023951872},
{
"id":"108",
"_version_":1664726407047020544},
{
"id":"109",
"_version_":1664726407070089216},
{
"id":"110",
"_version_":1664726407094206464},
{
"id":"111",
"_version_":1664726407119372288},
{
"id":"112",
"_version_":1664726407142440960},
{
"id":"113",
"_version_":1664726407165509632},
{
"id":"114",
"_version_":1664726407188578304},
{
"id":"115",
"_version_":1664726407211646976},
{
"id":"116",
"_version_":1664726407235764224},
{
"id":"117",
"_version_":1664726407258832896},
{
"id":"118",
"_version_":1664726407280852992},
{
"id":"119",
"_version_":1664726407303921664},
{
"id":"120",
"_version_":1664726407328038912},
{
"id":"121",
"_version_":1664726407355301888},
{
"id":"122",
"_version_":1664726407378370560},
{
"id":"123",
"_version_":1664726407401439232},
{
"id":"124",
"_version_":1664726407425556480},
{
"id":"125",
"_version_":1664726407449673728},
{
"id":"126",
"_version_":1664726407474839552},
{
"id":"127",
"_version_":1664726407500005376},
{
"id":"128",
"_version_":1664726407522025472},
{
"id":"129",
"_version_":1664726407549288448},
{
"id":"130",
"_version_":1664726407572357120},
{
"id":"131",
"_version_":1664726407596474368},
{
"id":"132",
"_version_":1664726407619543040},
{
"id":"133",
"_version_":1664726407643660288},
{
"id":"134",
"_version_":1664726407668826112},
{
"id":"135",
"_version_":1664726407691894784},
{
"id":"136",
"_version_":1664726407716012032},
{
"id":"137",
"_version_":1664726407739080704},
{
"id":"138",
"_version_":1664726407760052224},
{
"id":"139",
"_version_":1664726407783120896},
{
"id":"140",
"_version_":1664726407805140992},
{
"id":"141",
"_version_":1664726407830306816},
{
"id":"142",
"_version_":1664726407854424064},
{
"id":"143",
"_version_":1664726407877492736}]
},
"nextCursorMark":"AoEjMTQz"}
# lets decode that cursormark - is it pointing at id:143 ?? YES!
[johng solr (SOLR-14413)]$ python3 -c 'import base64;
print(base64.b64decode("AoEjMTQz"))'
b'\x02\x81#143'
# using it as the next cursormark works as expected:
[johng solr (SOLR-14413)]$ curl -sq
'localhost:8983/solr/test_timeallowed_cursormark/query?q=*:*&cursorMark=AoEjMTQz&timeAllowed=1&sort=id+asc&rows=100'
{
"responseHeader":{
"zkConnected":true,
"partialResults":true,
"status":0,
"QTime":6,
"params":{
"q":"*:*",
"cursorMark":"AoEjMTQz",
"timeAllowed":"1",
"sort":"id asc",
"rows":"100"}},
"response":{"numFound":81,"start":0,"docs":[
{
"id":"144",
"_version_":1664726407908950016},
....ETC.....{code}
[^timeallowed_cursormarks_results.txt]
> allow timeAllowed and cursorMark parameters
> -------------------------------------------
>
> Key: SOLR-14413
> URL: https://issues.apache.org/jira/browse/SOLR-14413
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
> Reporter: John Gallagher
> Priority: Minor
> Attachments: SOLR-14413.patch, timeallowed_cursormarks_results.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and
> timeAllowed parameters were not allowed in combination ("Can not search using
> both cursorMark and timeAllowed")
> , from [QueryComponent.java|#L359]]:
>
> {code:java}
>
> if (null != rb.getCursorMark() && 0 < timeAllowed) {
> // fundamentally incompatible
> throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not
> search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " +
> CommonParams.TIME_ALLOWED);
> } {code}
> While theoretically impure to use them in combination, it is often desirable
> to support cursormarks-style deep paging and attempt to protect Solr nodes
> from runaway queries using timeAllowed, in the hopes that most of the time,
> the query completes in the allotted time, and there is no conflict.
>
> However if the query takes too long, it may be preferable to end the query
> and protect the Solr node and provide the user with a somewhat inaccurate
> sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is
> frequently used to prevent runaway load. In fact, cursorMark and
> shards.tolerant are allowed in combination, so any argument in favor of
> purity would be a bit muddied in my opinion.
>
> This was discussed once in the mailing list that I can find:
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%[email protected]%3E]
> It did not look like there was strong support for preventing the combination.
>
> I have tested cursorMark and timeAllowed combination together, and even when
> partial results are returned because the timeAllowed is exceeded, the
> cursorMark response value is still valid and reasonable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
