Re: behavior of scrolling search with "from" parameter in newer ES versions
Thank you Martijn!
On Wednesday, January 21, 2015 at 8:04:23 AM UTC-6, Martijn v Groningen
wrote:
>
> I agree, this should be fixed:
> https://github.com/elasticsearch/elasticsearch/issues/9373
>
> On 21 January 2015 at 14:27, Gordon Tillman > wrote:
>
>> Thanks Martijn,
>>
>> That is what I have observed. But is is a regression from ES version
>> 1.0.1 and before. And I can't find anything that even implies that `from`
>> is not supported for a scroll search, except of course when
>> `search_type=scan`.
>>
>> I would love to get support for that back into the product.
>>
>> --g
>>
>>
>>
>> On Wednesday, January 21, 2015 at 2:57:29 AM UTC-6, Martijn v Groningen
>> wrote:
>>>
>>> Hi Gordon,
>>>
>>> This `from` is kind of ignored for scroll search. I don't remember why
>>> that was the case, but it seems to me that scroll search can/should take
>>> into account the `from` option during the first scroll search request.
>>>
>>> Martijn
>>>
>>> On 20 January 2015 at 23:33, Gordon Tillman wrote:
>>>
Greetings All,
I ran into an interesting issue when upgrading from ES version 1.0.1 to
newer versions. In particular, I tested the following with versions
1.2.4,
1.3.4 and 1.4.2.
*summary*
When doing a normal scroll search (not one with a *search_type=scan*),
it appears that the "from" parameter is being ignored.
*example 1 (no scrolling, result is correct)*
In this simplified example, only one document matches the supplied
query, so with *from=1* no documents are returned.
curl localhost:9200/hm-community-alias/FileInfo/_search?q=parents:
c10ed0583104036a94e110f0a8b5fd7d4\&from=1
*example 2 (with scrolling, incorrect result)*
In this example, where we specify the same query and from parameters,
but also specify a scroll parameter, we incorrectly get the single
document
returned from the query.
curl localhost:9200/hm-community-alias/FileInfo/_search?q=parents:
c10ed0583104036a94e110f0a8b5fd7d4\&from=1\&scroll=2s | json
*notes*
1. Both of the above test cases work correctly in version 1.0.1
2. In newer versions (where example 2 fails), I noticed that the
*from* value is not present in the data that was returned from the
query.
I understand with with a *search_type=scan, *this would be expected
because sorting is disabled. Also, please note this is a greatly
simplified query just for illustration purposes. I've attached a sample
of
an actual query at the bottom of this post, and *it does contain
sorting specifications*.
Is this a known issue or is this being done by design in the newer ES
versions. A quick scan through the release notes was unrevealing.
Many thanks for any insight!
--g
*sample of full query*
{
"sort": [
{
"_type": {
"order": "asc",
"ignore_unmapped": true
}
},
{
"name_lower": {
"order": "asc",
"ignore_unmapped": true
}
},
{
"dds_key": {
"order": "asc",
"ignore_unmapped": true
}
}
],
"query": {
"filtered": {
"filter": {
"and": [
{
"or": [
{
"term": {
"parent": "
c10ed0583104012f94e11ad0ac36f2aaf"
}
}
]
},
{
"not": {
"term": {
"vcn": "DeleteMarker"
}
}
},
{
"not": {
"exists": {
"field": "notfinalized"
}
}
}
]
},
"query": {
"match_all": {}
}
}
},
"from": 1,
"size": 1000
}
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on t
Re: behavior of scrolling search with "from" parameter in newer ES versions
I agree, this should be fixed:
https://github.com/elasticsearch/elasticsearch/issues/9373
On 21 January 2015 at 14:27, Gordon Tillman wrote:
> Thanks Martijn,
>
> That is what I have observed. But is is a regression from ES version
> 1.0.1 and before. And I can't find anything that even implies that `from`
> is not supported for a scroll search, except of course when
> `search_type=scan`.
>
> I would love to get support for that back into the product.
>
> --g
>
>
>
> On Wednesday, January 21, 2015 at 2:57:29 AM UTC-6, Martijn v Groningen
> wrote:
>>
>> Hi Gordon,
>>
>> This `from` is kind of ignored for scroll search. I don't remember why
>> that was the case, but it seems to me that scroll search can/should take
>> into account the `from` option during the first scroll search request.
>>
>> Martijn
>>
>> On 20 January 2015 at 23:33, Gordon Tillman wrote:
>>
>>> Greetings All,
>>>
>>> I ran into an interesting issue when upgrading from ES version 1.0.1 to
>>> newer versions. In particular, I tested the following with versions 1.2.4,
>>> 1.3.4 and 1.4.2.
>>>
>>> *summary*
>>>
>>> When doing a normal scroll search (not one with a *search_type=scan*),
>>> it appears that the "from" parameter is being ignored.
>>>
>>>
>>> *example 1 (no scrolling, result is correct)*
>>>
>>> In this simplified example, only one document matches the supplied
>>> query, so with *from=1* no documents are returned.
>>>
>>> curl localhost:9200/hm-community-alias/FileInfo/_search?q=parents:
>>> c10ed0583104036a94e110f0a8b5fd7d4\&from=1
>>>
>>> *example 2 (with scrolling, incorrect result)*
>>>
>>> In this example, where we specify the same query and from parameters,
>>> but also specify a scroll parameter, we incorrectly get the single document
>>> returned from the query.
>>>
>>> curl localhost:9200/hm-community-alias/FileInfo/_search?q=parents:
>>> c10ed0583104036a94e110f0a8b5fd7d4\&from=1\&scroll=2s | json
>>>
>>> *notes*
>>>
>>>
>>>1. Both of the above test cases work correctly in version 1.0.1
>>>2. In newer versions (where example 2 fails), I noticed that the
>>>*from* value is not present in the data that was returned from the
>>>query.
>>>
>>>
>>> I understand with with a *search_type=scan, *this would be expected
>>> because sorting is disabled. Also, please note this is a greatly
>>> simplified query just for illustration purposes. I've attached a sample of
>>> an actual query at the bottom of this post, and *it does contain
>>> sorting specifications*.
>>>
>>> Is this a known issue or is this being done by design in the newer ES
>>> versions. A quick scan through the release notes was unrevealing.
>>>
>>> Many thanks for any insight!
>>>
>>> --g
>>>
>>> *sample of full query*
>>>
>>>
>>> {
>>> "sort": [
>>> {
>>> "_type": {
>>> "order": "asc",
>>> "ignore_unmapped": true
>>> }
>>> },
>>> {
>>> "name_lower": {
>>> "order": "asc",
>>> "ignore_unmapped": true
>>> }
>>> },
>>> {
>>> "dds_key": {
>>> "order": "asc",
>>> "ignore_unmapped": true
>>> }
>>> }
>>> ],
>>> "query": {
>>> "filtered": {
>>> "filter": {
>>> "and": [
>>> {
>>> "or": [
>>> {
>>> "term": {
>>> "parent": "
>>> c10ed0583104012f94e11ad0ac36f2aaf"
>>> }
>>> }
>>> ]
>>> },
>>> {
>>> "not": {
>>> "term": {
>>> "vcn": "DeleteMarker"
>>> }
>>> }
>>> },
>>> {
>>> "not": {
>>> "exists": {
>>> "field": "notfinalized"
>>> }
>>> }
>>> }
>>> ]
>>> },
>>> "query": {
>>> "match_all": {}
>>> }
>>> }
>>> },
>>> "from": 1,
>>> "size": 1000
>>> }
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/elasticsearch/ce8145ac-59c6-4a22-b4c4-eacb90281c4d%
>>> 40googlegroups.com
>>>
>>> .
>>> For more options, visit https:
Re: behavior of scrolling search with "from" parameter in newer ES versions
Thanks Martijn,
That is what I have observed. But is is a regression from ES version 1.0.1
and before. And I can't find anything that even implies that `from` is not
supported for a scroll search, except of course when `search_type=scan`.
I would love to get support for that back into the product.
--g
On Wednesday, January 21, 2015 at 2:57:29 AM UTC-6, Martijn v Groningen
wrote:
>
> Hi Gordon,
>
> This `from` is kind of ignored for scroll search. I don't remember why
> that was the case, but it seems to me that scroll search can/should take
> into account the `from` option during the first scroll search request.
>
> Martijn
>
> On 20 January 2015 at 23:33, Gordon Tillman > wrote:
>
>> Greetings All,
>>
>> I ran into an interesting issue when upgrading from ES version 1.0.1 to
>> newer versions. In particular, I tested the following with versions 1.2.4,
>> 1.3.4 and 1.4.2.
>>
>> *summary*
>>
>> When doing a normal scroll search (not one with a *search_type=scan*),
>> it appears that the "from" parameter is being ignored.
>>
>>
>> *example 1 (no scrolling, result is correct)*
>>
>> In this simplified example, only one document matches the supplied query,
>> so with *from=1* no documents are returned.
>>
>> curl
>> localhost:9200/hm-community-alias/FileInfo/_search?q=parents:c10ed0583104036a94e110f0a8b5fd7d4\&from=1
>>
>> *example 2 (with scrolling, incorrect result)*
>>
>> In this example, where we specify the same query and from parameters, but
>> also specify a scroll parameter, we incorrectly get the single document
>> returned from the query.
>>
>> curl
>> localhost:9200/hm-community-alias/FileInfo/_search?q=parents:c10ed0583104036a94e110f0a8b5fd7d4\&from=1\&scroll=2s
>>
>> | json
>>
>> *notes*
>>
>>
>>1. Both of the above test cases work correctly in version 1.0.1
>>2. In newer versions (where example 2 fails), I noticed that the
>>*from* value is not present in the data that was returned from the
>>query.
>>
>>
>> I understand with with a *search_type=scan, *this would be expected
>> because sorting is disabled. Also, please note this is a greatly
>> simplified query just for illustration purposes. I've attached a sample of
>> an actual query at the bottom of this post, and *it does contain sorting
>> specifications*.
>>
>> Is this a known issue or is this being done by design in the newer ES
>> versions. A quick scan through the release notes was unrevealing.
>>
>> Many thanks for any insight!
>>
>> --g
>>
>> *sample of full query*
>>
>>
>> {
>> "sort": [
>> {
>> "_type": {
>> "order": "asc",
>> "ignore_unmapped": true
>> }
>> },
>> {
>> "name_lower": {
>> "order": "asc",
>> "ignore_unmapped": true
>> }
>> },
>> {
>> "dds_key": {
>> "order": "asc",
>> "ignore_unmapped": true
>> }
>> }
>> ],
>> "query": {
>> "filtered": {
>> "filter": {
>> "and": [
>> {
>> "or": [
>> {
>> "term": {
>> "parent":
>> "c10ed0583104012f94e11ad0ac36f2aaf"
>> }
>> }
>> ]
>> },
>> {
>> "not": {
>> "term": {
>> "vcn": "DeleteMarker"
>> }
>> }
>> },
>> {
>> "not": {
>> "exists": {
>> "field": "notfinalized"
>> }
>> }
>> }
>> ]
>> },
>> "query": {
>> "match_all": {}
>> }
>> }
>> },
>> "from": 1,
>> "size": 1000
>> }
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/ce8145ac-59c6-4a22-b4c4-eacb90281c4d%40googlegroups.com
>>
>>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Met vriendelijke groet,
>
> Martijn van Groningen
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emai
Re: behavior of scrolling search with "from" parameter in newer ES versions
Hi Gordon,
This `from` is kind of ignored for scroll search. I don't remember why that
was the case, but it seems to me that scroll search can/should take into
account the `from` option during the first scroll search request.
Martijn
On 20 January 2015 at 23:33, Gordon Tillman wrote:
> Greetings All,
>
> I ran into an interesting issue when upgrading from ES version 1.0.1 to
> newer versions. In particular, I tested the following with versions 1.2.4,
> 1.3.4 and 1.4.2.
>
> *summary*
>
> When doing a normal scroll search (not one with a *search_type=scan*), it
> appears that the "from" parameter is being ignored.
>
>
> *example 1 (no scrolling, result is correct)*
>
> In this simplified example, only one document matches the supplied query,
> so with *from=1* no documents are returned.
>
> curl
> localhost:9200/hm-community-alias/FileInfo/_search?q=parents:c10ed0583104036a94e110f0a8b5fd7d4\&from=1
>
> *example 2 (with scrolling, incorrect result)*
>
> In this example, where we specify the same query and from parameters, but
> also specify a scroll parameter, we incorrectly get the single document
> returned from the query.
>
> curl
> localhost:9200/hm-community-alias/FileInfo/_search?q=parents:c10ed0583104036a94e110f0a8b5fd7d4\&from=1\&scroll=2s
> | json
>
> *notes*
>
>
>1. Both of the above test cases work correctly in version 1.0.1
>2. In newer versions (where example 2 fails), I noticed that the *from*
>value is not present in the data that was returned from the query.
>
>
> I understand with with a *search_type=scan, *this would be expected
> because sorting is disabled. Also, please note this is a greatly
> simplified query just for illustration purposes. I've attached a sample of
> an actual query at the bottom of this post, and *it does contain sorting
> specifications*.
>
> Is this a known issue or is this being done by design in the newer ES
> versions. A quick scan through the release notes was unrevealing.
>
> Many thanks for any insight!
>
> --g
>
> *sample of full query*
>
>
> {
> "sort": [
> {
> "_type": {
> "order": "asc",
> "ignore_unmapped": true
> }
> },
> {
> "name_lower": {
> "order": "asc",
> "ignore_unmapped": true
> }
> },
> {
> "dds_key": {
> "order": "asc",
> "ignore_unmapped": true
> }
> }
> ],
> "query": {
> "filtered": {
> "filter": {
> "and": [
> {
> "or": [
> {
> "term": {
> "parent":
> "c10ed0583104012f94e11ad0ac36f2aaf"
> }
> }
> ]
> },
> {
> "not": {
> "term": {
> "vcn": "DeleteMarker"
> }
> }
> },
> {
> "not": {
> "exists": {
> "field": "notfinalized"
> }
> }
> }
> ]
> },
> "query": {
> "match_all": {}
> }
> }
> },
> "from": 1,
> "size": 1000
> }
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/ce8145ac-59c6-4a22-b4c4-eacb90281c4d%40googlegroups.com
>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
Met vriendelijke groet,
Martijn van Groningen
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76Tw7_MvnXLAD_4LrQUHvxasGkEUL%2BQRBbVv_UDU3UAq1vg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
behavior of scrolling search with "from" parameter in newer ES versions
Greetings All,
I ran into an interesting issue when upgrading from ES version 1.0.1 to
newer versions. In particular, I tested the following with versions 1.2.4,
1.3.4 and 1.4.2.
*summary*
When doing a normal scroll search (not one with a *search_type=scan*), it
appears that the "from" parameter is being ignored.
*example 1 (no scrolling, result is correct)*
In this simplified example, only one document matches the supplied query,
so with *from=1* no documents are returned.
curl
localhost:9200/hm-community-alias/FileInfo/_search?q=parents:c10ed0583104036a94e110f0a8b5fd7d4\&from=1
*example 2 (with scrolling, incorrect result)*
In this example, where we specify the same query and from parameters, but
also specify a scroll parameter, we incorrectly get the single document
returned from the query.
curl
localhost:9200/hm-community-alias/FileInfo/_search?q=parents:c10ed0583104036a94e110f0a8b5fd7d4\&from=1\&scroll=2s
| json
*notes*
1. Both of the above test cases work correctly in version 1.0.1
2. In newer versions (where example 2 fails), I noticed that the *from*
value is not present in the data that was returned from the query.
I understand with with a *search_type=scan, *this would be expected because
sorting is disabled. Also, please note this is a greatly simplified query
just for illustration purposes. I've attached a sample of an actual query
at the bottom of this post, and *it does contain sorting specifications*.
Is this a known issue or is this being done by design in the newer ES
versions. A quick scan through the release notes was unrevealing.
Many thanks for any insight!
--g
*sample of full query*
{
"sort": [
{
"_type": {
"order": "asc",
"ignore_unmapped": true
}
},
{
"name_lower": {
"order": "asc",
"ignore_unmapped": true
}
},
{
"dds_key": {
"order": "asc",
"ignore_unmapped": true
}
}
],
"query": {
"filtered": {
"filter": {
"and": [
{
"or": [
{
"term": {
"parent":
"c10ed0583104012f94e11ad0ac36f2aaf"
}
}
]
},
{
"not": {
"term": {
"vcn": "DeleteMarker"
}
}
},
{
"not": {
"exists": {
"field": "notfinalized"
}
}
}
]
},
"query": {
"match_all": {}
}
}
},
"from": 1,
"size": 1000
}
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ce8145ac-59c6-4a22-b4c4-eacb90281c4d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
