Re: exists filter broken on 1.5.0 with restored index?
Seems like it also impacts "missing". Just checking that the patch will
address that too? Looks like it will from the file changes in the patch
code, but thought I'd better mention it, just in case! :)
On Thursday, 26 March 2015 18:31:44 UTC, Mads Martin Jørgensen wrote:
>
> Thanks for fixing!
>
> On Thursday, March 26, 2015 at 3:29:17 PM UTC+1, Igor Motov wrote:
>>
>> Thanks for checking. It's a bug, which should be fixed in 1.5.1
>> https://github.com/elastic/elasticsearch/pull/10268
>>
>> On Wednesday, 25 March 2015 13:43:28 UTC-4, Mads Martin Jørgensen wrote:
>>>
>>> They're similar. The 1.5.0 cluster has "created" : "1000199", and the
>>> 1.4.1 cluster also has "created" : "1000199"
>>>
>>> On Wednesday, March 25, 2015 at 4:45:30 PM UTC+1, Igor Motov wrote:
>>>>
>>>> Hi Mads Martin,
>>>>
>>>> Could you check the version that is returned when you run curl
>>>> "localhost:9200/my_own_index/_settings?pretty". The version will be in
>>>>
>>>> "version" : {
>>>> "created" : "XXX"
>>>> }
>>>>
>>>> Could you compare it to the version that is returned by the same index
>>>> in the pre-1.5.0 cluster?
>>>>
>>>> Igor
>>>>
>>>> On Wednesday, 25 March 2015 09:27:03 UTC-4, Mads Martin Jørgensen wrote:
>>>>>
>>>>> Hello all,
>>>>>
>>>>> Just installed es-1.5.0 with cloud-aws-2.5.0 on a machine. Did a
>>>>> restore of a snapshot made with es-1.4.1. All documents are there, but
>>>>> the
>>>>> exists filter seems broken. The query that used to return all documents
>>>>> matching, now return 0 documents, even though the field exists when
>>>>> reading
>>>>> the documents.
>>>>>
>>>>> curl -XGET "http://localhost:9200/my_own_index/document/_search
>>>>> <http://www.google.com/url?q=http%3A%2F%2Fcarter%3A9200%2Fskyfish_v2%2Fdocument%2F_search&sa=D&sntz=1&usg=AFQjCNHx9tllVr6Ip2n5zS9uz9FNweGcWg>"
>>>>>
>>>>> -d'
>>>>>
>>>>> {
>>>>>
>>>>>"query": {
>>>>>
>>>>> "constant_score": {
>>>>>
>>>>> "filter": {
>>>>>
>>>>> "exists": {
>>>>>
>>>>>"field": "history"
>>>>>
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>>}
>>>>>
>>>>> }'
>>>>>
>>>>>
>>>>> If we populate new documents, then the exists filter works just fine.
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Mads Martin
>>>>>
>>>>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4f4e4538-37e0-4da5-8761-1a65fbd74957%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: exists filter broken on 1.5.0 with restored index?
Thanks for fixing!
On Thursday, March 26, 2015 at 3:29:17 PM UTC+1, Igor Motov wrote:
>
> Thanks for checking. It's a bug, which should be fixed in 1.5.1
> https://github.com/elastic/elasticsearch/pull/10268
>
> On Wednesday, 25 March 2015 13:43:28 UTC-4, Mads Martin Jørgensen wrote:
>>
>> They're similar. The 1.5.0 cluster has "created" : "1000199", and the
>> 1.4.1 cluster also has "created" : "1000199"
>>
>> On Wednesday, March 25, 2015 at 4:45:30 PM UTC+1, Igor Motov wrote:
>>>
>>> Hi Mads Martin,
>>>
>>> Could you check the version that is returned when you run curl
>>> "localhost:9200/my_own_index/_settings?pretty". The version will be in
>>>
>>> "version" : {
>>> "created" : "XXX"
>>> }
>>>
>>> Could you compare it to the version that is returned by the same index
>>> in the pre-1.5.0 cluster?
>>>
>>> Igor
>>>
>>> On Wednesday, 25 March 2015 09:27:03 UTC-4, Mads Martin Jørgensen wrote:
>>>>
>>>> Hello all,
>>>>
>>>> Just installed es-1.5.0 with cloud-aws-2.5.0 on a machine. Did a
>>>> restore of a snapshot made with es-1.4.1. All documents are there, but the
>>>> exists filter seems broken. The query that used to return all documents
>>>> matching, now return 0 documents, even though the field exists when
>>>> reading
>>>> the documents.
>>>>
>>>> curl -XGET "http://localhost:9200/my_own_index/document/_search
>>>> <http://www.google.com/url?q=http%3A%2F%2Fcarter%3A9200%2Fskyfish_v2%2Fdocument%2F_search&sa=D&sntz=1&usg=AFQjCNHx9tllVr6Ip2n5zS9uz9FNweGcWg>"
>>>>
>>>> -d'
>>>>
>>>> {
>>>>
>>>>"query": {
>>>>
>>>> "constant_score": {
>>>>
>>>> "filter": {
>>>>
>>>> "exists": {
>>>>
>>>>"field": "history"
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>}
>>>>
>>>> }'
>>>>
>>>>
>>>> If we populate new documents, then the exists filter works just fine.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Mads Martin
>>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5ef4c402-3609-4780-bc64-049ea5c76751%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: exists filter broken on 1.5.0 with restored index?
Thanks for checking. It's a bug, which should be fixed in
1.5.1 https://github.com/elastic/elasticsearch/pull/10268
On Wednesday, 25 March 2015 13:43:28 UTC-4, Mads Martin Jørgensen wrote:
>
> They're similar. The 1.5.0 cluster has "created" : "1000199", and the
> 1.4.1 cluster also has "created" : "1000199"
>
> On Wednesday, March 25, 2015 at 4:45:30 PM UTC+1, Igor Motov wrote:
>>
>> Hi Mads Martin,
>>
>> Could you check the version that is returned when you run curl
>> "localhost:9200/my_own_index/_settings?pretty". The version will be in
>>
>> "version" : {
>> "created" : "XXX"
>> }
>>
>> Could you compare it to the version that is returned by the same index in
>> the pre-1.5.0 cluster?
>>
>> Igor
>>
>> On Wednesday, 25 March 2015 09:27:03 UTC-4, Mads Martin Jørgensen wrote:
>>>
>>> Hello all,
>>>
>>> Just installed es-1.5.0 with cloud-aws-2.5.0 on a machine. Did a restore
>>> of a snapshot made with es-1.4.1. All documents are there, but the exists
>>> filter seems broken. The query that used to return all documents matching,
>>> now return 0 documents, even though the field exists when reading the
>>> documents.
>>>
>>> curl -XGET "http://localhost:9200/my_own_index/document/_search
>>> <http://www.google.com/url?q=http%3A%2F%2Fcarter%3A9200%2Fskyfish_v2%2Fdocument%2F_search&sa=D&sntz=1&usg=AFQjCNHx9tllVr6Ip2n5zS9uz9FNweGcWg>"
>>>
>>> -d'
>>>
>>> {
>>>
>>>"query": {
>>>
>>> "constant_score": {
>>>
>>> "filter": {
>>>
>>> "exists": {
>>>
>>>"field": "history"
>>>
>>> }
>>>
>>> }
>>>
>>> }
>>>
>>>}
>>>
>>> }'
>>>
>>>
>>> If we populate new documents, then the exists filter works just fine.
>>>
>>>
>>> Regards,
>>>
>>> Mads Martin
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/531c6e3c-5697-4fc0-b037-d8fbc438c5dd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: exists filter broken on 1.5.0 with restored index?
They're similar. The 1.5.0 cluster has "created" : "1000199", and the 1.4.1
cluster also has "created" : "1000199"
On Wednesday, March 25, 2015 at 4:45:30 PM UTC+1, Igor Motov wrote:
>
> Hi Mads Martin,
>
> Could you check the version that is returned when you run curl
> "localhost:9200/my_own_index/_settings?pretty". The version will be in
>
> "version" : {
> "created" : "XXX"
> }
>
> Could you compare it to the version that is returned by the same index in
> the pre-1.5.0 cluster?
>
> Igor
>
> On Wednesday, 25 March 2015 09:27:03 UTC-4, Mads Martin Jørgensen wrote:
>>
>> Hello all,
>>
>> Just installed es-1.5.0 with cloud-aws-2.5.0 on a machine. Did a restore
>> of a snapshot made with es-1.4.1. All documents are there, but the exists
>> filter seems broken. The query that used to return all documents matching,
>> now return 0 documents, even though the field exists when reading the
>> documents.
>>
>> curl -XGET "http://localhost:9200/my_own_index/document/_search
>> <http://www.google.com/url?q=http%3A%2F%2Fcarter%3A9200%2Fskyfish_v2%2Fdocument%2F_search&sa=D&sntz=1&usg=AFQjCNHx9tllVr6Ip2n5zS9uz9FNweGcWg>"
>>
>> -d'
>>
>> {
>>
>> "query": {
>>
>> "constant_score": {
>>
>> "filter": {
>>
>> "exists": {
>>
>>"field": "history"
>>
>> }
>>
>> }
>>
>> }
>>
>>}
>>
>> }'
>>
>>
>> If we populate new documents, then the exists filter works just fine.
>>
>>
>> Regards,
>>
>> Mads Martin
>>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ff38b5a8-ebab-4061-a963-5e74a807de3e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: exists filter broken on 1.5.0 with restored index?
Hi Mads Martin,
Could you check the version that is returned when you run curl
"localhost:9200/my_own_index/_settings?pretty". The version will be in
"version" : {
"created" : "XXX"
}
Could you compare it to the version that is returned by the same index in
the pre-1.5.0 cluster?
Igor
On Wednesday, 25 March 2015 09:27:03 UTC-4, Mads Martin Jørgensen wrote:
>
> Hello all,
>
> Just installed es-1.5.0 with cloud-aws-2.5.0 on a machine. Did a restore
> of a snapshot made with es-1.4.1. All documents are there, but the exists
> filter seems broken. The query that used to return all documents matching,
> now return 0 documents, even though the field exists when reading the
> documents.
>
> curl -XGET "http://localhost:9200/my_own_index/document/_search
> <http://www.google.com/url?q=http%3A%2F%2Fcarter%3A9200%2Fskyfish_v2%2Fdocument%2F_search&sa=D&sntz=1&usg=AFQjCNHx9tllVr6Ip2n5zS9uz9FNweGcWg>"
>
> -d'
>
> {
>
>"query": {
>
> "constant_score": {
>
> "filter": {
>
> "exists": {
>
>"field": "history"
>
> }
>
> }
>
> }
>
>}
>
> }'
>
>
> If we populate new documents, then the exists filter works just fine.
>
>
> Regards,
>
> Mads Martin
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3937725f-6381-442b-9e2b-638913514d3b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
exists filter broken on 1.5.0 with restored index?
Hello all,
Just installed es-1.5.0 with cloud-aws-2.5.0 on a machine. Did a restore of
a snapshot made with es-1.4.1. All documents are there, but the exists
filter seems broken. The query that used to return all documents matching,
now return 0 documents, even though the field exists when reading the
documents.
curl -XGET "http://localhost:9200/my_own_index/document/_search
<http://carter:9200/skyfish_v2/document/_search>" -d'
{
"query": {
"constant_score": {
"filter": {
"exists": {
"field": "history"
}
}
}
}
}'
If we populate new documents, then the exists filter works just fine.
Regards,
Mads Martin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4d7d4080-c997-412e-85cd-04d7aeda0e27%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Exists filter on fields with index_name not getting all documents
Hi guys,
How to reproduce: https://gist.github.com/rmat0n/7b6f4f0398dc5f0966b4
I created 2 mapping with a testtext field on both, the difference is that
each mapping have a different index_name for the field: the first mapping
use 1.testtext and the second mapping use 2.testtext.
Then putting 1 object on each mapping. Here I have 2 objects with the same
field test text but on different mappings.
Now I query using the exists filter *(also tested using the _exists_ syntax
in the query_string)* on the field name:
{
"filter": {
"exists": {
"field" : "testtext"
}
}
}
This query only returns the document added to the first mapping and not the
document added to the second mapping.
I expected to get back the 2 objects because both have the testtext field.
Reading the documentation
<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-object-type.html#_path_3>:
*there is an automatic detection from logical full path and into the
**index_name** and vice versa*.
So, even if I don't query the index_name, I should be able to get it by the
field name *(path)*.
I could have expected to get both or none but only the first seems weird. Maybe
I am missing something.
Thanks in advance.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/09ea42ae-3d7c-4306-a334-cd5ee4171ad4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Exists filter does not respect must_not bool filter
>
>
>
> Appreciate your explanation, and as per your suggestion range filter gives
> correct results.
> I am still confused with the usage of exists filter.
>
> As per my understanding the implementation of exists filter is changed in
> v1.3 to increase the speed but why it deviates from it's expected behavior.
>
No doubt exists filter might be fast or optimized but it is chopping off
more than half of the results.
>
> For Example:
>
>I want to find out number of documents where "giving.assignee" field
exists.
Note: "giving.assignee" is a string analyzed field.
>
>
> {
> "query": {
> "filtered": {
> "filter": {
> "exists": {
> "field": "giving.assignee"
> }
> }
> }
> },
> "size": 2000
> }
>
>
Above query returns only 25607 documents whereas it should return 110827
documents.
And If I run above query using range filter it gives me expected results
i.e. 110827 documents.
Query:
{
"query": {
"filtered": {
"filter": {
"range": {
"giving.assignee": {}
}
}
}
},
"size": 2000
}
Can someone please explain the difference?
It would be helpful to know when to use Exists filter ?
Thanks,
Ayush Sangani
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/db4b6fdd-cbc1-49db-aa37-2abd4fd08247%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Exists filter does not respect must_not bool filter
Yes, range filter operates on all fields.
The missing/exists operation has been slightly changed in recent versions.
For high cardinality fields, operations on the field content were very
expensive. So, an optimization was introduced: each doc carries a list of
the field names in a hidden field, and missing/exists refer to this new
hidden field, which is extremely fast.
The downside is that operations that depend on field values (like your
boolean must_not) can no longer be mixed with the new exists/missing field
name filter.
Jörg
On Thu, Sep 4, 2014 at 7:01 PM, ElasticRabbit
wrote:
> Hi Jorg,
>
> I was in a assumption that range filter has to be used for numeric fields.
> But this works thanks for the help.
>
> If anyone could enlighten me why must_not bool filter doesn't respect
> exists filter?
>
> Thanks,
> Ayush Sangani
>
> On Wednesday, September 3, 2014 5:20:54 PM UTC-4, Jörg Prante wrote:
>
>> The behavior of exists/missing has slightly changed but this is
>> unfortunately not well documented yet.
>>
>> Can you please try
>>
>> {
>> "query": {
>> "filtered": {
>> "filter": {
>> "not": {
>> "filter": {
>> "range": {
>> "giving.assignee": {
>> }
>> }
>> }
>> }
>> }
>> }
>> }
>> }
>>
>> instead and see if it works better for your case?
>>
>> See also https://github.com/elasticsearch/elasticsearch/issues/7348
>>
>> Jörg
>>
>>
>>
>> On Wed, Sep 3, 2014 at 11:07 PM, ElasticRabbit
>> wrote:
>>
>>> Hi Everyone,
>>>
>>> Goal: I want to find all the documents which does not have
>>> giving.assignee field.
>>>
>>> I am executing below query on ES version 1.3.2 involving exists filter
>>> and boolean filter.
>>>
>>> {
>>> "query": {
>>> "filtered": {
>>>"query": {
>>> "match_all": {}
>>>},
>>>"filter": {
>>>"bool": {
>>>"must_not": [
>>> {
>>> "exists": {
>>> "field": "giving.assignee"
>>> }
>>> }
>>>]
>>>}
>>>}
>>> }
>>> },
>>> "size": 2000
>>> }
>>>
>>> While executing this query it gives me those documents also where
>>> giving.assignee field exists or has some value in it.
>>> We have around 2 million documents and it's returning almost close to 2
>>> million documents.
>>>
>>> I have also tried using the missing filter but no luck.
>>>
>>> {
>>> "query": {
>>> "filtered": {
>>> "query": {
>>> "match_all": {}
>>> },
>>> "filter": {
>>> "bool": {
>>> "must": [
>>> {
>>> "missing": {
>>> "field": "giving.assignee"
>>> }
>>> }
>>> ]
>>> }
>>> }
>>> }
>>> },
>>> "size": 2
>>> }
>>>
>>> Same result as of the above query.
>>> If someone can point me what am I doing wrong here or if further
>>> information is needed please let me know.
>>> Looking forward for help.
>>>
>>> Thanks,
>>> Ayush Sangani
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>>
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%
>>> 40googlegroups.com
Re: Exists filter does not respect must_not bool filter
Hi Jorg,
I was in a assumption that range filter has to be used for numeric fields.
But this works thanks for the help.
If anyone could enlighten me why must_not bool filter doesn't respect
exists filter?
Thanks,
Ayush Sangani
On Wednesday, September 3, 2014 5:20:54 PM UTC-4, Jörg Prante wrote:
>
> The behavior of exists/missing has slightly changed but this is
> unfortunately not well documented yet.
>
> Can you please try
>
> {
> "query": {
> "filtered": {
> "filter": {
> "not": {
> "filter": {
> "range": {
> "giving.assignee": {
> }
> }
> }
> }
> }
> }
> }
> }
>
> instead and see if it works better for your case?
>
> See also https://github.com/elasticsearch/elasticsearch/issues/7348
>
> Jörg
>
>
>
> On Wed, Sep 3, 2014 at 11:07 PM, ElasticRabbit > wrote:
>
>> Hi Everyone,
>>
>> Goal: I want to find all the documents which does not have
>> giving.assignee field.
>>
>> I am executing below query on ES version 1.3.2 involving exists filter
>> and boolean filter.
>>
>> {
>> "query": {
>> "filtered": {
>>"query": {
>> "match_all": {}
>>},
>>"filter": {
>>"bool": {
>>"must_not": [
>> {
>> "exists": {
>> "field": "giving.assignee"
>> }
>> }
>>]
>>}
>>}
>> }
>> },
>> "size": 2000
>> }
>>
>> While executing this query it gives me those documents also where
>> giving.assignee field exists or has some value in it.
>> We have around 2 million documents and it's returning almost close to 2
>> million documents.
>>
>> I have also tried using the missing filter but no luck.
>>
>> {
>> "query": {
>> "filtered": {
>> "query": {
>> "match_all": {}
>> },
>> "filter": {
>> "bool": {
>> "must": [
>> {
>> "missing": {
>> "field": "giving.assignee"
>> }
>> }
>> ]
>> }
>> }
>> }
>> },
>> "size": 2
>> }
>>
>> Same result as of the above query.
>> If someone can point me what am I doing wrong here or if further
>> information is needed please let me know.
>> Looking forward for help.
>>
>> Thanks,
>> Ayush Sangani
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e2903b18-475b-48b4-b905-365d5b49e66e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Exists filter does not respect must_not bool filter
Hi Jorg,
"giving.assignee" is a string field I tried your suggestion also but it
didn't work.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"not": {
"filter": {
"exists": {
"field": "giving.assignee"
}
}
}
}
}
},
"size": 2
}
I wonder if it is a bug in ES 1.3.2.
Please let me know there is any other way to fix this.
Thanks,
Ayush Sangani
On Wednesday, September 3, 2014 5:20:54 PM UTC-4, Jörg Prante wrote:
>
> The behavior of exists/missing has slightly changed but this is
> unfortunately not well documented yet.
>
> Can you please try
>
> {
> "query": {
> "filtered": {
> "filter": {
> "not": {
> "filter": {
> "range": {
> "giving.assignee": {
> }
> }
> }
> }
> }
> }
> }
> }
>
> instead and see if it works better for your case?
>
> See also https://github.com/elasticsearch/elasticsearch/issues/7348
>
> Jörg
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/10d3b16d-1363-4dcd-ad52-0dfb3c84142e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Exists filter does not respect must_not bool filter
Hi Ivan,
Thanks for reply.
Please find below the mapping for the giving field.
{
"giving": {
"properties": {
"assignee": {
"type": "string",
"fields": {
"assignee": {
"type": "string",
"index": "analyzed",
"store": "yes",
"include_in_all": false
},
"untouched": {
"type": "string",
"index": "not_analyzed",
"store": "yes"
}
}
}
}
}
}
Thanks,
Ayush
On Wednesday, September 3, 2014 5:17:29 PM UTC-4, Ivan Brusic wrote:
>
> Is giving.assignee a sub-object or a nested document? Can you provide your
> mapping? Use the mapping API for exact results (
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-get-mapping.html
> )
>
> Perhaps enabling explain would provide some hints,
>
> --
> Ivan
>
>
> On Wed, Sep 3, 2014 at 2:07 PM, ElasticRabbit > wrote:
>
>> Hi Everyone,
>>
>> Goal: I want to find all the documents which does not have
>> giving.assignee field.
>>
>> I am executing below query on ES version 1.3.2 involving exists filter
>> and boolean filter.
>>
>> {
>> "query": {
>> "filtered": {
>>"query": {
>> "match_all": {}
>>},
>>"filter": {
>>"bool": {
>>"must_not": [
>> {
>> "exists": {
>> "field": "giving.assignee"
>> }
>> }
>>]
>>}
>>}
>> }
>> },
>> "size": 2000
>> }
>>
>> While executing this query it gives me those documents also where
>> giving.assignee field exists or has some value in it.
>> We have around 2 million documents and it's returning almost close to 2
>> million documents.
>>
>> I have also tried using the missing filter but no luck.
>>
>> {
>> "query": {
>> "filtered": {
>> "query": {
>> "match_all": {}
>> },
>> "filter": {
>> "bool": {
>> "must": [
>> {
>> "missing": {
>> "field": "giving.assignee"
>> }
>> }
>> ]
>> }
>> }
>> }
>> },
>> "size": 2
>> }
>>
>> Same result as of the above query.
>> If someone can point me what am I doing wrong here or if further
>> information is needed please let me know.
>> Looking forward for help.
>>
>> Thanks,
>> Ayush Sangani
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/465afb27-b018-4aa7-83fc-69e5cd740409%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Exists filter does not respect must_not bool filter
The behavior of exists/missing has slightly changed but this is
unfortunately not well documented yet.
Can you please try
{
"query": {
"filtered": {
"filter": {
"not": {
"filter": {
"range": {
"giving.assignee": {
}
}
}
}
}
}
}
}
instead and see if it works better for your case?
See also https://github.com/elasticsearch/elasticsearch/issues/7348
Jörg
On Wed, Sep 3, 2014 at 11:07 PM, ElasticRabbit
wrote:
> Hi Everyone,
>
> Goal: I want to find all the documents which does not have giving.assignee
> field.
>
> I am executing below query on ES version 1.3.2 involving exists filter and
> boolean filter.
>
> {
> "query": {
> "filtered": {
>"query": {
> "match_all": {}
>},
>"filter": {
>"bool": {
>"must_not": [
> {
> "exists": {
> "field": "giving.assignee"
> }
> }
>]
>}
>}
> }
> },
> "size": 2000
> }
>
> While executing this query it gives me those documents also where
> giving.assignee field exists or has some value in it.
> We have around 2 million documents and it's returning almost close to 2
> million documents.
>
> I have also tried using the missing filter but no luck.
>
> {
> "query": {
> "filtered": {
> "query": {
> "match_all": {}
> },
> "filter": {
> "bool": {
> "must": [
> {
> "missing": {
> "field": "giving.assignee"
> }
> }
> ]
> }
> }
> }
> },
> "size": 2
> }
>
> Same result as of the above query.
> If someone can point me what am I doing wrong here or if further
> information is needed please let me know.
> Looking forward for help.
>
> Thanks,
> Ayush Sangani
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com
> <https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFjgTOVnX1hfNOJEowYyz3ZUS8epu2QQjnNFK3me6TdRA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Re: Exists filter does not respect must_not bool filter
Is giving.assignee a sub-object or a nested document? Can you provide your
mapping? Use the mapping API for exact results (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-get-mapping.html
)
Perhaps enabling explain would provide some hints,
--
Ivan
On Wed, Sep 3, 2014 at 2:07 PM, ElasticRabbit
wrote:
> Hi Everyone,
>
> Goal: I want to find all the documents which does not have giving.assignee
> field.
>
> I am executing below query on ES version 1.3.2 involving exists filter and
> boolean filter.
>
> {
> "query": {
> "filtered": {
>"query": {
> "match_all": {}
>},
>"filter": {
>"bool": {
>"must_not": [
> {
> "exists": {
> "field": "giving.assignee"
> }
> }
>]
>}
>}
> }
> },
> "size": 2000
> }
>
> While executing this query it gives me those documents also where
> giving.assignee field exists or has some value in it.
> We have around 2 million documents and it's returning almost close to 2
> million documents.
>
> I have also tried using the missing filter but no luck.
>
> {
> "query": {
> "filtered": {
> "query": {
> "match_all": {}
> },
> "filter": {
> "bool": {
> "must": [
> {
> "missing": {
> "field": "giving.assignee"
> }
> }
> ]
> }
> }
> }
> },
> "size": 2
> }
>
> Same result as of the above query.
> If someone can point me what am I doing wrong here or if further
> information is needed please let me know.
> Looking forward for help.
>
> Thanks,
> Ayush Sangani
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com
> <https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBxKY4_GOfEP2PXhd0on16KZvT6Z%3D%2Bx2zwci%3D9KHJy6sQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Exists filter does not respect must_not bool filter
Hi Everyone,
Goal: I want to find all the documents which does not have giving.assignee
field.
I am executing below query on ES version 1.3.2 involving exists filter and
boolean filter.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must_not": [
{
"exists": {
"field": "giving.assignee"
}
}
]
}
}
}
},
"size": 2000
}
While executing this query it gives me those documents also where
giving.assignee field exists or has some value in it.
We have around 2 million documents and it's returning almost close to 2
million documents.
I have also tried using the missing filter but no luck.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"missing": {
"field": "giving.assignee"
}
}
]
}
}
}
},
"size": 2
}
Same result as of the above query.
If someone can point me what am I doing wrong here or if further
information is needed please let me know.
Looking forward for help.
Thanks,
Ayush Sangani
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/89f2193a-5f11-447f-901c-29790318ddbf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: How to use 'exists' filter on no indexed objects/fields
Thanks Binh. In fact the fields I'm working on are large lists and their only role in the query part is `exists`. Since they are large number lists, my intuition tells me that it's not a very good idea to include in the index. Am I correct? 在 2014年3月3日星期一UTC+1下午9时43分59秒，Binh Ly写道： > > Other than indexing the field, I can't think of another way. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/52e9b9e3-0693-418e-adf5-4294c9698d11%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: How to use 'exists' filter on no indexed objects/fields
Other than indexing the field, I can't think of another way. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cb704734-70ad-4e8b-8595-c3d1e66e7bce%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
How to use 'exists' filter on no indexed objects/fields
Hi, I have some object type like:
{
'person': {
'properties': {
'age': {'index': 'no', 'type': 'long'},
'name': {'index': 'no', 'type': 'string'}
}
}
}
So in fact these fields are not indexed at all, they are not visible to
filters. But what I want to is to be able to use 'exists' and 'missing'
filters on these fields ('person', 'person.age' or 'person.name').
A simple hack is to add an indexed, boolean field for each no index field,
like:
{
'person': {
'properties': {
'age': {'index': 'no', 'type': 'long'},
'age_exists': {'type': 'boolean'},
'name': {'index': 'no', 'type': 'string'}
'name_exists': {'index': 'no', 'type': 'boolean'}
}
}
}
And modify my queries that check field existence on these boolean flag
fields.
This works for a few fields like this but now I have plenty of them and it
also complicates my query logic. So dear elasticsearch users, is there a
better solution to this?
Thanks.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/67cc1526-f4a8-4c0b-aaff-a78cf23fca6c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: exists filter
You would need to combine your two filters with a bool filter:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-bool-filter.html
The constant score query should be at the top level and the exists/term
filters should be combined with a bool filter underneath it. The constant
score wrapper is not really necessary since your filters are not doing any
scoring, so you can just skip it and use the bool filter.
Something like (not tested!)
POST phrase/test/_search
{
"filter": {
"bool": {
"must": [
{
"term": {
"b_id": "6"
}
},
{
"exists": {
"field": "rear_placement"
}
}
]
}
},
"fields": [
"rear_placement",
"price",
"controller",
"name",
"short_name",
"nm"
],
"sort": [
"rear_placement"
]
}
Cheers,
Ivan
On Tue, Jan 14, 2014 at 4:48 AM, James Reynolds wrote:
> I'm trying to run a query that looks sort of like this:
>
> {
> "filter": {
> "term": {
> "b_id": "6"
> },
> "constant_score": {
> "filter": {
> "exists": {
> "field": "rear_placement"
> }
> }
> }
> },
> "fields": [
> "rear_placement",
> "price",
> "controller",
> "name",
> "short_name",
> "nm"
> ],
> "sort": [
> "rear_placement"
> ]
> }
>
>
>
> When I add the "constant score" bit the "fields" do not return. Only
> _source. Removing the constant score exists bit, the six fields above
> return without issue.
>
> It isn't the end of the world for me to return the entire document in this
> case, I was just wondering what the deal is with this and if there is
> something I'm doing wrong.
>
> James
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/a06d4480-16bb-426e-b97a-467b2cd5bf9f%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDdx_i0WALXGNhFaUsQjsPzUuNERbJMtwW7uztSXVOuCw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
exists filter
I'm trying to run a query that looks sort of like this:
{
"filter": {
"term": {
"b_id": "6"
},
"constant_score": {
"filter": {
"exists": {
"field": "rear_placement"
}
}
}
},
"fields": [
"rear_placement",
"price",
"controller",
"name",
"short_name",
"nm"
],
"sort": [
"rear_placement"
]
}
When I add the "constant score" bit the "fields" do not return. Only
_source. Removing the constant score exists bit, the six fields above
return without issue.
It isn't the end of the world for me to return the entire document in this
case, I was just wondering what the deal is with this and if there is
something I'm doing wrong.
James
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a06d4480-16bb-426e-b97a-467b2cd5bf9f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
