Optimal configuration for 2 high powered nodes
Hi, I think the 2nd proposition could be the worst. With more than 32GB the JVM will use uncompressed pointer. Take a look at this : http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html Hope this help -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aa905691-102d-4a6c-8c2f-a2b50a0164af%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Elasticsearch give a factor to aggregations
Hi all, I'm wondering something about aggregations, say "Percentiles". When a percentile aggregation is processed, it uses a specific field as a reference. If the 50th percentile for field 'f' is 10, it means there are 50% of documents with 'f' under 10. => Each document has the same weight in the aggregation (=1) I'm wondering if it could be possible to give a different weight for each document. The following Gist give an example of what I'd like to do : https://gist.github.com/rnonnon/093c111014bd14a46efe I'd like to compute some percentiles on the "age" field. But for each document, there is a "count" field associated. For example, there are 5 person who are 10 years old ; 1 who is 20 years old... If the percentile agg runs, it won't use my factor to compute the percentile... I don't think that feature is natively supported, but, do you guess it could be easily supported? Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/17f66762-e51c-4241-9957-e34403e4f054%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Range Filter Aggregation on array
Hi,
It's ok, I think you are right, nested will be the solution.
Regards,
Rémi.
Le mardi 7 octobre 2014 11:38:44 UTC+2, David Pilato a écrit :
>
> I have no idea. It's hard to understand what you exactly did without a
> full reproduction.
> A SENSE script posted as a GIST would help a lot I think.
>
> --
> *David Pilato* | Technical Advocate | *elasticsearch.com
> <http://elasticsearch.com>*
> david@elasticsearch.com
> @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr
> <https://twitter.com/elasticsearchfr> | @scrutmydocs
> <http://twitter.com/scrutmydocs>
> <https://twitter.com/scrutmydocs>
>
>
>
> Le 7 octobre 2014 à 10:49:54, Rémi Nonnon (remi@gmail.com
> ) a écrit:
>
> Hi,
>
> Thanks for your immediate help!
> (sorry i deleted this post because I didn't find how to edit it. I posted
> a new one)
>
> I did it with nested documents but the array seems to be faster on other
> aggregations like extended stats or percentiles then I wondered if my
> filters are well written.
>
> Thanks again.
>
>
> Le mardi 7 octobre 2014 10:38:43 UTC+2, David Pilato a écrit :
>>
>> One of the solution I can think is by using nested documents for your
>> array.
>> In that case, each element of the array will be seen as the document.
>>
>> Hope this helps
>>
>>
>> --
>> * David Pilato* | Technical Advocate | *elasticsearch.com
>> <http://elasticsearch.com>*
>> david@elasticsearch.com
>> @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr
>> <https://twitter.com/elasticsearchfr> | @scrutmydocs
>> <http://twitter.com/scrutmydocs>
>> <https://twitter.com/scrutmydocs>
>>
>>
>>
>> Le 7 octobre 2014 à 10:36:28, Rémi Nonnon (remi@gmail.com) a écrit:
>>
>> Hi all,
>>
>> I have some troubles when I try to use Range Filter aggregation on an
>> array.
>>
>> Example of 1 document :
>> ```json
>> {
>>"_index": "test",
>>"_type": "values",
>>"_id": "1",
>>"_version": 1,
>>"found": true,
>>"_source": {
>> "array": [927,425,455,120]
>> }
>> }
>> ```
>>
>> For all my "values" document, I'd like to count for "array" field how
>> many numbers are less than 200 and how many are greater than 500.
>>
>> I tried this aggregation :
>>
>> GET /test/values/_search
>> ```json
>> {
>> "aggs" : {
>> "less" : {
>> "filter":{"range":{"array":{ "lt" : 200}}}
>> },
>> "greater" : {
>>"filter":{"range":{"array":{ "gt" : 500}}}
>> }
>> }
>> }
>> ```
>>
>> But the 1st filter count the number of documents which have an array
>> containing a value <200 and the 2nd how many have a value >500.
>> What I'd like is to count how many values are <200 and how many are >500
>> (not how many document).
>>
>> If I make a sum / min / max aggregation, it will be on each value in
>> arrays but not with a filter.
>> Do you have an idea how to do that thing?
>>
>>
>> I did it with 2 script filters, it works, but the computing time is too
>> bad :
>>
>> ```json
>> {
>> "aggs" : {
>> "less" : {
>> "sum":{
>> "script":" def sum = 0; doc['array'].values.each(){if(it < 200)
>> sum++}; return sum;"}
>> },
>> "greater" : {
>>"sum":{
>> "script":"def sum = 0; doc['array'].values.each(){if(it > 500)
>> sum++}; return sum;"}
>> }
>> }
>> ```
>>
>> Any Idea?
>> Thanks!
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googleg
Re: Range Filter Aggregation on array
Hi,
Thanks for your answer. I think it will be the solution.
Le mardi 7 octobre 2014 11:53:18 UTC+2, Adrien Grand a écrit :
>
> Hi,
>
> Aggregations can only count documents. So if you want to count values, you
> need to model your data in such a way that each value is going to be a
> document, for instance by using nested documents. Here is an example how
> you could do that:
>
> DELETE test
>
> PUT test
> {
> "mappings": {
> "test": {
> "properties": {
> "array": {
> "type": "nested",
> "properties": {
> "value": {
> "type": "integer"
> }
> }
> }
> }
> }
> }
> }
>
> PUT test/test/1
> {
> "array": [
> {
> "value": 927
> },
> {
> "value": 425
> },
> {
> "value": 455
> },
> {
> "value": 120
> }
> ]
> }
>
> GET test/_search
> {
> "aggs": {
> "array": {
> "nested": {
> "path": "array"
> },
> "aggs": {
> "less_than_500": {
> "filter": {
> "range": {
> "array.value": {
> "to": 500
> }
> }
> }
> }
> }
> }
> }
> }
>
> On Tue, Oct 7, 2014 at 10:41 AM, Rémi Nonnon > wrote:
>
>> Hi all,
>>
>> I have some troubles when I try to use Range Filter aggregation on an
>> array.
>>
>> Example of 1 document :
>>
>> {
>>"_index": "test",
>>"_type": "values",
>>"_id": "1",
>>"_version": 1,
>>"found": true,
>>"_source": {
>> "array": [927,425,455,120]
>> }
>> }
>>
>> For all my "values" document, I'd like to count, on "array" field, how
>> many numbers are less than 200 and how many are greater than 500.
>>
>> I tried this aggregation :
>>
>> GET /test/values/_search
>>
>> {
>> "aggs" : {
>> "less" : {
>> "filter":{"range":{"array":{ "lt" : 200}}}
>> },
>> "greater" : {
>>"filter":{"range":{"array":{ "gt" : 500}}}
>> }
>> }
>> }
>>
>> But the 1st filter count the number of documents which have an array
>> containing a value <200 and the 2nd how many have a value >500. What I'd
>> like is to count, for all documents, how many values (not how many
>> document) are <200 and how many are >500.
>>
>> If I make a sum / min / max aggregation, it will be on each value in
>> arrays but not with a filter. Do you have an idea how to do that thing?
>>
>> I did it with 2 script filters, it works, but the computing time is too
>> bad :
>>
>> {
>> "aggs" : {
>> "less" : {
>> "sum":{
>> "script":" def sum = 0; doc['array'].values.each(){if(it < 200)
>> sum++}; return sum;"}
>> },
>> "greater" : {
>>"sum":{
>> "script":"def sum = 0; doc['array'].values.each(){if(it > 500)
>> sum++}; return sum;"}
>> }
>> }
>>
>> Any Idea? Thanks!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/76084de4-472f-4084-9a27-9f158d018043%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/elasticsearch/76084de4-472f-4084-9a27-9f158d018043%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Adrien Grand
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/eea06154-57d6-4e49-a680-18fe77857be6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Range Filter Aggregation on array
Hi,
Thanks for your immediate help!
(sorry i deleted this post because I didn't find how to edit it. I posted a
new one)
I did it with nested documents but the array seems to be faster on other
aggregations like extended stats or percentiles then I wondered if my
filters are well written.
Thanks again.
Le mardi 7 octobre 2014 10:38:43 UTC+2, David Pilato a écrit :
>
> One of the solution I can think is by using nested documents for your
> array.
> In that case, each element of the array will be seen as the document.
>
> Hope this helps
>
>
> --
> *David Pilato* | Technical Advocate | *elasticsearch.com
> <http://elasticsearch.com>*
> david@elasticsearch.com
> @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr
> <https://twitter.com/elasticsearchfr> | @scrutmydocs
> <http://twitter.com/scrutmydocs>
> <https://twitter.com/scrutmydocs>
>
>
>
> Le 7 octobre 2014 à 10:36:28, Rémi Nonnon (remi@gmail.com
> ) a écrit:
>
> Hi all,
>
> I have some troubles when I try to use Range Filter aggregation on an
> array.
>
> Example of 1 document :
> ```json
> {
>"_index": "test",
>"_type": "values",
>"_id": "1",
>"_version": 1,
>"found": true,
>"_source": {
> "array": [927,425,455,120]
> }
> }
> ```
>
> For all my "values" document, I'd like to count for "array" field how many
> numbers are less than 200 and how many are greater than 500.
>
> I tried this aggregation :
>
> GET /test/values/_search
> ```json
> {
> "aggs" : {
> "less" : {
> "filter":{"range":{"array":{ "lt" : 200}}}
> },
> "greater" : {
>"filter":{"range":{"array":{ "gt" : 500}}}
> }
> }
> }
> ```
>
> But the 1st filter count the number of documents which have an array
> containing a value <200 and the 2nd how many have a value >500.
> What I'd like is to count how many values are <200 and how many are >500
> (not how many document).
>
> If I make a sum / min / max aggregation, it will be on each value in
> arrays but not with a filter.
> Do you have an idea how to do that thing?
>
>
> I did it with 2 script filters, it works, but the computing time is too
> bad :
>
> ```json
> {
> "aggs" : {
> "less" : {
> "sum":{
> "script":" def sum = 0; doc['array'].values.each(){if(it < 200)
> sum++}; return sum;"}
> },
> "greater" : {
>"sum":{
> "script":"def sum = 0; doc['array'].values.each(){if(it > 500)
> sum++}; return sum;"}
> }
> }
> ```
>
> Any Idea?
> Thanks!
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com
>
> <https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ea6b7b2e-2874-4387-b314-86a425e88b77%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Range Filter Aggregation on array
Hi all,
I have some troubles when I try to use Range Filter aggregation on an array.
Example of 1 document :
{
"_index": "test",
"_type": "values",
"_id": "1",
"_version": 1,
"found": true,
"_source": {
"array": [927,425,455,120]
}
}
For all my "values" document, I'd like to count, on "array" field, how many
numbers are less than 200 and how many are greater than 500.
I tried this aggregation :
GET /test/values/_search
{
"aggs" : {
"less" : {
"filter":{"range":{"array":{ "lt" : 200}}}
},
"greater" : {
"filter":{"range":{"array":{ "gt" : 500}}}
}
}
}
But the 1st filter count the number of documents which have an array
containing a value <200 and the 2nd how many have a value >500. What I'd
like is to count, for all documents, how many values (not how many
document) are <200 and how many are >500.
If I make a sum / min / max aggregation, it will be on each value in arrays
but not with a filter. Do you have an idea how to do that thing?
I did it with 2 script filters, it works, but the computing time is too bad
:
{
"aggs" : {
"less" : {
"sum":{
"script":" def sum = 0; doc['array'].values.each(){if(it < 200) sum++};
return sum;"}
},
"greater" : {
"sum":{
"script":"def sum = 0; doc['array'].values.each(){if(it > 500) sum++};
return sum;"}
}
}
Any Idea? Thanks!
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/76084de4-472f-4084-9a27-9f158d018043%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Range Filter Aggregation on array
Hi all,
I have some troubles when I try to use Range Filter aggregation on an array.
Example of 1 document :
```json
{
"_index": "test",
"_type": "values",
"_id": "1",
"_version": 1,
"found": true,
"_source": {
"array": [927,425,455,120]
}
}
```
For all my "values" document, I'd like to count for "array" field how many
numbers are less than 200 and how many are greater than 500.
I tried this aggregation :
GET /test/values/_search
```json
{
"aggs" : {
"less" : {
"filter":{"range":{"array":{ "lt" : 200}}}
},
"greater" : {
"filter":{"range":{"array":{ "gt" : 500}}}
}
}
}
```
But the 1st filter count the number of documents which have an array
containing a value <200 and the 2nd how many have a value >500.
What I'd like is to count how many values are <200 and how many are >500
(not how many document).
If I make a sum / min / max aggregation, it will be on each value in arrays
but not with a filter.
Do you have an idea how to do that thing?
I did it with 2 script filters, it works, but the computing time is too bad
:
```json
{
"aggs" : {
"less" : {
"sum":{
"script":" def sum = 0; doc['array'].values.each(){if(it < 200)
sum++}; return sum;"}
},
"greater" : {
"sum":{
"script":"def sum = 0; doc['array'].values.each(){if(it > 500)
sum++}; return sum;"}
}
}
```
Any Idea?
Thanks!
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Aggregation using the results of an aggregation?
Hi,
If I am not wrong it's not possible yet to use the result of an aggregation
in another one.
You have to do that outside elasticsearch.
Regards,
Rémi
Le vendredi 11 juillet 2014 02:39:16 UTC+2, Greg Day a écrit :
>
> Hi guys
>
> Im wondering if it is possible to use the results of an aggregation in a
> aggregation calculation?
> eg: How would I calculate stocktosales in the example below?
>
>
> "aggregations": {
> "product": {
> "aggregations": {
> "sales": {
> "sum": {
> "field": "Sale"
> }
> },
> "onhand": {
> "avg": {
> "field": "Onhand"
> }
> },
>"*stocktosales*" : {
> * should be onhand/sales*
>}
> }
> }
> }
>
> Thanks!
>
> Greg
>
> Watch the fun new 2 minute and 2 second video tour of Vend
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0e701b9f-068d-4410-8eaf-51903bc08b2c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Histogram aggregation keys
Hi,
I'm working with histogram aggregation but there is something strange with
keys.
For instance (cf :
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-histogram-aggregation.html):
*If I use this request :*
{
"aggs" : {
"prices" : {
"histogram" : {
"field" : "price",
"interval" : 50
}
}
}
}
*I obtain something like this :*
{
"aggregations": {
"prices" : {
"buckets": [
{
*"key_as_string" : "0",*
"key": 0,
"doc_count": 2
},
{
*"key_as_string" : "50",*
"key": 50,
"doc_count": 4
},
{
*"key_as_string" : "150",*
"key": 150,
"doc_count": 3
}
]
}
}
}
*Instead of :*
{
"aggregations": {
"prices" : {
"buckets": [
{
"key": 0,
"doc_count": 2
},
{
"key": 50,
"doc_count": 4
},
{
"key": 150,
"doc_count": 3
}
]
}
}
}
You could say, it's not important but it generates json ~1/3 bigger...
*Is there a mean to disable this ???*
Moreover, in Elasticsearch Java API, it could be fine to have a method to
request the response as a hash instead keyed by the buckets keys (cf
:http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-histogram-aggregation.html#_response_format)
*Thanks!!!*
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/47624c50-32d8-4308-b6e5-b07707ad353d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
