Optimal configuration for 2 high powered nodes
Hi, I think the 2nd proposition could be the worst. With more than 32GB the JVM will use uncompressed pointer. Take a look at this : http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html Hope this help -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aa905691-102d-4a6c-8c2f-a2b50a0164af%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Elasticsearch give a factor to aggregations
Hi all, I'm wondering something about aggregations, say Percentiles. When a percentile aggregation is processed, it uses a specific field as a reference. If the 50th percentile for field 'f' is 10, it means there are 50% of documents with 'f' under 10. = Each document has the same weight in the aggregation (=1) I'm wondering if it could be possible to give a different weight for each document. The following Gist give an example of what I'd like to do : https://gist.github.com/rnonnon/093c111014bd14a46efe I'd like to compute some percentiles on the age field. But for each document, there is a count field associated. For example, there are 5 person who are 10 years old ; 1 who is 20 years old... If the percentile agg runs, it won't use my factor to compute the percentile... I don't think that feature is natively supported, but, do you guess it could be easily supported? Thanks. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/17f66762-e51c-4241-9957-e34403e4f054%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Range Filter Aggregation on array
Hi,
Thanks for your answer. I think it will be the solution.
Le mardi 7 octobre 2014 11:53:18 UTC+2, Adrien Grand a écrit :
Hi,
Aggregations can only count documents. So if you want to count values, you
need to model your data in such a way that each value is going to be a
document, for instance by using nested documents. Here is an example how
you could do that:
DELETE test
PUT test
{
mappings: {
test: {
properties: {
array: {
type: nested,
properties: {
value: {
type: integer
}
}
}
}
}
}
}
PUT test/test/1
{
array: [
{
value: 927
},
{
value: 425
},
{
value: 455
},
{
value: 120
}
]
}
GET test/_search
{
aggs: {
array: {
nested: {
path: array
},
aggs: {
less_than_500: {
filter: {
range: {
array.value: {
to: 500
}
}
}
}
}
}
}
}
On Tue, Oct 7, 2014 at 10:41 AM, Rémi Nonnon remi@gmail.com
javascript: wrote:
Hi all,
I have some troubles when I try to use Range Filter aggregation on an
array.
Example of 1 document :
{
_index: test,
_type: values,
_id: 1,
_version: 1,
found: true,
_source: {
array: [927,425,455,120]
}
}
For all my values document, I'd like to count, on array field, how
many numbers are less than 200 and how many are greater than 500.
I tried this aggregation :
GET /test/values/_search
{
aggs : {
less : {
filter:{range:{array:{ lt : 200}}}
},
greater : {
filter:{range:{array:{ gt : 500}}}
}
}
}
But the 1st filter count the number of documents which have an array
containing a value 200 and the 2nd how many have a value 500. What I'd
like is to count, for all documents, how many values (not how many
document) are 200 and how many are 500.
If I make a sum / min / max aggregation, it will be on each value in
arrays but not with a filter. Do you have an idea how to do that thing?
I did it with 2 script filters, it works, but the computing time is too
bad :
{
aggs : {
less : {
sum:{
script: def sum = 0; doc['array'].values.each(){if(it 200)
sum++}; return sum;}
},
greater : {
sum:{
script:def sum = 0; doc['array'].values.each(){if(it 500)
sum++}; return sum;}
}
}
Any Idea? Thanks!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/76084de4-472f-4084-9a27-9f158d018043%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/76084de4-472f-4084-9a27-9f158d018043%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
Adrien Grand
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/eea06154-57d6-4e49-a680-18fe77857be6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Range Filter Aggregation on array
Hi,
It's ok, I think you are right, nested will be the solution.
Regards,
Rémi.
Le mardi 7 octobre 2014 11:38:44 UTC+2, David Pilato a écrit :
I have no idea. It's hard to understand what you exactly did without a
full reproduction.
A SENSE script posted as a GIST would help a lot I think.
--
*David Pilato* | Technical Advocate | *elasticsearch.com
http://elasticsearch.com*
david@elasticsearch.com javascript:
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
http://twitter.com/scrutmydocs
https://twitter.com/scrutmydocs
Le 7 octobre 2014 à 10:49:54, Rémi Nonnon (remi@gmail.com
javascript:) a écrit:
Hi,
Thanks for your immediate help!
(sorry i deleted this post because I didn't find how to edit it. I posted
a new one)
I did it with nested documents but the array seems to be faster on other
aggregations like extended stats or percentiles then I wondered if my
filters are well written.
Thanks again.
Le mardi 7 octobre 2014 10:38:43 UTC+2, David Pilato a écrit :
One of the solution I can think is by using nested documents for your
array.
In that case, each element of the array will be seen as the document.
Hope this helps
--
* David Pilato* | Technical Advocate | *elasticsearch.com
http://elasticsearch.com*
david@elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
http://twitter.com/scrutmydocs
https://twitter.com/scrutmydocs
Le 7 octobre 2014 à 10:36:28, Rémi Nonnon (remi@gmail.com) a écrit:
Hi all,
I have some troubles when I try to use Range Filter aggregation on an
array.
Example of 1 document :
```json
{
_index: test,
_type: values,
_id: 1,
_version: 1,
found: true,
_source: {
array: [927,425,455,120]
}
}
```
For all my values document, I'd like to count for array field how
many numbers are less than 200 and how many are greater than 500.
I tried this aggregation :
GET /test/values/_search
```json
{
aggs : {
less : {
filter:{range:{array:{ lt : 200}}}
},
greater : {
filter:{range:{array:{ gt : 500}}}
}
}
}
```
But the 1st filter count the number of documents which have an array
containing a value 200 and the 2nd how many have a value 500.
What I'd like is to count how many values are 200 and how many are 500
(not how many document).
If I make a sum / min / max aggregation, it will be on each value in
arrays but not with a filter.
Do you have an idea how to do that thing?
I did it with 2 script filters, it works, but the computing time is too
bad :
```json
{
aggs : {
less : {
sum:{
script: def sum = 0; doc['array'].values.each(){if(it 200)
sum++}; return sum;}
},
greater : {
sum:{
script:def sum = 0; doc['array'].values.each(){if(it 500)
sum++}; return sum;}
}
}
```
Any Idea?
Thanks!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ea6b7b2e-2874-4387-b314-86a425e88b77%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ea6b7b2e-2874-4387-b314-86a425e88b77%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/534e9e53-32af-4660-b56c-e63c01006b21%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Range Filter Aggregation on array
Hi all,
I have some troubles when I try to use Range Filter aggregation on an array.
Example of 1 document :
{
_index: test,
_type: values,
_id: 1,
_version: 1,
found: true,
_source: {
array: [927,425,455,120]
}
}
For all my values document, I'd like to count, on array field, how many
numbers are less than 200 and how many are greater than 500.
I tried this aggregation :
GET /test/values/_search
{
aggs : {
less : {
filter:{range:{array:{ lt : 200}}}
},
greater : {
filter:{range:{array:{ gt : 500}}}
}
}
}
But the 1st filter count the number of documents which have an array
containing a value 200 and the 2nd how many have a value 500. What I'd
like is to count, for all documents, how many values (not how many
document) are 200 and how many are 500.
If I make a sum / min / max aggregation, it will be on each value in arrays
but not with a filter. Do you have an idea how to do that thing?
I did it with 2 script filters, it works, but the computing time is too bad
:
{
aggs : {
less : {
sum:{
script: def sum = 0; doc['array'].values.each(){if(it 200) sum++};
return sum;}
},
greater : {
sum:{
script:def sum = 0; doc['array'].values.each(){if(it 500) sum++};
return sum;}
}
}
Any Idea? Thanks!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/76084de4-472f-4084-9a27-9f158d018043%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Range Filter Aggregation on array
Hi,
Thanks for your immediate help!
(sorry i deleted this post because I didn't find how to edit it. I posted a
new one)
I did it with nested documents but the array seems to be faster on other
aggregations like extended stats or percentiles then I wondered if my
filters are well written.
Thanks again.
Le mardi 7 octobre 2014 10:38:43 UTC+2, David Pilato a écrit :
One of the solution I can think is by using nested documents for your
array.
In that case, each element of the array will be seen as the document.
Hope this helps
--
*David Pilato* | Technical Advocate | *elasticsearch.com
http://elasticsearch.com*
david@elasticsearch.com javascript:
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
http://twitter.com/scrutmydocs
https://twitter.com/scrutmydocs
Le 7 octobre 2014 à 10:36:28, Rémi Nonnon (remi@gmail.com
javascript:) a écrit:
Hi all,
I have some troubles when I try to use Range Filter aggregation on an
array.
Example of 1 document :
```json
{
_index: test,
_type: values,
_id: 1,
_version: 1,
found: true,
_source: {
array: [927,425,455,120]
}
}
```
For all my values document, I'd like to count for array field how many
numbers are less than 200 and how many are greater than 500.
I tried this aggregation :
GET /test/values/_search
```json
{
aggs : {
less : {
filter:{range:{array:{ lt : 200}}}
},
greater : {
filter:{range:{array:{ gt : 500}}}
}
}
}
```
But the 1st filter count the number of documents which have an array
containing a value 200 and the 2nd how many have a value 500.
What I'd like is to count how many values are 200 and how many are 500
(not how many document).
If I make a sum / min / max aggregation, it will be on each value in
arrays but not with a filter.
Do you have an idea how to do that thing?
I did it with 2 script filters, it works, but the computing time is too
bad :
```json
{
aggs : {
less : {
sum:{
script: def sum = 0; doc['array'].values.each(){if(it 200)
sum++}; return sum;}
},
greater : {
sum:{
script:def sum = 0; doc['array'].values.each(){if(it 500)
sum++}; return sum;}
}
}
```
Any Idea?
Thanks!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ea6b7b2e-2874-4387-b314-86a425e88b77%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Histogram aggregation keys
Hi,
I'm working with histogram aggregation but there is something strange with
keys.
For instance (cf :
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-histogram-aggregation.html):
*If I use this request :*
{
aggs : {
prices : {
histogram : {
field : price,
interval : 50
}
}
}
}
*I obtain something like this :*
{
aggregations: {
prices : {
buckets: [
{
*key_as_string : 0,*
key: 0,
doc_count: 2
},
{
*key_as_string : 50,*
key: 50,
doc_count: 4
},
{
*key_as_string : 150,*
key: 150,
doc_count: 3
}
]
}
}
}
*Instead of :*
{
aggregations: {
prices : {
buckets: [
{
key: 0,
doc_count: 2
},
{
key: 50,
doc_count: 4
},
{
key: 150,
doc_count: 3
}
]
}
}
}
You could say, it's not important but it generates json ~1/3 bigger...
*Is there a mean to disable this ???*
Moreover, in Elasticsearch Java API, it could be fine to have a method to
request the response as a hash instead keyed by the buckets keys (cf
:http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-histogram-aggregation.html#_response_format)
*Thanks!!!*
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/47624c50-32d8-4308-b6e5-b07707ad353d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
