Re: Inconsistent results when aggregate by field from array

2015-04-14 Thread Nils Dijk
Hi, You want to look at nested objects: http://www.elastic.co/guide/en/elasticsearch/guide/master/nested-objects.html -- Nils Tip: try formatting your post, it was hard to read. On Tuesday, March 31, 2015 at 4:37:40 AM UTC+2, Iana Bondarskaia wrote: Hi All, I have array of objects in each

Re: Incorrect Aggregations returned from ES

2015-04-14 Thread Nils Dijk
Hi, To me this sounds a lot like an issue that was happening to me a week before the release of 1.0.0. This issue was related to internal memory reuse within Elasticsearch before the result was read out. The issue is documented here: https://github.com/elastic/elasticsearch/issues/5021 What I

Re: Disk Decommission

2015-03-03 Thread Nils Dijk
You might want to tune the allocator: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk cluster.routing.allocation.disk.watermark.high controls the high watermark. It defaults to 90%, meaning ES will attempt to relocate shards to another

Re: Is ElasticSearch truly scalable for analytics?

2015-01-15 Thread Nils Dijk
Adding a 'node reduce phase' to aggregations is something I'm very interested in, and also investigating for the project I'm currently working on. If you introduce an extra reduction phase (for multiple shards on the same node) you introduce further potential for inaccuracies in the final

Re: Inconsistent responses from aggregations (ES1.0.0RC1)

2014-02-07 Thread Nils Dijk
Hi Adrien, Good news! The problem is solved. Can't wait for the release containing the fix, but for now I will use my own build :) On Thursday, February 6, 2014 5:25:11 PM UTC+1, Nils Dijk wrote: Yay! I will try this somewhere tomorrow. Thanks for fixing, much appreciated! Seems like

Re: Inconsistent responses from aggregations (ES1.0.0RC1)

2014-02-06 Thread Nils Dijk
to understand the cause and to write a fix: https://github.com/elasticsearch/elasticsearch/pull/5039 Thanks very much for reporting this and for your help reproducing and debugging this issue! On Thu, Feb 6, 2014 at 2:08 PM, Nils Dijk m...@thanod.nl javascript:wrote: Good, It is always

Re: Inconsistent responses from aggregations (ES1.0.0RC1)

2014-02-05 Thread Nils Dijk
the issues here. On Wednesday, February 5, 2014 5:40:10 PM UTC+1, Adrien Grand wrote: I just installed 1.7u25 on a mac with maverick to try to reproduce the issue, but without success (on 1.0.0-RC2). On Wed, Feb 5, 2014 at 4:49 PM, Nils Dijk m...@thanod.nl javascript:wrote: Hi Adrien, I'm

Re: Inconsistent responses from aggregations (ES1.0.0RC1)

2014-02-05 Thread Nils Dijk
wrote: On Wed, Feb 5, 2014 at 6:01 PM, Nils Dijk m...@thanod.nl javascript:wrote: I was trying to find out if I could disable this unsafe string comparisons, but could not really find where that should be disabled. Is there an easy way for me to switch back that change? Do you know on what

Re: Inconsistent responses from aggregations (ES1.0.0RC1)

2014-02-05 Thread Nils Dijk
Hi Jörg, Glad you could reproduce with my updated gist. cb. On Wednesday, February 5, 2014 8:18:39 PM UTC+1, Jörg Prante wrote: Nils, I ran the test on my Mac, and I can reproduce the issue. And also on Linux. Unfortunately the Mac locked up and I had to cold reboot, and my copy/paste

Re: Inconsistent responses from aggregations (ES1.0.0RC1)

2014-02-04 Thread Nils Dijk
is broken from Beta2 to RC1! I would like to test this on master, but I could not find any nightly builds of elasticsearch. Is there a location where they are stored or should I compile it myself? On Friday, January 31, 2014 6:43:07 PM UTC+1, Nils Dijk wrote: Hi Binh Ly, Thanks for the response

Re: Inconsistent responses from aggregations (ES1.0.0RC1)

2014-02-04 Thread Nils Dijk
/elasticsearch? It would be nice to fix this problem before a release of 1.0.0 since that is the first release containing the aggregations for analytics. On Tuesday, February 4, 2014 12:31:10 PM UTC+1, Nils Dijk wrote: I've loaded the same dataset in ES1.0.0.Beta2 with the same index configuration

Inconsistent responses from aggregations (ES1.0.0RC1)

2014-01-31 Thread Nils Dijk
Hi, I am tinkering with elasticsearch 1.0.0RC1 for a bit. Especially the part of aggregations. When looking closer to the responses of the aggregations I noticed the numbers fluctuated all the time. I have an index: shards: 10 replicas: 0 documents: ~1M Currently I'm not ingesting data