Re: Parent/Child query performance in version 1.1.2

2014-08-26 Thread Mark Greene
Just wanted to close the loop on this in case anyone stumbled upon the same issue. After upgrading to version 1.3.2 which had the performance increase stemming from https://github.com/elasticsearch/elasticsearch/pull/5846, we were able to see a dramatic decrease in parent/child query latency. W

Re: Parent/Child query performance in version 1.1.2

2014-08-25 Thread Mark Greene
Hey Clinton, Thanks for the heads up on what's on the horizon. That definitely sounds like a drastic improvement. That being said, my fear here is that even with that improvement, this data model (parent/child) doesn't seem to that performant with a moderate amount of documents. In order for us

Re: Parent/Child query performance in version 1.1.2

2014-08-25 Thread Clinton Gormley
Something else to note: parent-child now uses global ordinals to make queries 3x faster than they were previously, but global ordinals need to be rebuilt after the index has refreshed (assuming some data has changed). Currently there is no way to refresh p/c global ordinals "eagerly" (ie during th

Re: Parent/Child query performance in version 1.1.2

2014-08-25 Thread Mark Greene
Hi Adrien, Thanks for reaching out. We actually were exited to see the performance improvements stated in the 1.2.0 release notes so we upgraded to 1.3.2. We saw some performance improvement but it wasn't orders of magnitude and queries are still running very slow. We also tried your suggesti

Re: Parent/Child query performance in version 1.1.2

2014-08-22 Thread Adrien Grand
Hi Mark, Given that you had 1 replica in your first setup, it could take several queries to warm up the field data cache completely, does the query still take 16 seconds to run if you run it eg. 10 times? (3 should be enough, but just to be sure) Does it change anything if you query elasticsearch

Re: Parent/Child query performance in version 1.1.2

2014-08-22 Thread Mark Greene
I wanted to update the list with an interesting piece of information. We found that when we took one of our two data nodes out of the cluster, leaving just one data node with no replicas, the query performance increased dramatically. The queries are now returning in <100ms on subsequent executi

Parent/Child query performance in version 1.1.2

2014-08-21 Thread Mark Greene
We are experiencing slow parent/child queries even when we run the query a second time and I wanted to know if this is just the limit of this feature within ElasticSearch. According to the ES Docs (http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/parent-child-performance.html)