Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Paul Smith
if you do use it, don't forget we build for ES 0.19, so change the pom.xml to your ES version otherwise it won't connect... :) On 31 January 2014 12:29, Paul Smith wrote: > if it helps at all, i've pushed the flappy item detector tool (*cough*) > here: > > https://github.com/Aconex/es-flappyite

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Paul Smith
if it helps at all, i've pushed the flappy item detector tool (*cough*) here: https://github.com/Aconex/es-flappyitem-detector We have a simple 3-node cluster, 5 shards, 1 replica, so I'm sure there's code in there that is built around those assumptions, but should be easily modified to suit your

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread xavier
Yeah, that should work. I'll take a look at that and see if it can help pinpoint problematic shards. On Thursday, January 30, 2014 4:59:04 PM UTC-8, tallpsmith wrote: > > the flappy detection tool I have connects to the cluster using the > standard java autodiscovery mechanism, and, works out w

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Paul Smith
the flappy detection tool I have connects to the cluster using the standard java autodiscovery mechanism, and, works out which shards are involved, and then creates explicit TransportClient connection to each host, so would need access to 9300 (the SMILE based protocol port). Would that help? (is

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread xavier
We have 4 query heads total (esq1.r6, esq2.r6, esq3.r7, esq4.r7). Interestingly query heads in the same rack give the same results. We don't do deletes at all on these indices so that shouldn't be an issue. Unfortunately at the moment I can't do preference=_local while getting the _id(s) direct

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Paul Smith
If you can narrow down a specific few IDs of results that appear/disappear based on the primary/replica shard, and confirm through an explicit GET of that ID with the preference=_local on the primary shard & replica for that result. To work out which shard # a specific ID belongs to, you can run t

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Binh Ly
Xavier, can you post an example of 1 full query and then also show how the results of this one query is inconsistent? Just trying to understand what is inconsistent. Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from

Primary vs. replica shard inconsistencies?

2014-01-30 Thread xavier
We're currently running ElasticSearch 0.90.5. When doing the same search across different query heads, I'm seeing an inconsistent number of results. However, if I add preference=_primary (or _primary_first) I get the same results across the board. I have checked that all query heads report the