Re: Replica Shard inconsistencies & disabling compression don't appear to help

2014-08-21 Thread Paul Smith
hecksum after > network transport? > > In other words, are you sure the inconsistency you observe is caused by a > compression issue in LZF? > > Jörg > > > On Thu, Aug 21, 2014 at 5:52 AM, Paul Smith > wrote: > >> Hi all, >> >> The recent ES 1.

Replica Shard inconsistencies & disabling compression don't appear to help

2014-08-20 Thread Paul Smith
ough because of said upgrade issues (all our side, not an ES problem). anyway, thought I would pass that data point on. regards, Paul Smith -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group an

Re: Shard data inconsistencies

2014-08-13 Thread Paul Smith
confirm/deny it being one of these cases. There is also support to issue basic move commands with some simple logic to attempt to rebuild that replica. Hope that helps. cheers, Paul Smith On 8 August 2014 01:14, aaron wrote: > I've noticed on a few of my clusters that some shard r

Re: Removing unused fields (more Lucene than ES but..)

2014-04-03 Thread Paul Smith
ately not so easy to fix... > > On Thu, Apr 3, 2014 at 7:49 PM, Paul Smith wrote: > > Thanks for the JIRA link Robert, I've added a comment to it just to share > > the real world aspect of what happened to us for background. > > > > > > On 1 April 2014 18:29

Re: Removing unused fields (more Lucene than ES but..)

2014-04-03 Thread Paul Smith
Thanks for the JIRA link Robert, I've added a comment to it just to share the real world aspect of what happened to us for background. On 1 April 2014 18:29, Robert Muir wrote: > On Tue, Apr 1, 2014 at 2:41 AM, Paul Smith wrote: > > > > Thanks Robert for the reply, all o

Re: Removing unused fields (more Lucene than ES but..)

2014-03-31 Thread Paul Smith
On 1 April 2014 15:23, Robert Muir wrote: > It is actually possible in lucene 4, but there is nothing really > convenient setup to do this. > > You have two choices there: > 1. trigger a massive merge (essentially an optimize), by wrapping all > readers and calling IndexWriter.addIndexes(Reader..

Re: rebuilding replicas

2014-02-05 Thread Paul Smith
You can issue a Move allocation command of just the replicas. A single move request with each of your replica shards shunted to a different node should do it. The move is implemented as a copy from the primary (well it is in ES 0.19 anyway) Note though that if you move things aroun such that a pos

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Paul Smith
if you do use it, don't forget we build for ES 0.19, so change the pom.xml to your ES version otherwise it won't connect... :) On 31 January 2014 12:29, Paul Smith wrote: > if it helps at all, i've pushed the flappy item detector tool (*cough*) > here: > > h

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Paul Smith
odified to suit your purpose perhaps. cheers, Paul On 31 January 2014 11:59, Paul Smith wrote: > the flappy detection tool I have connects to the cluster using the > standard java autodiscovery mechanism, and, works out which shards are > involved, and then creates explicit TransportC

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Paul Smith
27;flappy' (appearing/disappearing depending on the shard >> being searched) is supposed to be in there or not, perhaps checking the >> data store that is the source of the index (is it a dB?). >> >> We have very infrequent case where the replica shard is not properly >

Re: Primary vs. replica shard inconsistencies?

2014-01-30 Thread Paul Smith
of just the replica shards by shunting them around (rather than a full rolling restart of ALL the shards, just the shard replicas you want) cheers, Paul Smith On 31 January 2014 09:44, Binh Ly wrote: > Xavier, can you post an example of 1 full query and then also show how the > result

Query filtered by _version range?

2014-01-22 Thread Paul Smith
This has been asked before (I googled!), but I thought I'd seen some recent-or-maybe-not-that recent changes in ES about the special _version field and being able to do 'something' extra with it other than it's use as the Optimistic Locking pattern. Certainly in ES 0.19 one can't query it, but is