Re: Why ES node starts recovering all the data from other nodes after reboot?

2014-11-22 Thread Yves Dorfsman
On 2014-11-22 09:35, Otis Gospodnetic wrote: Hi Konstantin, Check out http://gibrown.com/2014/11/19/elasticsearch-the-broken-bits/ Good writing! Thanks. I wonder if there's any drawback from cutting indices in smaller (tiny?) shards? My thinking is this: We don't really change data in our

Re: Why ES node starts recovering all the data from other nodes after reboot?

2014-11-21 Thread Yves Dorfsman
...@gmail.com mailto:markwal...@gmail.com wrote: It will enter recovery where it syncs at the segment level from the current primary, then the translog gets shipped over and (re)played, which brings it all up to date. On 21 November 2014 14:51, Yves Dorfsman y...@zioup.com

adding a new node: how to prime the data

2014-11-20 Thread Yves Dorfsman
We upgrade our clusters by adding new nodes, increase the number or replicas on the indices, let the new node catch up, then exclude the old node, and reduce the number of replicas on the indices. One cluster has a large index for which this operation takes hours. We tried to copy data from an

Re: Why ES node starts recovering all the data from other nodes after reboot?

2014-11-20 Thread Yves Dorfsman
If you do disable allocation before you reboot a node and a client writes to a shard that had a replica on that node, does the entire replica gets copied when the node come up? Or does it get just updated? On Thursday, 20 November 2014 19:52:26 UTC-7, Mark Walkom wrote: You should disable

Re: priming data for a new node

2014-11-20 Thread Yves Dorfsman
So if a shard has been updated since the data copy, will it copy the entire shard, or just update it? On Wednesday, 19 November 2014 23:34:01 UTC-7, Mark Walkom wrote: It doesn't copy everything, only what it needs to balance the shards. On 20 November 2014 17:20, Yves Dorfsman yv

upgrading from 0.90.7 to 1.4. Gotchas?

2014-11-19 Thread Yves Dorfsman
Are there any precautions to take before upgrading from 0.9 to 1.4? Different data types? Different API calls? etc... And, what is the best way to upgrade? Can we just add a node at the newer version and let it pull the data? Thanks. http://yves.zioup.com gpg: 4096R/32B0F416 -- You

priming data for a new node

2014-11-19 Thread Yves Dorfsman
When adding a new node to a cluster, is there a way to prevent it from having to copy all the data from the other nodes? We tried to copy the data on disk from an existing node (one that had all the data for the given indices), but it still copied everything. Is there a way to make it update what

Is it possible to isolate search quesries to a single node

2014-03-10 Thread Yves Dorfsman
I have a job that makes heavy use to ES, to the point that it affects the cluster. Is it possible to: - add a replica - force the extra replica to a specific node - isolate some of the queries to that particular node? Thanks. -- You received this message because you are subscribed to

Can a replica be updated with the deltas only?

2014-03-10 Thread Yves Dorfsman
When I shutdown a node that holds a replica and updates are happening to the rest of the cluster, then re-start this node, it seems that the entire replica is being copied again to that node. Is there a way to make ES just update that node with the updates that happened while it was down?