Hi Attila, I don't have any direct experience on this, but I'm sure I read that you can migrate shards from one server to another by specifying some kind of preference (in the elasticsearch.yml or for the index).
http://blog.sematext.com/2012/05/29/elasticsearch-shard-placement-control/ http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html This kind of action has been previously discussed regarding migrating old/less-used indexes from fast servers to ones with slower disks. With this in mind, would it not be easier to: * expand the existing 1-node cluster (eg another 3 high-spec servers) * ensure the existing index has at least 1 replica (or add a replica) * configure new indexes/shards to prefer the new servers * migrate the old shards to the new servers * shutdown old server Anything connecting to the ES cluster will either have to point to a new (loadbalanced?) IP pointing at the new nodes .... or can use the multicast (discovery) group address and same cluster name without changing anything. Assuming I am correct (that there is a way to do this), you end up with no affect to the ES-cluster service. Cheers Ivan On 19/02/2014 16:26, Attila Bukor wrote: > Hi Tony, > > Thank you for your response, reading this I'm pretty sure I'm going with > Yann's suggestion to simply backup & restore. Your first point confirms > that > this is a good idea anyway. > > To answer your second point, both clusters are on the same /24 subnet, the > problem is that oldcluster is not designed to do that, we put a production > site's index on a server with twenty-odd GB free space and 4GB RAM > total, so > we *need* to move it to the dedicated cluster. > > The other two points are great suggestions and thank you very much for > them, > but I think the backup & restore way would be more suitable for my case. > > Thank you very much, > Attila > > On 02/19/2014 04:50 PM, Tony Su wrote:> Hi, >> My testing has done a little bit of what you're describing... >> - FWIW and I don't know why, when I re-insert the data from scratch >> instead of upgrading a cluster, I'm seeing substantial improvements in >> node general health. They seem faster, less latencies related to >> indexing (I assume there should be no diff searching, but ??). If a >> reason why I'm seeing this can't be found, then maybe what I'm seeing is >> specific to my setup and hardware. >> - I don't know that there should be any reason to name your cluster >> differently if the networks are completely isolated, but of course if >> there should be a mistake, the consequences could be serious. >> - I've been using various methods that display index activity, the >> easiest to use (and today work on both 0.90 and 1.0) are >> elasticsearch-head and elasticsearch-hq. I've also been toying around >> recently with simply probing each node using a curl command, which >> depending on how you deploy web tools likely has a lower load effect on >> resources (I'm currently running Apache on one node and when I use one >> or more of the above tools, has a significant and measurable load). >> - If you do an upgrade, at least the one time after the upgrade you may >> want to force your cluster to wait until all nodes in the cluster are >> active before restarting to minimize shard thrashing (if a node is not >> online yet, the cluster may discard the indices on that node and >> re-allocate from whatever copy already is active. Then, when the missing >> node finally joins, then the shards are re-balanced). One way to do that >> is setting the following to be the same as the total nodes in the cluster >> gateway.recovery_after_nodes >> HTH, >> Tony >> >> On Wednesday, February 19, 2014 7:01:03 AM UTC-8, Attila Bukor wrote: >> >> Hey everybody, >> >> I have a question regarding the migration of the indices to a new >> cluster >> seamlessly. Let me first describe the situation: >> >> I have a project which uses Elasticsearch since a few weeks ago. >> This is our >> first Elasticsearch project, and as we are satisfied with it, we >> decided to >> dedicate a cluster of 3 nodes to run Elasticsearch (lets call it >> "newcluster"). It is up and running separately from our old 1-node >> Elasticsearch installation ("oldcluster"). We want to migrate >> everything to >> newcluster, and turn oldcluster off. All 4 servers are running >> Ubuntu Server, >> Elasticsearch is installed from the official deb package, > oldcluster is >> 0.90.10 and newcluster 1.0.0. >> >> I came up with the following solution, but I'm not sure it will work: >> >> - Upgrade oldcluster to 1.0.0 (tested it in dev, should work) >> - Backup the index (we have only one index right now) in case >> something goes >> wrong. >> - Change oldcluster's name to newcluster. >> - Wait until the nodes synchronize themselves (how can I check if >> they're in >> sync?) >> - Shut down oldcluster. >> >> Thank you very much, >> Attila -- Ivan Beveridge -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5308D0CB.3050609%40livejournalinc.com. For more options, visit https://groups.google.com/groups/opt_out.
