After 2 hard restarts of all nodes, the cluster is back. But 7 of 18 nodes are dead. the NN is aware of all nodes, data (of non-dead) nodes is available. We're pulling off data now, will give hardware back to the HW team next week. *the emails in response to my original are very helpful in learning more about Hadoop concepts.many thanksJohn
Date: Sat, 23 Mar 2013 13:45:18 +0800 Subject: Re: Cluster lost IP addresses From: azury...@gmail.com To: user@hadoop.apache.org oh,yes,it's not persist, only in memory. so there is no issue. On Mar 23, 2013 1:13 PM, "Harsh J" <ha...@cloudera.com> wrote: NameNode does not persist block locations; so this is still recoverable if the configs are changed to use the new set of hostnames to bind to/look up. On Sat, Mar 23, 2013 at 9:01 AM, Azuryy Yu <azury...@gmail.com> wrote: > it has issues, namenode save blockid->nodes, using ip addr if your slaves > config file using ip addr instead of hostname. > > On Mar 23, 2013 10:14 AM, "Balaji Narayanan (பாலாஜி நாராயணன்)" > <li...@balajin.net> wrote: >> >> Assuming you are using hostnAmes and not ip address in your config files >> What happens when you start the cluster? If you are using IP address in your >> configs just update them and start. It should work with no issues. >> >> On Friday, March 22, 2013, John Meza wrote: >>> >>> I have a 18 node cluster that had to be physically moved. >>> Unfortunately all the ip addresses were lost (recreated). >>> >>> This must have happened to someone before. >>> Nothing else on the machines has been changed. Most importantly the data >>> in HDFS is still sitting there. >>> >>> Is there a way to recover this cluster to a useable state? >>> thanks >>> John >> >> >> >> -- >> http://balajin.net/blog >> http://flic.kr/balajijegan -- Harsh J
