Thank you all for helping me out, I was able to install a cluster successfully. It was in fact the problem that I was passing id_rsa.pub rather than id_rsa file. I did notice that cluster install would stall if I was to yum update the ambari server first. I did notice that there were some new HDP update files. If I would do a cluster install first, it works successfully. After the cluster install, I was prompted to do an HDP upgrade to 1.0.1, I"m doing that now and if it breaks, I will respond back. I'm very impressed with this project, hopefully it will be promoted to the top level project at some point. I'd like to know if I am able to setup an ambari node as an internet-connected node and the rest of the nodes as disconnected. Security is an issue for us. I did see some documentation for setting up a mirror. How would you guys recommend to go about with it? Setup an external internet attached server in the DMZ, I would think having that server run ambari would be a security risk?
Thanks. From: Suhas Vitthal Gogate <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, August 6, 2012 1:40 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Ambari initialize cluster step 2 issue Usually it should not take long time as add nodes would the basic discovery and sanity checks on the node. See if you have any errors in /var/log/hmc/hmc.log ? On Mon, Aug 6, 2012 at 10:33 AM, Artem Ervits <[email protected]<mailto:[email protected]>> wrote: Looks like I was doing everything wrong. I was pointing id_rsa.pub instead of id_rsa file. Doing ssh –I id_rsa root@namenode 'ls' " works. Now when I go through the install, after I click add nodes, it takes a long time, w/out any progress notifications. Is that normal? From: Suhas Vitthal Gogate <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Saturday, August 4, 2012 2:44 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Ambari initialize cluster step 2 issue Artem, looking at the following output you sent, looks like you still requires password input from user when run "ssh -I priv_key root@remote" from ambari server, Is it true? Could you ssh to root@remote without password, after protecting private key? (Sorry if I am repeating something others have already mentioned but in that case ignore this message. Just want to make sure we take care of basic things before look into further.. ). --Suhas On Fri, Aug 3, 2012 at 2:19 PM, Artem Ervits <[email protected]<mailto:[email protected]>> wrote: I just setup a one-way root passwordless ssh access from Ambari to the two other nodes. Still can't get it to work. I disabled iptables and SElinux on all machines. I found Hitesh's powerpoint on slideshare :), http://www.slideshare.net/Hadoop_Summit/managing-hadoop-clusters-with-ambari and am looking at the Hortonworks install guide as well. Result of ssh -I priv_key root@remote returned: [cid:[email protected]] In the error log, I found errors here: [2012:08:03 20:52:47][ERROR][sequentialScriptExecutor][sequentialScriptRunner.php:272][]: Encountered total failure in transaction 100 while running cmd: /usr/bin/php ./addNodes/findSshableNodes.php with args: demo root 41 100 42 /var/run/hmc/clusters/demo/hosts.txt My hosts.txt file looks like this: namenode datanode -----Original Message----- From: Jennie Cochran-Chinn [mailto:[email protected]<mailto:[email protected]>] Sent: Friday, August 03, 2012 4:49 PM To: [email protected]<mailto:[email protected]> Subject: Re: Ambari initialize cluster step 2 issue What if hduser has passwordless sudo? On 8/3/12 1:33 PM, "Hitesh Shah" <[email protected]<mailto:[email protected]>> wrote: >Yes - the ssh key needs to be for root. The node discovery and setup >phase involves installing rpms, etc that require root-level permissions. > >-- Hitesh > >On Aug 3, 2012, at 1:20 PM, Jennie Cochran-Chinn wrote: > >> Gotta be root user? Sounds like Artem is using hduser? >> >> On 8/3/12 1:16 PM, "Vitthal "Suhas" Gogate" >> <[email protected]<mailto:[email protected]>> >>wrote: >> >>> Looks like things are failing while contacting and doing sanity >>> checks for the nodes being added.. although I guess you need to use >>> the private rsa key to do the passwd-less ssh to remote nodes.. You >>> can manually try from ambari server ssh -i priv_key >>> root@remote_host to see that works.. >>> >>> >>> Also can you post the relevant part of the log at >>> /var/log/hmc/hmc.log to see what exact error you getting.. >>> >>> --Suhas >>> >>> On Fri, Aug 3, 2012 at 12:09 PM, Artem Ervits >>> <[email protected]<mailto:[email protected]>> wrote: >>>> Hello all, I'm trying to install a Hadoop cluster with Ambari on >>>>one VM and two additional nodes that will serve as a hadoop >>>>cluster. On step 2, where it says to add nodes, I select the >>>>id_rsa.pub file for the user I'm running the install with, let's >>>>call it hduser. I already copied the ssh keys to all machines and >>>>setup passwordless ssh to each and all machines. I also select >>>>/etc/hosts file that has all three VMs. When I hit next, it fails >>>>for some or all of the machines. I then created another file called >>>>hosts, containing the names of only two machines participating in a >>>>hadoop cluster, excluding the Ambari server. The add nodes step >>>>fails again due to permission denied again. It does, still include >>>>my Ambari server as the 3rd server to install hadoop on. Is there >>>>anything that I'm doing wrong? >>>> >>>> Thank you. >>>> >>>> This electronic message is intended to be for the use only of the >>>>named recipient, and may contain information that is confidential >>>>or privileged. If you are not the intended recipient, you are >>>>hereby notified that any disclosure, copying, distribution or use >>>>of the contents of this message is strictly prohibited. If you have >>>>received this message in error or are not the named recipient, >>>>please notify us immediately by contacting the sender at the >>>>electronic mail address noted above, and delete and destroy all >>>>copies of this message. Thank you. >>>> >>>> >>>> -------------------- >>>> >>>> This electronic message is intended to be for the use only of the >>>>named recipient, and may contain information that is confidential >>>>or privileged. If you are not the intended recipient, you are >>>>hereby notified that any disclosure, copying, distribution or use >>>>of the contents of this message is strictly prohibited. If you >>>>have received this message in error or are not the named recipient, >>>>please notify us immediately by contacting the sender at the >>>>electronic mail address noted above, and delete and destroy all >>>>copies of this message. Thank you. >>>> >>>> >>>> >>>> >>>> -------------------- >>>> >>>> This electronic message is intended to be for the use only of the >>>>named recipient, and may contain information that is confidential >>>>or privileged. If you are not the intended recipient, you are >>>>hereby notified that any disclosure, copying, distribution or use >>>>of the contents of this message is strictly prohibited. If you >>>>have received this message in error or are not the named recipient, >>>>please notify us immediately by contacting the sender at the >>>>electronic mail address noted above, and delete and destroy all >>>>copies of this message. Thank you. >>>> >>>> >>>> >> >> >> Please NOTE: This electronic message, including any attachments, may >>include privileged, confidential and/or inside information owned by >>Demand Media, Inc. Any distribution or use of this communication by >>anyone other than the intended recipient(s) is strictly prohibited and >>may be unlawful. If you are not the intended recipient, please notify >>the sender by replying to this message and then delete it from your >>system. Thank you. > Please NOTE: This electronic message, including any attachments, may include privileged, confidential and/or inside information owned by Demand Media, Inc. Any distribution or use of this communication by anyone other than the intended recipient(s) is strictly prohibited and may be unlawful. If you are not the intended recipient, please notify the sender by replying to this message and then delete it from your system. Thank you. ________________________________ This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. -------------------- This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. -------------------- This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. -------------------- This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. -------------------- This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. -------------------- This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. -------------------- This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
<<attachment: image001.png>>
