Thanks much. Will follow your suggestions. A quick question: If I have to install Hive, Sqoop and Flume on the my cluster nodes, what is the suggested way? --> Log into each cluster node and install them manually? --> Maybe the I have to hack the install script (/whirr/services/cdh/src/main/resources/functions) or create a new one for each of the tools?
Thanks, PD. On Sun, Dec 4, 2011 at 12:44 PM, Andrei Savu <[email protected]> wrote: > Here are a few advices: > > * use Ubuntu 10.04 LTS 64bit on the cluster nodes - this is the OS version > we are testing against before making a release > * whirr.hadoop.tarball.url is irrelevant if you specify > whirr.hadoop.install-function. Same is true for whirr.hadoop.version. > * install_cdh_hadoop will install by default the latest release. > We've implemented a way of selecting the version in 0.7.0: > https://issues.apache.org/jira/browse/WHIRR-415 > * use something bigger than t1.micro - this is just too small to run > Hadoop > > Let me know if you are still having problems. > > -- Andrei Savu / andreisavu.ro > > > On Sun, Dec 4, 2011 at 9:44 PM, Periya.Data <[email protected]> wrote: > >> Hi, >> I am trying Whirr to spin-up EC2 clusters. Here are the specs: >> >> >> - Local laptop - Ubuntu 11.10 Oneiric Ocelot + Hadoop-0.20.2-cdh3u2 >> - EC2 Whirr config : >> - >> - whirr.hadoop.install-function=install_cdh_hadoop >> - whirr.hadoop.configure-function=configure_cdh_hadoop >> - whirr.hadoop.version=0.20.2 >> - whirr.hadoop.tarball.url= >> http://archive.cloudera.com/cdh/3/hadoop-0.20.2-cdh3u2.tar.gz >> - whirr.hardware-id=t1.micro >> # Ubuntu 11.10 Oneiric EBS boot from alestic >> - whirr.image-id=us-east-1/ami-a7f539ce >> - whirr.location-id=us-east-1 >> >> >> *Problem:* The cluster seems to be up and running. But, cannot see Java >> and Hadoop on the instance. >> >> *Question:* Is this a problem or should I clearly mention in my Whirr >> config/properties file what version of JDK to be installed and its download >> URL? >> >> >> *Some logs:* (I ssh-ed into one of the nodes and tried to see the logs) >> >> sri@domU-12-31-39-02-21-20:/tmp$ ls -al >> total 40 >> drwxrwxrwt 4 root root 4096 2011-12-04 06:25 . >> drwxr-xr-x 22 root root 4096 2011-12-04 05:46 .. >> -rw-r--r-- 1 root root 938 2011-12-04 05:46 core-site.xml >> -rw-r--r-- 1 root root 701 2011-12-04 05:46 hdfs-site.xml >> drwxr-xr-x 2 root root 4096 2011-12-04 05:46 >> jclouds-script-1322977554131 >> drwxr-xr-x 2 root root 4096 2011-12-04 05:44 logs >> -rw-r--r-- 1 root root 1447 2011-12-04 05:46 mapred-site.xml >> -rwxr--r-- 1 root root 11212 2011-12-04 05:44 setup-sri.sh >> sri@domU-12-31-39-02-21-20:/tmp$ >> sri@domU-12-31-39-02-21-20:/tmp$ >> sri@domU-12-31-39-02-21-20:/tmp$ >> sri@domU-12-31-39-02-21-20:/tmp$ cd logs/ >> sri@domU-12-31-39-02-21-20:/tmp/logs$ ls -l >> total 12 >> -rw-r--r-- 1 root root 117 2011-12-04 05:45 stderr.log >> -rw-r--r-- 1 root root 4535 2011-12-04 05:45 stdout.log >> sri@domU-12-31-39-02-21-20:/tmp/logs$ more stderr.log >> E: Package 'sun-java6-jdk' has no installation candidate >> /tmp/setup-sri.sh: line 161: java: command not found >> sri@domU-12-31-39-02-21-20:/tmp/logs$ >> >> ---------- >> >> Get:36 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe >> i386 Packages [57.2 kB] >> Get:37 http://us-east-1.ec2.archive.ubuntu.comoneiric-updates/multiverse >> i386 Packages [4396 B] >> Get:38 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/main >> TranslationIndex [73 B] >> Get:39 http://us-east-1.ec2.archive.ubuntu.comoneiric-updates/multiverse >> TranslationIndex [72 B >> ] >> Get:40 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe >> TranslationIndex [73 B] >> Get:41 http://us-east-1.ec2.archive.ubuntu.com oneiric/main >> Translation-en [701 kB] >> Hit http://us-east-1.ec2.archive.ubuntu.com oneiric/multiverse >> Translation-en >> Get:42 http://us-east-1.ec2.archive.ubuntu.com oneiric/universe >> Translation-en [3165 kB] >> Get:43 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/main >> Translation-en [99.2 kB] >> Get:44 http://us-east-1.ec2.archive.ubuntu.comoneiric-updates/multiverse >> Translation-en [2419 B >> ] >> Get:45 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe >> Translation-en [35.4 kB] >> Fetched 16.2 MB in 12s (1290 kB/s) >> Reading package lists... >> Reading package lists... >> Building dependency tree... >> Reading state information... >> Package sun-java6-jdk is not available, but is referred to by another >> package. >> This may mean that the package is missing, has been obsoleted, or >> is only available from another source >> >> sri@domU-12-31-39-02-21-20:/tmp/logs$ >> Get:36 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe >> i386 Packages [57.2 kB] >> Get:37 http://us-east-1.ec2.archive.ubuntu.comoneiric-updates/multiverse >> i386 Packages [4396 B] >> Get:38 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/main >> TranslationIndex [73 B] >> Get:39 http://us-east-1.ec2.archive.ubuntu.comoneiric-updates/multiverse >> TranslationIndex [72 B] >> Get:40 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe >> TranslationIndex [73 B] >> Get:41 http://us-east-1.ec2.archive.ubuntu.com oneiric/main >> Translation-en [701 kB] >> Hit http://us-east-1.ec2.archive.ubuntu.com oneiric/multiverse >> Translation-en >> Get:42 http://us-east-1.ec2.archive.ubuntu.com oneiric/universe >> Translation-en [3165 kB] >> Get:43 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/main >> Translation-en [99.2 kB] >> Get:44 http://us-east-1.ec2.archive.ubuntu.comoneiric-updates/multiverse >> Translation-en [2419 B] >> Get:45 http://us-east-1.ec2.archive.ubuntu.com oneiric-updates/universe >> Translation-en [35.4 kB] >> Fetched 16.2 MB in 12s (1290 kB/s) >> Reading package lists... >> Reading package lists... >> Building dependency tree... >> Reading state information... >> Package sun-java6-jdk is not available, but is referred to by another >> package. >> This may mean that the package is missing, has been obsoleted, or >> is only available from another source >> >> sri@domU-12-31-39-02-21-20:/tmp/logs$ >> >> *[On a different cluster spun-up at a later time]* >> >> sri@domU-12-31-39-10-6D-38:~$ java >> The program 'java' can be found in the following packages: >> * gcj-4.4-jre-headless >> * gcj-4.6-jre-headless >> * openjdk-6-jre-headless >> * gcj-4.5-jre-headless >> * openjdk-7-jre-headless >> Ask your administrator to install one of them >> sri@domU-12-31-39-10-6D-38:~$ java -version >> The program 'java' can be found in the following packages: >> * gcj-4.4-jre-headless >> * gcj-4.6-jre-headless >> * openjdk-6-jre-headless >> * gcj-4.5-jre-headless >> * openjdk-7-jre-headless >> Ask your administrator to install one of them >> sri@domU-12-31-39-10-6D-38:~$ >> >> sri@domU-12-31-39-10-6D-38:~$ hadoop version >> hadoop: command not found >> sri@domU-12-31-39-10-6D-38:~$ >> >> =========================================== >> >> Finally, a quick Googling pointed to a JIRA issue: >> https://issues.apache.org/jira/browse/WHIRR-435 >> >> >> I am seeing the same stuff. Any updates? If this is a real issues, I do >> not have any problems in using Ubuntu 11.04 AMI for now. >> >> >> Your help is very much appreciated. >> >> >> Thanks, >> >> PD. >> > >
