AWS Private and Public Ip

Hi, I am using Elastic Ip Address and assigned Elastic Ip Address to the instances. After doing lot of trial and error , I could atleast install Cloudera Manager on AWS. But a very starnge thing I have notice, I am not sure if I am doing something wrong . When I have installed CM on AWS on an

Re: Cloudera Agent not starting

2015-03-12 14:39 GMT-03:00 Krish Donald gotomyp...@gmail.com: Firewall is not running [root@ec2-52-10-19-232 ~]# netstat -anp | grep 9000 [root@ec2-52-10-19-232 ~]# service iptables status iptables: Firewall is not running. On Thu, Mar 12, 2015 at 7:36 AM, Gangavarupu, Venkata

Re: How to create hdfs directory or file in a cluster which was created using Cloudera manager

://community.cloudera.com/ On Fri, Mar 13, 2015 at 6:15 PM, Krish Donald gotomyp...@gmail.com wrote: Hi, I have create a cluster using Cloudera Manager. I have used id Hadoop while setting up cluster which has the name password on all the node. After installation while tried to list

How to create hdfs directory or file in a cluster which was created using Cloudera manager

Hi, I have create a cluster using Cloudera Manager. I have used id Hadoop while setting up cluster which has the name password on all the node. After installation while tried to list the Hadoop filesystem using Hadoop user, I am getting below uoutout: [hadoop@dn1 ~]$ hadoop fs -ls / -bash:

Re: Cloudera Agent not starting

-WlyX1peqQKxCuWyzgUJvP1Qs=-2aTyiJDKALsZBR88RlkVyf4ILC5rIdwBPTSJxMo1Nke= On Wed, Mar 11, 2015 at 11:54 PM, Krish Donald gotomyp...@gmail.com wrote: Hi, I am trying to setup cluster but getting below error: error: No socket could be created on ('nn1.amazonaws.com https

Re: Not able to ping AWS host

your VPC -- subnet -- does the route table have a internet gateway attached(that should have a gateway of 0.0.0.0/0 as well)... On Mon, Mar 9, 2015 at 10:23 PM, Krish Donald gotomyp...@gmail.com wrote: Yes security group has all open ports to 0.0.0.0 and yes cluster is under VPC On Mon, Mar 9

Not able to ping AWS host

Hi, I am trying to setup Hadoop cluster on AWS . After creating an instance, I got the public ip and dns. But I tried to ping it from my windows machine I am not able to ping it. I am not able to logon to machine using putty . It is saying Network timed out. Security group in the AWS cluster

Re: Not able to ping AWS host

are the instance inside of a VPC ?? On Mon, Mar 9, 2015 at 5:05 PM, Krish Donald gotomyp...@gmail.com wrote: Hi, I am trying to setup Hadoop cluster on AWS . After creating an instance, I got the public ip and dns. But I tried to ping it from my windows machine I am not able to ping it. I am

What skills to Learn to become Hadoop Admin

Hi, I would like to enter into Big Data world as Hadoop Admin and I have setup 7 nodes cluster using Ambari, Cloudera Manager and Apache Hadoop. I have installed the services like hive, oozie, zookeeper etc. I have done a web log integration using flume and twitter sentiment analysis. I wanted

Re: AWS Setting for setting up Hadoop cluster

Hadoop on Amazon EC2 instances and use the free tier for new members but you can also use Amazon EMR which is not free but is up and running in a couple of seconds... 2015-03-05 23:28 GMT+01:00 Krish Donald gotomyp...@gmail.com: Hi, I am tired of setting Hadoop cluster using my laptop which has

Re: t2.micro on AWS; Is it enough for setting up Hadoop cluster ?

, Jonathan Aquilina Founder Eagle Eye T On 2015-03-06 00:41, Krish Donald wrote: Hi, I am new to AWS and would like to setup Hadoop cluster using cloudera manager for 6-7 nodes. t2.micro on AWS; Is it enough for setting up Hadoop cluster ? I would like to use free service as of now. Please

AWS Setting for setting up Hadoop cluster

Hi, I am tired of setting Hadoop cluster using my laptop which has 8GB RAM. I tried 2gb for namenode and 1-1 gb for 3 datanoded so total 5gb I was using . And I was using very basic Hadoop services only. But it is so slow that I am not able to do anything on that. Hence I would like to try the

t2.micro on AWS; Is it enough for setting up Hadoop cluster ?

Hi, I am new to AWS and would like to setup Hadoop cluster using cloudera manager for 6-7 nodes. t2.micro on AWS; Is it enough for setting up Hadoop cluster ? I would like to use free service as of now. Please advise. Thanks Krish

Cloudera monitoring Services not starting

Hi, I have setup a 4 node cliuster , 1 namenode and 3 datanode using cloudera manager 5.2 . But it is not starting Cloudra Monitorinf service and for Hosts health it is showing unknown. How can I disable Monitoring service completely and work with only cluster different feature. Thanks Krish

Re: How to Backup and Restore various components of Hadoop ?

to another cluster, however I want to be able to restore in case a table is delete and replicated to the backup cluster. Thanks, Motty On 03/01/2015 10:09 PM, Krish Donald wrote: Hi, Wanted to know the backup and recovery strategy for your Hadoop cluster. How do you Backup and Restore various

Cloudera Manager Installation is failing

Hi, I am trying to install Cloudera manager but it is failing and below is the log file: I have uninstalled postgres and tried again but still the same error. [root@nncloudera cloudera-manager-installer]# more 5.start-embedded-db.log mktemp: failed to create file via template `/tmp/':

Re: Cloudera Manager Installation is failing

Thanks Rich On Mon, Mar 2, 2015 at 2:23 PM, Rich Haase rha...@pandora.com wrote: Try posting this question on the Cloudera forum. http://community.cloudera.com/ On Mar 2, 2015, at 3:21 PM, Krish Donald gotomyp...@gmail.com wrote: Hi, I am trying to install Cloudera manager

How to troubleshoot failed or stuck jobs

Hi, Wanted to understand, How to troubleshoot failed or stuck jobs ? Thanks Krish

How to find bottlenecks of the cluster ?

Hi, I wanted to understand, how should we find out the bottleneck of the cluster? Thanks Krish

Re: How to troubleshoot failed or stuck jobs

://search-hadoop.com/m/LgpTk2YLArE https://issues.apache.org/jira/browse/MAPREDUCE-6190 Cheers On Sun, Mar 1, 2015 at 8:41 PM, Krish Donald gotomyp...@gmail.com wrote: Hi, Wanted to understand, How to troubleshoot failed or stuck jobs ? Thanks Krish

How to Backup and Restore various components of Hadoop ?

Hi, Wanted to know the backup and recovery strategy for your Hadoop cluster. How do you Backup and Restore various components of Hadoop ? Thanks Krish

Re: How to troubleshoot failed or stuck jobs

, it is expected reducers should be pre-empted. Need to identify whether reducers are getting pre-empted. MRAppMaster log would help for some extent to analyze the issue. Thanks Regards Rohith Sharma K S *From:* Krish Donald [mailto:gotomyp...@gmail.com] *Sent:* 02 March 2015 11:09 *To:* user

Re: How to find bottlenecks of the cluster ?

are there in your data pipe line and how data flows in to system and the way its processed.There are several inbuilt tests like testDFSIO and terasort that you can run. -Nishan On Sun, Mar 1, 2015 at 9:45 PM, Krish Donald gotomyp...@gmail.com wrote: Hi, I wanted to understand, how should we

error: [Errno 113] No route to host cloudera

Hi, I tried hard to debug the issue but nothing worked. I am getting error: [Errno 113] No route to host cloudera in cloudera agent log file. Below are some output : [root@snncloudera cloudera-scm-agent]# more /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4

Cloudera Manager Version and release date table

Hi, I searched a lot in google but couldn't find really well documented about the cloudera manager version and release date table i.e. VersionRelease date CM 4.1 June 2013 CM 4.2 Nov 2013 etc. Above is the hypothetical date . Does anybody has data like this ? Thanks Krish

Backup of individual component of Hadoop ecosystem

Hi, As per my understanding we don't take backup of Hadoop cluster as the size is very large generally . However in case if somebody has dropped a table by mistake then how should we recover the data ? How to take backup of Hadoop ecosystem individual component. Thanks Krish

Default Block Size in HDFS

Hi, I have read somewhere that default block size in Hadoop 2.4 is 256MB . Is it correct ? In which version default block size was 128MB ? Thanks Krish

How to Tune Hadoop Cluster from Administrator prospective

Hi, How to Tune Hadoop Cluster from Administrator prospective ? What parameters we should consider etc? What to look for performance tuning ? Thanks Krish

Apache Sentry

Hi, Has anybody worked on Apache Sentry ? Is yes, what are the challenges, pros and cons ? Thanks Krish

Kerberos Security in Hadoop

Hi, Has anybody worked on Kerberos security on Hadoop ? Can you please guide me , any document link will be appreciated ? Thanks Krish

Re: Jobs for Hadoop Admin in SFO Bay Area

GMT-08:00 Krish Donald gotomyp...@gmail.com: Hi, Does any of you aware of the job opening for entry level Hadoop Admin in SFO Bay Area ? If yes, please let me know. Thanks Krish

Jobs for Hadoop Admin in SFO Bay Area

Hi, Does any of you aware of the job opening for entry level Hadoop Admin in SFO Bay Area ? If yes, please let me know. Thanks Krish

Interview Questions asked for Hadoop Admin

Hi, Does anybody has interview questions which was asked during their interview on Hadoop admin role? I found few on internet but if somebody who has attended the interview can give us an idea , that will be great. Thanks Krish

Interview Questions asked

Hi, Does anybody has interview questions which was asked during their interview on hadoop? I found few on internet but if somebody who has attended the interview can give us an idea , that will be great. Thanks Krish

Hadoop Cluster setup using Ambari - Few problems

Hi, I am trying to setup multimode cluster using Ambari. I have setup password less ssh using root . I have installed ambari server on the master node . Setup ntp, disabled iptables. 1. When I am going to masternode ipaddress:8080 and after logining using admin userid with admin password, I am

Any working VM of Apache Hadoop ?

Hi, I am looking for working VM of Apache Hadoop. Not looking for cloudera or Horton works VMs. If anybody has it and if they can share that would be great . Thanks Krish

Re: Able to access namenode UI through ip address but not using hostname

that is running the browser can resolve the hostname. Sent from Samsung tablet Original message From: Krish Donald gotomyp...@gmail.com Date: 14/01/2015 19:30 (GMT+02:00) To: user@hadoop.apache.org Subject: Re: Able to access namenode UI through ip address

Re: Able to access namenode UI through ip address but not using hostname

Hi Johny, I have all the entries for my namenode and datanode in the below format in /etc/hosts ipaddress hostname fully_qualified_hostname But still I am not sure why I am facing this issue. Thanks Krish On Tue, Jan 13, 2015 at 9:11 PM, johny casanova pcgamer2...@outlook.com wrote:

Error in make command for zeromq

Hi, I am trying to install and configure zeromq. While issuing make command I am getting below error: Anybody has an idea , how to solve below ? [hadoop@mynamenode zeromq]$ make Making all in src make[1]: Entering directory `/opt/hadoop/zeromq/src' make all-am make[2]: Entering directory

Able to access namenode UI through ip address but not using hostname

Hi, I don't know where to look into but I am able to access namenode UI through ip address but not using hostname . Please help. Thanks Krish

New to this group.

Hi, I am new to this group and hadoop. Please help me to learn hadoop and suggest some self study project . Thanks Krish Donald

Re: New to this group.

would agree it's great place to get immersed in the hadoop ecosystem. On Jan 2, 2015, at 1:05 PM, Krish Donald gotomyp...@gmail.com wrote: I would like to work on some kind of case studies like I have seen couple on Horton works like twitter sentiment analysis, web log analysis etc

Re: New to this group.

FYI On Fri, Jan 2, 2015 at 10:24 AM, Krish Donald gotomyp...@gmail.com wrote: I have fair understanding of hadoop eco system... I have setup multinode cluster using VMs in my personal laptop for Hadoop 2.0 . But beyond that i would like to work on some project to get a good hold

Re: New to this group.

: http://www.manning.com/lam2/ hadoop in practive: http://www.manning.com/holmes2/ A list is here: http://wiki.apache.org/hadoop/Books Hope this helps. Best wishes, Wilm Am 02.01.2015 um 19:02 schrieb Krish Donald: Hi, I am new to this group and hadoop. Please help me to learn

Re: New to this group.

higher level language like Pig. Marco On Fri, Jan 2, 2015 at 3:43 PM, Krish Donald gotomyp...@gmail.com wrote: I would like to go towards administration side not in development side as I don't know java at all...

Re: New to this group.

think you're going to get a good quick hit with anything that you can add to a CV. You likely need to think about a 6-9 month project that actually has some kind of measurable impact to impress anyone. My opinion... Marco On Jan 2, 2015, at 3:58 PM, Krish Donald gotomyp...@gmail.com wrote

Re: How to get Hadoop Admin job being fresher in Big Data Field?

Waiting for people input On Fri, Jan 2, 2015 at 2:26 PM, Krish Donald gotomyp...@gmail.com wrote: Hi, In this group we have so many experienced people and i wanted to know how to get a job as Hadoop admin being fresher in Big Data field. I have 13+ years of exp as DBA and now I would like

How to get Hadoop Admin job being fresher in Big Data Field?

Hi, In this group we have so many experienced people and i wanted to know how to get a job as Hadoop admin being fresher in Big Data field. I have 13+ years of exp as DBA and now I would like to move into Big Data field basically as Hadoop Admin as per my New Year resolution ;) I have fair