Re: collecting CPU, mem, iops of hadoop jobs

2012-01-03 Thread Chris Smith
Have a look at OpenTSDB (http://opentsdb.net/overview.html) as this does not have the same down sampling issue as Ganglia and stores the metrics in HBase making it easier to access and process the data. It's also pretty easy to add your own metrics. Another useful utility is 'collectl' (http://col

Eclipse plugin and Hadoop 1.0.0

2012-01-03 Thread chris 0
Has anyone managed to get the eclipse plugin working with Hadoop 1.0.0, I keep getting errors such as: Caused by: java.io.IOException: Call to /192.168.1.200:50010 failed on local exception: java.io.EOFException cheers Chris

Re: Eclipse plugin and Hadoop 1.0.0

2012-01-03 Thread Harsh J
Chris, Did you build the plugin yourself or sourced it from somewhere else? Building your own plugin JAR and using that mostly always works. On 03-Jan-2012, at 6:25 PM, chris 0 wrote: > Has anyone managed to get the eclipse plugin > working with Hadoop 1.0.0, I keep getting > errors such as: >

Re: Eclipse plugin and Hadoop 1.0.0

2012-01-03 Thread chris 0
No I sourced the dev plugin from http://code.google.com/p/hadoop-eclipse-plugin/ So if I build it myself from that sourcecode, it'd probably work? cheers Chris On Tue, Jan 3, 2012 at 12:59 PM, Harsh J wrote: > Chris, > > Did you build the plugin yourself or sourced it from somewhere else? > Bu

Re: Eclipse plugin and Hadoop 1.0.0

2012-01-03 Thread Harsh J
Chris, You'd need to build from your distribution's src/contrib/eclipse source directory. You can also reuse a jar that's built at least for 0.20.205, but am not aware of a pre-built location presently. Do check the $HADOOP_HOME/contrib directory if it is already present in the downloaded distribu

Re: Eclipse plugin and Hadoop 1.0.0

2012-01-03 Thread Harsh J
Here is a build against Eclipse 3.7 (May work with other versions), and Hadoop 1.0.0: http://dl.dropbox.com/u/24999702/Apache/hadoop-eclipse-plugin-1.0.0.jar Should work. On 03-Jan-2012, at 6:48 PM, Harsh J wrote: > Chris, > > You'd need to build from your distribution's src/contrib/eclipse >

Re: Dynamically adding nodes in Hadoop

2012-01-03 Thread madhu phatak
Thanks for all the input. I am trying to do cluster setup in EC2 but not able to find how i can do dns updation centrally. If anyone one knows how to do this please help me .. On Sat, Dec 17, 2011 at 8:10 PM, Michel Segel wrote: > Actually I would recommend avoiding /etc/hosts and using DNS if th

Re: Allowing multiple users to submit jobs in hadoop 0.20.205 ?

2012-01-03 Thread Praveen Sripati
By default `security.job.submission.protocol.acl` is set to * in the hadoop-policy.xml, so it will allow any/multiple users to submit/query job status. Check this (1) for more details. security.job.submission.protocol.acl *ACL for JobSubmissionProtocol, used by job clients to commu

Re: Hive starting error

2012-01-03 Thread Praveen Sripati
http://hive.apache.org/releases.html#21+June%2C+2011%3A+release+0.7.1+available > 21 June, 2011: release 0.7.1 available > This release is the latest release of Hive and it works with Hadoop 0.20.1 and 0.20.2 I don't see the method the method thrown in the exception in 0.20.205. Praveen On Fri

Re: Allowing multiple users to submit jobs in hadoop 0.20.205 ?

2012-01-03 Thread Harsh J
Could you detail what _doesn't_ work when you try to run jobs as multiple users? The policies are only used when security is enabled. Is security enabled and if so, properly configured? On 03-Jan-2012, at 8:08 PM, Praveen Sripati wrote: > By default `security.job.submission.protocol.acl` is set

RE: Eclipse plugin and Hadoop 1.0.0

2012-01-03 Thread Will L
Hello, I am not able to get this to work under OS X Lion using Eclipse 3.7.1. I have filed a JIRA issue for this including the screenshot of the error I get. There are not issues with this under OS X Snow Leopard using any version of Eclipse 3.6 or higher. Please see: https://issues.apache.or