Error at at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
Hi All I wrote an MR job to fetch url content from web. Also I am using DBInputformat to fetch url and write the URL content to DB. When I execute the job I am getting the following error. Any clue on the cause or solution !!! 11/12/30 13:36:15 INFO mapred.JobClient: Task Id : attempt_201112300754_0004_m_00_1, Status : FAILED java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav -- ** JAGANADH G http://jaganadhg.in *ILUGCBE* http://ilugcbe.org.in
Hive starting error
Hi, I am using Hive 0.7.1 on hadoop 0.20.205 While running hive. its giving me following error : Exception in thread main java.lang.NoSuchMethodError: org.apache.hadoop.security.UserGroupInformation.login(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/security/UserGroupInformation; at org.apache.hadoop.hive.shims.Hadoop20Shims.getUGIForConf(Hadoop20Shims.java:448) at org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator.setConf(HadoopDefaultAuthenticator.java:51) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthenticator(HiveUtils.java:222) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:219) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:417) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Any Idea on how to resolve this issue ? Thanks, Praveenesh
Re: Automate Hadoop installation
Hey guys, If you're okay with Clojure, I've put together a Hadoop deploy using Pallet. Here's a demo project with instructions: https://github.com/pallet/pallet-hadoop-example I routinely boot and configure (from scratch) 50 node clusters on EC2 with no issues. Cheers, Sam On Thu, Dec 22, 2011 at 7:57 AM, Tom Wilcox tom.wil...@cse.org.uk wrote: I'm interested in this too. You might find this article helpful: http://hstack.org/hstack-automated-deployment-using-puppet/ Apparently the Adobe SaaS guys are responsible for creating a project for puppet-Hadoop stack deployments... -Original Message- From: warren [mailto:hadoop.com...@gmail.com] Sent: 22 December 2011 13:28 To: common-user@hadoop.apache.org Subject: Re: Automate Hadoop installation if you use ubuntu,you can try ubuntu-orchestra-modules-hadoop On 2011年12月06日 03:20, Konstantin Boudnik wrote: These that great project called BigTop (in the apache incubator) which provides for building of Hadoop stack. The part of what it provides is a set of Puppet recipes which will allow you to do exactly what you're looking for with perhaps some minor corrections. Serious, look at Puppet - otherwise it will be a living through nightmare of configuration mismanagements. Cos On Mon, Dec 05, 2011 at 04:02PM, praveenesh kumar wrote: Hi all, Can anyone guide me how to automate the hadoop installation/configuration process? I want to install hadoop on 10-20 nodes which may even exceed to 50-100 nodes ? I know we can use some configuration tools like puppet/or shell-scripts ? Has anyone done it ? How can we do hadoop installations on so many machines parallely ? What are the best practices for this ? Thanks, Praveenesh -- Sam Ritchie, Twitter Inc 703.662.1337 @sritchie09 (Too brief? Here's why! http://emailcharter.org)
