One reason I know for this error is not setting up HADOOP_HOME. It is right to not set this variable since it was deprecated and replaced with HADOOP_PREFIX and HADOOP_MAPRED_HOME. However, it seems like hive still has some haunting references to HADOOP_HOME causing this error, specially after the query has failed and it attempts to grab debugging info from the tasktracker URL.
As far as your job is concerned, the failure has nothing to do with this error. Check your tasktrackers logs to know what blew up. On Thu, Feb 7, 2013 at 5:33 AM, Viral Bajaria <viral.baja...@gmail.com>wrote: > Are you seeing this after a few of the jobs have finished or on the first > stage itself ? Also is this error on all boxes or just a few ? You can > check MR logs and see which box or boxes are the culprits and debug from > there. > > Viral > ------------------------------ > From: Krishna Rao > Sent: 2/7/2013 2:46 AM > To: user@hive.apache.org > Subject: NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil > > Hi all, > > I'm occasionally getting the following error, usually after running an > expensive Hive query (creating 20 or so MR jobs): > > *** > Error during job, obtaining debugging information... > Examining task ID: task_201301291405_1640_r_000001 (and more) from job > job_201301291405_1640 > Exception in thread "Thread-29" java.lang.NoClassDefFoundError: > org/apache/hadoop/mapreduce/util/HostUtil > at > org.apache.hadoop.hive.shims.Hadoop23Shims.getTaskAttemptLogUrl(Hadoop23Shims.java:51) > at > org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.getTaskInfos(JobDebugger.java:186) > at > org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.run(JobDebugger.java:142) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapreduce.util.HostUtil > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > ... 4 more > CmdRunner::runCmd: Error running cmd in script, error: FAILED: Execution > Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask > *** > > Any ideas on what's causing it? > How can I find out more info on this error? > > Cheers, > > Krishna > -- Swarnim
