NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil
Hi all, I'm occasionally getting the following error, usually after running an expensive Hive query (creating 20 or so MR jobs): *** Error during job, obtaining debugging information... Examining task ID: task_201301291405_1640_r_01 (and more) from job job_201301291405_1640 Exception in thread Thread-29 java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil at org.apache.hadoop.hive.shims.Hadoop23Shims.getTaskAttemptLogUrl(Hadoop23Shims.java:51) at org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.getTaskInfos(JobDebugger.java:186) at org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.run(JobDebugger.java:142) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.util.HostUtil at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 4 more CmdRunner::runCmd: Error running cmd in script, error: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask *** Any ideas on what's causing it? How can I find out more info on this error? Cheers, Krishna
RE: NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil
Are you seeing this after a few of the jobs have finished or on the first stage itself ? Also is this error on all boxes or just a few ? You can check MR logs and see which box or boxes are the culprits and debug from there. Viral -- From: Krishna Rao Sent: 2/7/2013 2:46 AM To: user@hive.apache.org Subject: NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil Hi all, I'm occasionally getting the following error, usually after running an expensive Hive query (creating 20 or so MR jobs): *** Error during job, obtaining debugging information... Examining task ID: task_201301291405_1640_r_01 (and more) from job job_201301291405_1640 Exception in thread Thread-29 java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil at org.apache.hadoop.hive.shims.Hadoop23Shims.getTaskAttemptLogUrl(Hadoop23Shims.java:51) at org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.getTaskInfos(JobDebugger.java:186) at org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.run(JobDebugger.java:142) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.util.HostUtil at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 4 more CmdRunner::runCmd: Error running cmd in script, error: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask *** Any ideas on what's causing it? How can I find out more info on this error? Cheers, Krishna
