we are getting a random (happening about 20% of the time, if we repeatedly
run the same query) error with hive 0.13.0.2
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:300)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:520)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:512)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:72)
Job Submission failed with exception 'java.lang.NullPointerException(null)'
from google search
https://www.google.com/search?q=hive+null+pointer+exception+.HiveInputFormat.init(HiveInputFormat.java%3A255)&oq=hive+null+pointer+exception++.HiveInputFormat.init(HiveInputFormat.java%3A255)&aqs=chrome..69i57.16859j0j7&sourceid=chrome&es_sm=93&ie=UTF-8
the top 2 results are pretty relevant:
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CB8QFjAA&url=http%3A%2F%2Fmail-archives.apache.org%2Fmod_mbox%2Fhive-user%2F201302.mbox%2F%253CCAKm%3DR7XyX52-AW5fm5N1vdtW30WQfsw-XAw5%3DmLqn55XeRNSJQ%40mail.gmail.com%253E&ei=FwjKU_rDAdPfoATbsIKgBA&usg=AFQjCNEfg7cyVbTxEl-HeZayFdWfb7Qn_w&sig2=M_0aiHBqrifHIChqiFWXrg&bvm=bv.71198958,d.cGU
this one suggests that it's due to resourcemanager being set to something
as localhost
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0CCkQFjAB&url=http%3A%2F%2Fmail-archives.apache.org%2Fmod_mbox%2Fhive-user%2F201308.mbox%2F%253CCAPt%2B2w-0mvzNEk8-9pdpsOag04uG%3D53WF_aRZXG68LsA3MfWGA%40mail.gmail.com%253E&ei=FwjKU_rDAdPfoATbsIKgBA&usg=AFQjCNG4uoRrzD8z1mHy35ztCmt7KRDVOw&sig2=LlreqybSuOurYtbJqRISUQ&bvm=bv.71198958,d.cGU
second one suggests that you have to set the resource manager instead of
leaving it empty.
since we do have a valid server value set to the
yarn.resourcemanager.address , and the above error only appears about 20%
time, does it mean that our resourcemanager is unstable?
thanks
Yang
