[
https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894136#action_12894136
]
Ning Zhang commented on HIVE-1422:
----------------------------------
Counters are used in for reporting fatal errors from tasks to Driver so that it
can kill the job. Currently fatal errors can be thrown from dynamic partition
(when there are too many partitions created by a mapper) or map-side join when
the threshold of # of rows of the small table has encountered. I think the
fatal error mechanism is only enabled for the first case (DP) by default.
> skip counter update when RunningJob.getCounters() returns null
> --------------------------------------------------------------
>
> Key: HIVE-1422
> URL: https://issues.apache.org/jira/browse/HIVE-1422
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: Joydeep Sen Sarma
> Fix For: 0.7.0
>
> Attachments: HIVE-1422.1.patch
>
>
> Under heavy load circumstances on some Hadoop versions, we may get a NPE from
> trying to dereference a null Counters object. I don't have a unit test which
> can reproduce it, but here's an example stack from a production cluster we
> saw today:
> 10/06/21 13:01:10 ERROR exec.ExecDriver: Ended Job = job_201005200457_701060
> with exception 'java.lang.NullPointerException(null)'
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.Operator.updateCounters(Operator.java:999)
> at
> org.apache.hadoop.hive.ql.exec.ExecDriver.updateCounters(ExecDriver.java:503)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:390)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:697)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
