[ https://issues.apache.org/jira/browse/HIVE-26436?focusedWorklogId=800267&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-800267 ]
ASF GitHub Bot logged work on HIVE-26436: ----------------------------------------- Author: ASF GitHub Bot Created on: 12/Aug/22 10:29 Start Date: 12/Aug/22 10:29 Worklog Time Spent: 10m Work Description: ganlute commented on code in PR #3485: URL: https://github.com/apache/hive/pull/3485#discussion_r944327963 ########## ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java: ########## @@ -675,6 +678,13 @@ public void close(boolean abort) throws HiveException { return; } + // if the operator has not been initialized, there is no need to close + if (!this.rootInitializeCalled) { + LOG.debug("No need to close operator {}", this); + return; + } + + Review Comment: > why this is specific to MR and doesn't happen with Tez? And what is the actual scenario/query where this happens, the extended test doesn't seems to be very indicative @anishek I am so sorry for the slow reply. In fact, I can not reproduce the problem. It happened in one of my online clusters and I can not reproduce it in my new cluster. Some of configs may be different but I am not sure which one cause the problem. Here is the map file printed when exception happened: ```text <MAP>Id =0 <Children> <TS>Id =156 <Children> <UNION>Id =26 <Children> <SEL>Id =40 <Children> <FS>Id =146 <Children> <\Children> <Parent>Id = 40 null<\Parent> <\FS> <\Children> <Parent>Id = 26 null<\Parent> <\SEL> <\Children> <Parent>Id = 156 nullId = 162 <TS>Id =162 <Children>null <\Children> <Parent><\Parent> <\TS>Id = 166 <TS>Id =166 <Children>null <\Children> <Parent><\Parent> <\TS><\Parent> <\UNION> <\Children> <Parent>Id = 0 null<\Parent> <\TS> <\Children> <Parent><\Parent> <\MAP> ``` However, I think for whatever reason the operator is not initialized, close should not throw an exception. Issue Time Tracking ------------------- Worklog Id: (was: 800267) Time Spent: 1h 20m (was: 1h 10m) > Hive on MR NullPointerException > ------------------------------- > > Key: HIVE-26436 > URL: https://issues.apache.org/jira/browse/HIVE-26436 > Project: Hive > Issue Type: Bug > Components: Query Planning > Affects Versions: 3.1.2 > Reporter: TE > Priority: Minor > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > 2022-07-27 11:23:26,337 INFO [AsyncDispatcher event handler] > org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics > report from attempt_1658508427196_17228_m_000000_3: Error: > java.lang.RuntimeException: Hive Runtime Error while closing operators > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:211) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) > at > org.apache.hadoop.hive.ql.exec.mr.ExecMapRunner.run(ExecMapRunner.java:37) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168) > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.Operator.getCounterName(Operator.java:502) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:742) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:757) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:757) > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:193) > ... 9 more > > Hive on MR run failed and the error details is above. > Hive version: 3.1.2 > > When initializeOp has not been called and close called, this exception will > happen. -- This message was sent by Atlassian Jira (v8.20.10#820010)