[jira] [Created] (HIVE-23670) hive on spark may encounter error when use map join

[zhoukang (Jira)]

zhoukang created HIVE-23670:
-------------------------------

             Summary: hive on spark may encounter error when use map join
                 Key: HIVE-23670
                 URL: https://issues.apache.org/jira/browse/HIVE-23670
             Project: Hive
          Issue Type: Bug
    Affects Versions: 2.4.0
            Reporter: zhoukang


when using spark as hive engine, some task may failed with exception below.
The cause is that when we load table, we may issued with:

{code:java}
if (tableContainer == null) {
            tableContainer = useOptimizedContainer ?
                new MapJoinBytesTableContainer(hconf, valueContext, -1, 0) :
                create(name, metaData);
          }
{code}

then cast exception will issued with code below:

{code:java}
 @Override
    public JoinUtil.JoinResult setFromOther(ReusableGetAdaptor other) {
      assert other instanceof GetAdaptor;
      GetAdaptor other2 = (GetAdaptor)other;
      this.key = other2.key;
      this.isFirstKey = other2.isFirstKey;
      this.currentValue = mHash.get(key);
      if (this.currentValue == null) {
        return JoinUtil.JoinResult.NOMATCH;
      }
      else {
        return JoinUtil.JoinResult.MATCH;
      }
    }
{code}


{code:java}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected 
exception from MapJoinOperator : 
org.apache.hadoop.hive.ql.exec.persistence.MapJoinBytesTableContainer$GetAdaptor
 cannot be cast to 
org.apache.hadoop.hive.ql.exec.persistence.HashMapWrapper$GetAdaptor
        at 
org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:465)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
        at 
org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
        at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
        at 
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:148)
        at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:547)
        ... 19 more
{code}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)