[
https://issues.apache.org/jira/browse/HIVE-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150330#comment-16150330
]
liyunzhang_intel commented on HIVE-17383:
-----------------------------------------
why say "The failures can't be reproduced locally.". Actually it can be
reproduced in my env. Do you mean in latest master, this is fixed?
Not very well understand the logic of vectorization.
why {{firstOutputColumnIndex}} starts from {{initialColumnNames.length}}. for
example,if have 1 column, the {{firstOutputColumnIndex}} is from 1( normally
the index is from 0). When we construct the output batch, the column is from
1, is this right?
{code}
// Convenient constructor for initial batch creation takes
// a list of columns names and maps them to 0..n-1 indices.
public VectorizationContext(String contextName, List<String>
initialColumnNames,
HiveConf hiveConf) {
this.contextName = contextName;
level = 0;
this.initialColumnNames = initialColumnNames;
this.projectionColumnNames = initialColumnNames;
projectedColumns = new ArrayList<Integer>();
projectionColumnMap = new HashMap<String, Integer>();
for (int i = 0; i < this.projectionColumnNames.size(); i++) {
projectedColumns.add(i);
projectionColumnMap.put(projectionColumnNames.get(i), i);
}
int firstOutputColumnIndex = projectedColumns.size();
this.ocm = new OutputColumnManager(firstOutputColumnIndex);
this.firstOutputColumnIndex = firstOutputColumnIndex;
vMap = new VectorExpressionDescriptor();
if (hiveConf != null) {
setHiveConfVars(hiveConf);
}
}
{code}
> ArrayIndexOutOfBoundsException in VectorGroupByOperator
> -------------------------------------------------------
>
> Key: HIVE-17383
> URL: https://issues.apache.org/jira/browse/HIVE-17383
> Project: Hive
> Issue Type: Bug
> Reporter: Rui Li
> Assignee: Rui Li
> Attachments: HIVE-17383.1.patch
>
>
> Query to reproduce:
> {noformat}
> set hive.cbo.enable=false;
> select count(*) from (select key from src group by key) s where s.key='98';
> {noformat}
> The stack trace is:
> {noformat}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupKeyHelper.copyGroupKey(VectorGroupKeyHelper.java:107)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeReduceMergePartial.doProcessBatch(VectorGroupByOperator.java:831)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processBatch(VectorGroupByOperator.java:174)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.process(VectorGroupByOperator.java:1046)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:462)
> ... 18 more
> {noformat}
> More details can be found in HIVE-16823
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
