[
https://issues.apache.org/jira/browse/HIVE-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066093#comment-14066093
]
Matt McCline commented on HIVE-7421:
------------------------------------
Here is the explain output for query 47 with SPECIAL annotation showing the
VectorExpression(s):
{code}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
TableScan
alias: staples
Statistics: Num rows: 54860 Data size: 158216240 Basic stats:
COMPLETE Column stats: NONE
Filter Operator
predicate: (((concat(to_date(order_date_), ' 00:00:00') =
'1997-01-01 00:00:00') or (concat(to_date(order_date_), ' 00:00:00') =
'1997-01-03 00:00:00')) and ((to_date(order_date_) = '1997-01-01') or
(to_date(order_date_) = '1997-01-03'))) (type: boolean)
Statistics: Num rows: 54860 Data size: 158216240 Basic stats:
COMPLETE Column stats: NONE
vector filter expressions:
FilterExprAndExpr[-1](FilterExprOrExpr[-1](FilterStringColEqualStringScalar[-1](StringConcatColScalar[51](VectorUDFDateString[50]))
FilterStringColEqualStringScalar[-1](StringConcatColScalar[51](VectorUDFDateString[50])))
FilterExprOrExpr[-1](FilterStringColEqualStringScalar[-1](VectorUDFDateString[50])
FilterStringColEqualStringScalar[-1](VectorUDFDateString[50])))
Select Operator
expressions: order_priority (type: string)
outputColumnNames: order_priority
Statistics: Num rows: 54860 Data size: 158216240 Basic stats:
COMPLETE Column stats: NONE
vector select expressions: IdentityExpression[2]
Group By Operator
keys: order_priority (type: string)
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 54860 Data size: 158216240 Basic stats:
COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 54860 Data size: 158216240 Basic
stats: COMPLETE Column stats: NONE
Execution mode: vectorized
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 27430 Data size: 79108120 Basic stats: COMPLETE
Column stats: NONE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 27430 Data size: 79108120 Basic stats:
COMPLETE Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 27430 Data size: 79108120 Basic stats:
COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
{code}
> Null pointer exception involving
> ql.exec.vector.expressions.StringConcatColScalar.evaluate
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-7421
> URL: https://issues.apache.org/jira/browse/HIVE-7421
> Project: Hive
> Issue Type: Bug
> Reporter: Matt McCline
> Assignee: Matt McCline
> Attachments: TestWithORC.zip, fail_47.sql, fail_62.sql, fail_932.sql
>
>
> One of several found by Raj Bains.
> M/R or Tez.
> {code}
> set hive.vectorized.execution.enabled=true;
> {code}
> Seems very similar to https://issues.apache.org/jira/browse/HIVE-6649
> Query:
> {code}
> SELECT FLOOR((7 + DATEDIFF(`Staples`.`order_date_`,
> CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01 00:00:00'))
> +pmod(8 + pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING),
> '-01-01 00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) AS `wk_order_date_ok`,
> SUM(`Staples`.`sales_total`) AS `sum_sales_total_ok` FROM
> `default`.`testv1_Staples` `Staples` GROUP BY FLOOR((7 +
> DATEDIFF(`Staples`.`order_date_`, CONCAT(CAST(YEAR(`Staples`.`order_date_`)
> AS STRING), '-01-01 00:00:00')) +pmod(8 +
> pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01
> 00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) ;
> {code}
> Stack trace:
> {code}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setConcat(BytesColumnVector.java:190)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.StringConcatColScalar.evaluate(StringConcatColScalar.java:78)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFDateDiffColCol.evaluate(VectorUDFDateDiffColCol.java:59)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongScalarAddLongColumn.evaluate(LongScalarAddLongColumn.java:65)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongColAddLongColumn.evaluate(LongColAddLongColumn.java:52)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.LongColDivideLongScalar.evaluate(LongColDivideLongScalar.java:52)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.gen.FuncFloorDoubleToLong.evaluate(FuncFloorDoubleToLong.java:47)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.evaluateBatch(VectorHashKeyWrapperBatch.java:147)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:289)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
