[
https://issues.apache.org/jira/browse/TEZ-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated TEZ-864:
------------------------
Attachment: TEZ-864.1.patch
> PipelinedSorter throws BufferOverflow exception
> ------------------------------------------------
>
> Key: TEZ-864
> URL: https://issues.apache.org/jira/browse/TEZ-864
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Hadoop 2.3.0, Hive 0.13, Tez 0.3.0
> Reporter: Rajesh Balamohan
> Assignee: Gopal V
> Attachments: TEZ-864-all.log.gz, TEZ-864.1.patch
>
>
> When running the following query, BufferOverflowException is thrown at times.
> >>
> SELECT SUBSTR(sourceIP, 1, 10), SUM(adRevenue) FROM uservisits GROUP BY
> SUBSTR(sourceIP, 1, 10)
> >>
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.nio.BufferOverflowException
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:287)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.flush(VectorGroupByOperator.java:320)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:249)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:129)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
> ... 9 more
> Caused by: java.nio.BufferOverflowException
> at java.nio.Buffer.nextPutIndex(Buffer.java:513)
> at
> java.nio.ByteBufferAsIntBufferL.put(ByteBufferAsIntBufferL.java:122)
> at
> org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.collect(PipelinedSorter.java:237)
> at
> org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.write(PipelinedSorter.java:183)
> at
> org.apache.tez.runtime.library.output.OnFileSortedOutput$1.write(OnFileSortedOutput.java:96)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor$KVOutputCollector.collect(TezProcessor.java:170)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.collect(ReduceSinkOperator.java:364)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:270)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
