[
https://issues.apache.org/jira/browse/TEZ-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025656#comment-14025656
]
Hitesh Shah commented on TEZ-1197:
----------------------------------
{code}
Caused by: java.io.IOException: Spill failed
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.checkSpillException(DefaultSorter.java:686)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.access$100(DefaultSorter.java:56)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter$Buffer.write(DefaultSorter.java:497)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter$Buffer.write(DefaultSorter.java:474)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:180)
at org.apache.hadoop.io.BytesWritable.write(BytesWritable.java:186)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:98)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:82)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.collect(DefaultSorter.java:275)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.write(DefaultSorter.java:185)
at
org.apache.tez.runtime.library.output.OnFileSortedOutput$1.write(OnFileSortedOutput.java:116)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor$TezKVOutputCollector.collect(TezProcessor.java:204)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.collect(ReduceSinkOperator.java:424)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:310)
... 21 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1241285372
at
org.apache.hadoop.io.FastByteComparisons$LexicographicalComparerHolder$UnsafeComparer.compareTo(FastByteComparisons.java:226)
at
org.apache.hadoop.io.FastByteComparisons$LexicographicalComparerHolder$UnsafeComparer.compareTo(FastByteComparisons.java:113)
at
org.apache.hadoop.io.FastByteComparisons.compareTo(FastByteComparisons.java:42)
at
org.apache.hadoop.io.WritableComparator.compareBytes(WritableComparator.java:158)
at org.apache.hadoop.hive.ql.io.HiveKey$Comparator.compare(HiveKey.java:80)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.compare(DefaultSorter.java:393)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:105)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:129)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:126)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:129)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:63)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.sortAndSpill(DefaultSorter.java:722)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter$SpillThread.run(DefaultSorter.java:655)
{code}
> Tasks failing with spill error
> ------------------------------
>
> Key: TEZ-1197
> URL: https://issues.apache.org/jira/browse/TEZ-1197
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Hitesh Shah
>
> Based on the logs from TEZ-1177, it was seen that some task attempts failed
> due to spill failure. Log files attached to TEZ-1177 have more details.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
