Rohini Palaniswamy created TEZ-1040:
---------------------------------------
Summary: TezMerger hangs when io.sort.mb value is 1
Key: TEZ-1040
URL: https://issues.apache.org/jira/browse/TEZ-1040
Project: Apache Tez
Issue Type: Bug
Reporter: Rohini Palaniswamy
TestCombiner.testMultiCombinerUse() in pig test suite sets io.sort.mb to 1 for
testing purposes. Tez is stuck at a loop while running the task with
0.4.0-incubating and 0.5.0-incubating-SNAPSHOT in
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.computeBytesInMerges.
It runs fine with 0.3.0-incubating.
{code}
at java.lang.Long.valueOf(Long.java:557)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.computeBytesInMerges(TezMerger.java:814)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:571)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:541)
at
org.apache.tez.runtime.library.common.sort.impl.TezMerger.merge(TezMerger.java:164)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.mergeParts(DefaultSorter.java:1078)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.flush(DefaultSorter.java:633)
at
org.apache.tez.runtime.library.output.OnFileSortedOutput.close(OnFileSortedOutput.java:134)
- locked <7f03a5b90> (a
org.apache.tez.runtime.library.output.OnFileSortedOutput)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.close(LogicalIOProcessorRuntimeTask.java:331)
at
org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:394)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
{code}
To reproduce:
ant clean test -Dtestcase=TestCombiner -Dexectype=tez -Dhadoopversion=23
-Dtest.output=true -logfile /tmp/log
--
This message was sent by Atlassian JIRA
(v6.2#6252)
