[jira] [Commented] (TEZ-3673) Allocate smaller buffers in UnorderedPartitionedKVWriter

[ https://issues.apache.org/jira/browse/TEZ-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988306#comment-15988306 ] TezQA commented on TEZ-3673: {color:green}+1 overall{color}. Here are the results of testing th

Success: TEZ-3673 PreCommit Build #2402

Jira: https://issues.apache.org/jira/browse/TEZ-3673 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2402/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Updated] (TEZ-3673) Allocate smaller buffers in UnorderedPartitionedKVWriter

[ https://issues.apache.org/jira/browse/TEZ-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated TEZ-3673: --- Attachment: TEZ-3673.04.patch Fixed test case failure in TezRuntimeConfiguration. The race condi

[jira] [Commented] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988231#comment-15988231 ] TezQA commented on TEZ-3700: {color:green}+1 overall{color}. Here are the results of testing th

Failed: TEZ-3700 PreCommit Build #2401

Jira: https://issues.apache.org/jira/browse/TEZ-3700 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2401/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-3673) Allocate smaller buffers in UnorderedPartitionedKVWriter

[ https://issues.apache.org/jira/browse/TEZ-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988218#comment-15988218 ] TezQA commented on TEZ-3673: {color:red}-1 overall{color}. Here are the results of testing the

Failed: TEZ-3673 PreCommit Build #2400

Jira: https://issues.apache.org/jira/browse/TEZ-3673 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2400/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Assigned] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-3701: - Assignee: Rajesh Balamohan (was: Harish Jaiprakash) > UnorderedPartitionedKVWriter does no

[jira] [Updated] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3700: -- Attachment: TEZ-3700.3.patch When debugging, encountered a deadlock in ShuffleScheduler. This is v

[jira] [Comment Edited] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988139#comment-15988139 ] Harish Jaiprakash edited comment on TEZ-3701 at 4/28/17 3:51 AM: -

[jira] [Commented] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988139#comment-15988139 ] Harish Jaiprakash commented on TEZ-3701: Found another issue, with use of Synchronou

Failed: TEZ-3703 PreCommit Build #2399

Jira: https://issues.apache.org/jira/browse/TEZ-3703 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2399/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-3703) Use a sha comparison to compare vertex and dag resources, if there is a mismatch

[ https://issues.apache.org/jira/browse/TEZ-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988067#comment-15988067 ] TezQA commented on TEZ-3703: {color:red}-1 overall{color}. Here are the results of testing the

[jira] [Commented] (TEZ-3698) UnorderedKV writer should be able to honor tez.runtime.enable.final-merge.in.output without pipelinedshuffle

[ https://issues.apache.org/jira/browse/TEZ-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988061#comment-15988061 ] Zhiyuan Yang commented on TEZ-3698: --- Change looks correct, but unit tests is not verifying

[jira] [Commented] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988062#comment-15988062 ] Harish Jaiprakash commented on TEZ-3701: Sorry [~jlowe]. Looks like a different issu

[jira] [Commented] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987986#comment-15987986 ] Harish Jaiprakash commented on TEZ-3701: Thanks [~jlowe]. I found few test cases fai

[jira] [Commented] (TEZ-3703) Use a sha comparison to compare vertex and dag resources, if there is a mismatch

[ https://issues.apache.org/jira/browse/TEZ-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987969#comment-15987969 ] Zhiyuan Yang commented on TEZ-3703: --- Patch looks good, except TestDAGVerify#testDAGWithCon

[jira] [Updated] (TEZ-3703) Use a sha comparison to compare vertex and dag resources, if there is a mismatch

[ https://issues.apache.org/jira/browse/TEZ-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3703: Attachment: TEZ-3703.01.patch Fairly straightforward patch. Main change is in DAG, and some methods mo

[jira] [Created] (TEZ-3703) Use a sha comparison to compare vertex and dag resources, if there is a mismatch

Siddharth Seth created TEZ-3703: --- Summary: Use a sha comparison to compare vertex and dag resources, if there is a mismatch Key: TEZ-3703 URL: https://issues.apache.org/jira/browse/TEZ-3703 Project: Apa

[jira] [Commented] (TEZ-3696) Jobs can hang when both concurrency and speculation are enabled

[ https://issues.apache.org/jira/browse/TEZ-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987603#comment-15987603 ] Eric Badger commented on TEZ-3696: -- {noformat} diff --git a/tez-dag/src/main/java/org/apac

[jira] [Commented] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987602#comment-15987602 ] Jason Lowe commented on TEZ-3701: - The code doesn't explicitly wait on the executor (e.g.: s

[jira] [Updated] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated TEZ-3701: Priority: Blocker (was: Major) Raising priority since any silent data corruption is catastrophically bad. [

[jira] [Resolved] (TEZ-3702) Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class

[ https://issues.apache.org/jira/browse/TEZ-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe resolved TEZ-3702. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: TEZ-3334 Thanks for the reviews! I comm

[jira] [Commented] (TEZ-3702) Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class

[ https://issues.apache.org/jira/browse/TEZ-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987230#comment-15987230 ] Jonathan Eagles commented on TEZ-3702: -- +1. > Tez shuffle jar includes service loader

[jira] [Commented] (TEZ-3702) Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class

[ https://issues.apache.org/jira/browse/TEZ-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987192#comment-15987192 ] Kuhu Shukla commented on TEZ-3702: -- Looks good to me. +1. > Tez shuffle jar includes servi

Success: TEZ-3696 PreCommit Build #2398

Jira: https://issues.apache.org/jira/browse/TEZ-3696 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2398/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-3696) Jobs can hang when both concurrency and speculation are enabled

[ https://issues.apache.org/jira/browse/TEZ-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987145#comment-15987145 ] TezQA commented on TEZ-3696: {color:green}+1 overall{color}. Here are the results of testing th

[jira] [Commented] (TEZ-3702) Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class

[ https://issues.apache.org/jira/browse/TEZ-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987112#comment-15987112 ] Jason Lowe commented on TEZ-3702: - TEZ-3383 is different since that's describing dependencie

[jira] [Commented] (TEZ-3702) Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class

[ https://issues.apache.org/jira/browse/TEZ-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986954#comment-15986954 ] Jonathan Eagles commented on TEZ-3702: -- Does this cover the case described in TEZ-3383

[jira] [Updated] (TEZ-3696) Jobs can hang when both concurrency and speculation are enabled

[ https://issues.apache.org/jira/browse/TEZ-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger updated TEZ-3696: - Attachment: TEZ-3696.002.patch Putting up a new patch with 2 new unit tests. 1 tests to make sure that we se

[jira] [Commented] (TEZ-3696) Jobs can hang when both concurrency and speculation are enabled

[ https://issues.apache.org/jira/browse/TEZ-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986868#comment-15986868 ] TezQA commented on TEZ-3696: {color:red}-1 overall{color}. Here are the results of testing the

Failed: TEZ-3696 PreCommit Build #2397

Jira: https://issues.apache.org/jira/browse/TEZ-3696 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2397/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Updated] (TEZ-3702) Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class

[ https://issues.apache.org/jira/browse/TEZ-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated TEZ-3702: Attachment: TEZ-3702.001.patch It doesn't look like the shuffle handler really needs the tez-mapreduce depend

[jira] [Created] (TEZ-3702) Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class

Jason Lowe created TEZ-3702: --- Summary: Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class Key: TEZ-3702 URL: https://issues.apache.org/jira/browse/TEZ-3702

Success: TEZ-3700 PreCommit Build #2396

Jira: https://issues.apache.org/jira/browse/TEZ-3700 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2396/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986600#comment-15986600 ] TezQA commented on TEZ-3700: {color:green}+1 overall{color}. Here are the results of testing th

[jira] [Updated] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3700: -- Attachment: TEZ-3700.2.patch > Consumer attempt should kill itself instead of failing during valid

[jira] [Created] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

Harish Jaiprakash created TEZ-3701: -- Summary: UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge. Key: TEZ-3701 URL: https://issues.apache.org/jira/browse/TEZ-3701

[jira] [Updated] (TEZ-3673) Allocate smaller buffers in UnorderedPartitionedKVWriter

[ https://issues.apache.org/jira/browse/TEZ-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated TEZ-3673: --- Attachment: TEZ-3673.03.patch * Changed config to buffer-merge-percent. * Sending a copy of fill

[jira] [Commented] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986278#comment-15986278 ] TezQA commented on TEZ-3700: {color:green}+1 overall{color}. Here are the results of testing th

Success: TEZ-3700 PreCommit Build #2395

Jira: https://issues.apache.org/jira/browse/TEZ-3700 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2395/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Updated] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3700: -- Attachment: TEZ-3700.1.patch > Consumer attempt should kill itself instead of failing during valid

[jira] [Commented] (TEZ-3690) Tez on hadoop 3 build failed due to hdfs client/server jar separation

[ https://issues.apache.org/jira/browse/TEZ-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986125#comment-15986125 ] Zhiyuan Yang commented on TEZ-3690: --- Thanks [~sseth]! Patch committed to master branch. {n

[jira] [Created] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

Rajesh Balamohan created TEZ-3700: - Summary: Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance Key: TEZ-3700 URL: https://issues.apache.org/jira/browse/TEZ-