[jira] [Updated] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3700: -- Attachment: TEZ-3700.addendum.patch - Renamed "failedTaskIds" in VertexImpl to "failedTaskAttemptI

[jira] [Commented] (TEZ-3707) TezSharedExecutor race condition in awaitTermination vs isTerminated.

[ https://issues.apache.org/jira/browse/TEZ-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994158#comment-15994158 ] Siddharth Seth commented on TEZ-3707: - +1 > TezSharedExecutor race condition in awaitTe

[jira] [Commented] (TEZ-3705) Modify DeletionTracker and deletion threads to be initialized only if enabled for tez_shuffle

[ https://issues.apache.org/jira/browse/TEZ-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994156#comment-15994156 ] Siddharth Seth commented on TEZ-3705: - Thanks [~kshukla] Since TEZ_AM_DAG_CLEANUP_ON_COM

[jira] [Commented] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994153#comment-15994153 ] Siddharth Seth commented on TEZ-3700: - Sorry for coming in late here. Nit and rename req

[jira] [Commented] (TEZ-3698) UnorderedKV writer should be able to honor tez.runtime.enable.final-merge.in.output without pipelinedshuffle

[ https://issues.apache.org/jira/browse/TEZ-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994129#comment-15994129 ] Siddharth Seth commented on TEZ-3698: - In the close() method - !pipelinedShuffle, !final

[jira] [Commented] (TEZ-3696) Jobs can hang when both concurrency and speculation are enabled

[ https://issues.apache.org/jira/browse/TEZ-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994095#comment-15994095 ] Siddharth Seth commented on TEZ-3696: - Sequence of operations on the task with speculati

[jira] [Commented] (TEZ-3700) Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance

[ https://issues.apache.org/jira/browse/TEZ-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993996#comment-15993996 ] Rajesh Balamohan commented on TEZ-3700: --- Thanks for the review [~harishjp]. Will commi

[jira] [Commented] (TEZ-3696) Jobs can hang when both concurrency and speculation are enabled

[ https://issues.apache.org/jira/browse/TEZ-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993729#comment-15993729 ] Eric Badger commented on TEZ-3696: -- Thanks for the response, [~bikassaha]! bq. If the abo

[jira] [Updated] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3701: Affects Version/s: 0.9.0 > UnorderedPartitionedKVWriter does not wait for submitted tasks to finish >

[jira] [Updated] (TEZ-3701) UnorderedPartitionedKVWriter - issues with parallel Deflater usage, synchronousqueue in threadpool

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3701: Summary: UnorderedPartitionedKVWriter - issues with parallel Deflater usage, synchronousqueue in threa

[jira] [Commented] (TEZ-3701) UnorderedPartitionedKVWriter does not wait for submitted tasks to finish before finalMerge.

[ https://issues.apache.org/jira/browse/TEZ-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993726#comment-15993726 ] Siddharth Seth commented on TEZ-3701: - Thanks for the analysis [~harishjp]. Summarizing

[jira] [Commented] (TEZ-3673) Allocate smaller buffers in UnorderedPartitionedKVWriter

[ https://issues.apache.org/jira/browse/TEZ-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993720#comment-15993720 ] Siddharth Seth commented on TEZ-3673: - Looked. +1, after one small fix. The last buffer

[jira] [Updated] (TEZ-3705) Modify DeletionTracker and deletion threads to be initialized only if enabled for tez_shuffle

[ https://issues.apache.org/jira/browse/TEZ-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated TEZ-3705: - Attachment: TEZ-3705.004.patch This patch moves the ExecutorService to DeletionTrackerImpl. Renames the conf

[jira] [Commented] (TEZ-3696) Jobs can hang when both concurrency and speculation are enabled

[ https://issues.apache.org/jira/browse/TEZ-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993419#comment-15993419 ] Bikas Saha commented on TEZ-3696: - Thanks for the ping. Looking at the code again, I am not

[jira] [Commented] (TEZ-3705) Modify DeletionTracker and deletion threads to be initialized only if enabled for tez_shuffle

[ https://issues.apache.org/jira/browse/TEZ-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993193#comment-15993193 ] Siddharth Seth commented on TEZ-3705: - The check already exists in the ContainerLauncher

[jira] [Commented] (TEZ-3673) Allocate smaller buffers in UnorderedPartitionedKVWriter

[ https://issues.apache.org/jira/browse/TEZ-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993191#comment-15993191 ] Siddharth Seth commented on TEZ-3673: - Apologies for the delay. Will get to this today.

[jira] [Commented] (TEZ-3691) Move ShuffleScheduler to use the framework ExecutorService.

[ https://issues.apache.org/jira/browse/TEZ-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993190#comment-15993190 ] Siddharth Seth commented on TEZ-3691: - [~harishjp] - think we're better off moving the F

[jira] [Commented] (TEZ-3705) Modify DeletionTracker and deletion threads to be initialized only if enabled for tez_shuffle

[ https://issues.apache.org/jira/browse/TEZ-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993009#comment-15993009 ] Kuhu Shukla commented on TEZ-3705: -- {quote} if (ShuffleUtils.isTezShuffleHandler(getConfig(

[jira] [Commented] (TEZ-3691) Move ShuffleScheduler to use the framework ExecutorService.

[ https://issues.apache.org/jira/browse/TEZ-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992677#comment-15992677 ] TezQA commented on TEZ-3691: {color:red}-1 overall{color}. Here are the results of testing the

Failed: TEZ-3691 PreCommit Build #2409

Jira: https://issues.apache.org/jira/browse/TEZ-3691 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2409/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

[jira] [Commented] (TEZ-3707) TezSharedExecutor race condition in awaitTermination vs isTerminated.

[ https://issues.apache.org/jira/browse/TEZ-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992574#comment-15992574 ] Harish Jaiprakash commented on TEZ-3707: Test case failures unrelated to this patch.

[jira] [Updated] (TEZ-3691) Move ShuffleScheduler to use the framework ExecutorService.

[ https://issues.apache.org/jira/browse/TEZ-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated TEZ-3691: --- Attachment: TEZ-3691.02.patch Fixed test case failures. > Move ShuffleScheduler to use the fram

[jira] [Commented] (TEZ-3691) Move ShuffleScheduler to use the framework ExecutorService.

[ https://issues.apache.org/jira/browse/TEZ-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992516#comment-15992516 ] TezQA commented on TEZ-3691: {color:red}-1 overall{color}. Here are the results of testing the

Failed: TEZ-3691 PreCommit Build #2408

Jira: https://issues.apache.org/jira/browse/TEZ-3691 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2408/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated