[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532076#comment-14532076 ] TezQA commented on TEZ-2404: {color:red}-1 overall{color}. Here are the results of testing the

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532007#comment-14532007 ] Jeff Zhang commented on TEZ-2404: - Upload a new patch ( add some comments to highlight the e

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531953#comment-14531953 ] Jeff Zhang commented on TEZ-2404: - Need to add some comment, and will commit soon. > Handl

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531932#comment-14531932 ] Siddharth Seth commented on TEZ-2404: - [~zjffdu] - are any more changes required here ?

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531791#comment-14531791 ] Bikas Saha commented on TEZ-2404: - The failures that I were seeing are unrelated to this or

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531647#comment-14531647 ] Siddharth Seth commented on TEZ-2404: - Just tried the .2 patch out on several large jobs

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531163#comment-14531163 ] Siddharth Seth commented on TEZ-2404: - Sounds like changing {code} if (eventType == Even

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531146#comment-14531146 ] Bikas Saha commented on TEZ-2404: - No they were not. > Handle DataMovementEvent before its

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531133#comment-14531133 ] Siddharth Seth commented on TEZ-2404: - Were the jobs running with parallel dispatchers ?

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531054#comment-14531054 ] Bikas Saha commented on TEZ-2404: - You will have to find out by trying large jobs in a clust

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14530259#comment-14530259 ] Jeff Zhang commented on TEZ-2404: - bq. This does still give us most of the benefits of TEZ-2

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-06 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14530062#comment-14530062 ] Siddharth Seth commented on TEZ-2404: - I'm not completely aware of how the recovery code

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529528#comment-14529528 ] Bikas Saha commented on TEZ-2404: - Mixing some events going directly and some events going i

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-05 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529509#comment-14529509 ] Hitesh Shah commented on TEZ-2404: -- Given this jira and TEZ-2418, maybe it might be better

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-05 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529440#comment-14529440 ] Hitesh Shah commented on TEZ-2404: -- The basic assumption in recovery is that all events are

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529432#comment-14529432 ] Bikas Saha commented on TEZ-2404: - I am afraid the approach in the patch would be a regressi

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-05 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529348#comment-14529348 ] Hitesh Shah commented on TEZ-2404: -- Patch looks good to me though a recovery related test t

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526931#comment-14526931 ] Hitesh Shah commented on TEZ-2404: -- BUmping up priority as this means recovery is potential

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526916#comment-14526916 ] Bikas Saha commented on TEZ-2404: - TEZ-1897 is not enabled yet. So we dont have to fix this

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526449#comment-14526449 ] Jeff Zhang commented on TEZ-2404: - [~hitesh] [~bikassaha] [~pramachandran] Please help revie

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526420#comment-14526420 ] TezQA commented on TEZ-2404: {color:green}+1 overall{color}. Here are the results of testing th

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-04 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526365#comment-14526365 ] TezQA commented on TEZ-2404: {color:red}-1 overall{color}. Here are the results of testing the

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-03 Thread TezQA (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526319#comment-14526319 ] TezQA commented on TEZ-2404: {color:red}-1 overall{color}. Here are the results of testing the

[jira] [Commented] (TEZ-2404) Handle DataMovementEvent before its TaskAttemptCompletedEvent

2015-05-03 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526248#comment-14526248 ] Jeff Zhang commented on TEZ-2404: - After rethinking, method 2 doesn't work. Because after TE