[jira] [Updated] (TEZ-2209) Fix pipelined shuffle to fetch data from any one attempt
[ https://issues.apache.org/jira/browse/TEZ-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2209: -- Attachment: TEZ-2209.4.patch Thanks [~sseth]. Addressed the review comments in the latest patch. Will commit it once the precommit build succeeds. Fix pipelined shuffle to fetch data from any one attempt Key: TEZ-2209 URL: https://issues.apache.org/jira/browse/TEZ-2209 Project: Apache Tez Issue Type: Improvement Reporter: Rajesh Balamohan Assignee: Rajesh Balamohan Attachments: TEZ-2209.1.patch, TEZ-2209.2.patch, TEZ-2209.3.patch, TEZ-2209.4.patch - Currently, pipelined shuffle will fail-fast the moment it receives data from an attempt other than 0. This was done as an add-on check to prevent data being copied from speculated attempts. - However, in some scenarios (like LLAP), it could be possible that that task attempt gets killed even before generating any data. In such cases, attempt #1 or later attempts, would generate the actual data. - This jira is created to allow pipelined shuffle to download data from any one attempt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2209) Fix pipelined shuffle to fetch data from any one attempt
[ https://issues.apache.org/jira/browse/TEZ-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2209: -- Attachment: TEZ-2209.1.patch [~sseth] - Please review when you have time. Fix pipelined shuffle to fetch data from any one attempt Key: TEZ-2209 URL: https://issues.apache.org/jira/browse/TEZ-2209 Project: Apache Tez Issue Type: Improvement Reporter: Rajesh Balamohan Assignee: Rajesh Balamohan Attachments: TEZ-2209.1.patch - Currently, pipelined shuffle will fail-fast the moment it receives data from an attempt other than 0. This was done as an add-on check to prevent data being copied from speculated attempts. - However, in some scenarios (like LLAP), it could be possible that that task attempt gets killed even before generating any data. In such cases, attempt #1 or later attempts, would generate the actual data. - This jira is created to allow pipelined shuffle to download data from any one attempt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2209) Fix pipelined shuffle to fetch data from any one attempt
[ https://issues.apache.org/jira/browse/TEZ-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2209: -- Fix Version/s: 0.7.0 Fix pipelined shuffle to fetch data from any one attempt Key: TEZ-2209 URL: https://issues.apache.org/jira/browse/TEZ-2209 Project: Apache Tez Issue Type: Improvement Reporter: Rajesh Balamohan Assignee: Rajesh Balamohan Fix For: 0.7.0 Attachments: TEZ-2209.1.patch - Currently, pipelined shuffle will fail-fast the moment it receives data from an attempt other than 0. This was done as an add-on check to prevent data being copied from speculated attempts. - However, in some scenarios (like LLAP), it could be possible that that task attempt gets killed even before generating any data. In such cases, attempt #1 or later attempts, would generate the actual data. - This jira is created to allow pipelined shuffle to download data from any one attempt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2209) Fix pipelined shuffle to fetch data from any one attempt
[ https://issues.apache.org/jira/browse/TEZ-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2209: -- Attachment: TEZ-2209.2.patch Fix pipelined shuffle to fetch data from any one attempt Key: TEZ-2209 URL: https://issues.apache.org/jira/browse/TEZ-2209 Project: Apache Tez Issue Type: Improvement Reporter: Rajesh Balamohan Assignee: Rajesh Balamohan Attachments: TEZ-2209.1.patch, TEZ-2209.2.patch - Currently, pipelined shuffle will fail-fast the moment it receives data from an attempt other than 0. This was done as an add-on check to prevent data being copied from speculated attempts. - However, in some scenarios (like LLAP), it could be possible that that task attempt gets killed even before generating any data. In such cases, attempt #1 or later attempts, would generate the actual data. - This jira is created to allow pipelined shuffle to download data from any one attempt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2209) Fix pipelined shuffle to fetch data from any one attempt
[ https://issues.apache.org/jira/browse/TEZ-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2209: -- Attachment: TEZ-2209.3.patch Fixed minor issue in test case. Fix pipelined shuffle to fetch data from any one attempt Key: TEZ-2209 URL: https://issues.apache.org/jira/browse/TEZ-2209 Project: Apache Tez Issue Type: Improvement Reporter: Rajesh Balamohan Assignee: Rajesh Balamohan Attachments: TEZ-2209.1.patch, TEZ-2209.2.patch, TEZ-2209.3.patch - Currently, pipelined shuffle will fail-fast the moment it receives data from an attempt other than 0. This was done as an add-on check to prevent data being copied from speculated attempts. - However, in some scenarios (like LLAP), it could be possible that that task attempt gets killed even before generating any data. In such cases, attempt #1 or later attempts, would generate the actual data. - This jira is created to allow pipelined shuffle to download data from any one attempt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
