[
https://issues.apache.org/jira/browse/TEZ-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296556#comment-14296556
]
Bikas Saha commented on TEZ-2001:
---------------------------------
bq. maxTaskAttempts is set to 1 when pipelined shuffle is enabled. Need to
create additional jiras to enhance error handling.
IMO this seems to be a pretty severe restriction. Essentially the feature is
not fault-tolerant. Could you please outline what the problem is and what
potential solutions could be.
The DME per spill file - does the nth DME signify that files 1-N are all
available to download? So having the last DME is enough and the previous ones
are not needed. e.g. can it be the case that 3 DME represent non-contiguous
files?
> Support pipelined data transfer for ordered output
> --------------------------------------------------
>
> Key: TEZ-2001
> URL: https://issues.apache.org/jira/browse/TEZ-2001
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2001.1.patch, TEZ-2001.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
