[ https://issues.apache.org/jira/browse/TEZ-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bikas Saha reopened TEZ-2599: ----------------------------- > Dont send obsoleted data movement events to tasks > ------------------------------------------------- > > Key: TEZ-2599 > URL: https://issues.apache.org/jira/browse/TEZ-2599 > Project: Apache Tez > Issue Type: Improvement > Reporter: Bikas Saha > Assignee: Bikas Saha > Fix For: 0.8.0 > > Attachments: TEZ-2599.1.patch, TEZ-2599.2.patch > > > Since events are sent in bulk and in sequential order to tasks they can end > up getting a bunch of data movement events in round 1 and then an input > failed event in round 2. The task may end up scheduling fetches for these > obsoleted events before round 2, leading to wastage. > Given that an input failed event and data movement event can be matched via > their source task attempt ids, the AM can use this match to obsolete the data > movements events ahead of time and not send them to the tasks in the first > place. > The input failed events still need to be sent to the tasks, so that they can > obsolete any data movements events that they may have received much earlier > from the failed task attempt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)