Siddharth Seth created TEZ-1539:
-----------------------------------
Summary: Allow a FIRE_ONCE_ON_SUCCESS model for events generated
by user code
Key: TEZ-1539
URL: https://issues.apache.org/jira/browse/TEZ-1539
Project: Apache Tez
Issue Type: Improvement
Reporter: Siddharth Seth
Assignee: Siddharth Seth
Specifically for InputInitalizerEvents and VertexManagerEvents.
Pasting comment from TEZ-1447
In a majority of cases, events generated by different attempts of the same task
will be identical - in which case just making use of the event generated by the
first successful attempt is adequate. Doing something like this manes that
users don't worry about retries, indices etc - and can just rely on receiving a
set of events which are to be processed once the vertex succeeds.
If different attempts of the same workload generate different events -
processing is likely to be incorrect, since it's very possible for all data to
be processed (VERTEX successful), then a failure and retry - which generates a
different event. The initializer doesn't even run at this point, since it's
already done it's work and is complete. Handling such scenarios, likely
involves re-running the entire initializer and re-starting the vertex which
processed the event from scratch. In situations like this, where data generated
may be different, the best bet is for speculation to be disabled (when it's
supported), and max-attempts to be set to 1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
