[
https://issues.apache.org/jira/browse/TEZ-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120151#comment-14120151
]
Hitesh Shah commented on TEZ-1345:
----------------------------------
Comments:
Regard patch 6:
- I am not sure this is correct. The root input events are sent to the
VertexManager which can potentially do some manipulation and then routes them
to the tasks. Logging the raw events from the InputInitializer is one thing but
that the actual events consumed by the tasks need not be the same as the ones
generated by the initializer.
For patch 5:
{code}
+ if (tezEvents.size() == 1 &&
+
tezEvents.get(0).getEventType().equals(EventType.VERTEX_MANAGER_INIT_DONE_EVENT))
{
+ org.apache.tez.dag.history.events.VertexManagerInitializationDone
historyEvent =
{code}
- why is there code in place to expect only one event to be sent? Could a
vertex manager not generate multiple events and then a done event at the end of
a collection? Is this not allowed?
- I don't see any vertex managers changed to generate the new event?
- Should startFromScratch() only be done for vertices which have root inputs
/ initializers? In all other cases, if there are no events being generated at
the start, this code path ok checking already initialized should not be needed
- correct?
- Whether to restart from scratch on the init event not being seen could
also be done by querying the VertexManager i.e. all VertexManagers should
implement a function saying something like isInitDoneEventRequired() ( may need
a better name ) to decide whether to restart from scratch if the event is not
seen.
> Add checks to guarantee all init events are written to recovery to consider
> vertex initialized
> ----------------------------------------------------------------------------------------------
>
> Key: TEZ-1345
> URL: https://issues.apache.org/jira/browse/TEZ-1345
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Hitesh Shah
> Assignee: Jeff Zhang
> Attachments: Tez-1345-2.patch, Tez-1345-3.patch, Tez-1345-4.patch,
> Tez-1345-5.patch, Tez-1345-6.patch, Tez-1345.patch
>
>
> Related to issue discovered in TEZ-1033
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
