[
https://issues.apache.org/jira/browse/TEZ-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383514#comment-14383514
]
Bikas Saha edited comment on TEZ-2242 at 3/27/15 8:24 AM:
----------------------------------------------------------
The patch refactors ShuffleVertexManager
1) moves initialization code (e.g.) checking egde properties etc. from
intiializer() to onVertexStarted(). Before a vertex starts, the edges could be
un-initialized.
2) Adds checks in the receiving methods of VertexStatus events to allow for
derived classes to forward events to this as a base class.
Pig's use case is to create a vertex manager that derives from shuffle vertex
manager. The vertex manager sets an initial parallelism and that allows the
vertex to start. After that it delegates to shuffle vertex manager to further
do auto reduce. Change 2 allows the derived class to forward all events to
shuffle vertex manager without bothering about which events can be handled.
Change 1 fixes the issue exposed in current code while using the derived class.
The shuffle vertex manager would read num tasks/edge types during initialize()
when they would not be valid.
[~rajesh.balamohan] Please review. The test failure is unrelated and the
findbugs warning is bogus.
was (Author: bikassaha):
The patch refactors ShuffleVertexManager
1) moves initialization code (e.g.) checking egde properties etc. from
intiializer() to onVertexStarted(). Before a vertex starts, the edges could be
un-initialized.
2) Adds checks in the receiving methods of VertexStatus events to allow for
derived classes to forward events to this as a base class.
Pig's use case is to create a vertex manager that derives from shuffle vertex
manager. The vertex manager sets an initial parallelism and that allows the
vertex to start. After that it delegates to shuffle vertex manager to further
do auto reduce. Change 2 allows the derived class to forward all events to
shuffle vertex manager without bothering about which events can be handled.
Change 1 fixes the issue exposed in current code while using the derived class.
The shuffle vertex manager would read num tasks/edge types during initialize()
when they would not be valid.
@rajesh balamoha
> Refactor ShuffleVertexManager code
> ----------------------------------
>
> Key: TEZ-2242
> URL: https://issues.apache.org/jira/browse/TEZ-2242
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-2242.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
