[
https://issues.apache.org/jira/browse/TEZ-933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935754#comment-13935754
]
Hitesh Shah commented on TEZ-933:
---------------------------------
[~sseth] The set parallelism code actually expects a -1 to be set. Also, I am
not sure whether this will be an issue with edges like 1:1 where the framework
is expected to set parallelism depending on upstream vertex.
> Race in getting source / destination numTasks on an Edge
> --------------------------------------------------------
>
> Key: TEZ-933
> URL: https://issues.apache.org/jira/browse/TEZ-933
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Fix For: 0.4.0
>
> Attachments: TEZ-933.1.txt
>
>
> Edges rely on getting properties (specifically numTasks in this case) from
> the source or destination vertex.
> This can end up with an incorrect value being used depending on the state of
> the vertex - whether the vertex has been initialized, whether the parallelism
> has been changed etc.
> As an example
> {code}
> edgeManager.getNumSourceTaskPhysicalOutputs(destinationVertex.getTotalTasks(),
> sourceTaskIndex))
> {code}
> destinationVertex.getTotalTasks() may be incorrect if the destinationVertex
> hasn't yet been initialized. Alternately, this value can change based on
> setParallelism calls.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
