[
https://issues.apache.org/jira/browse/TEZ-933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935812#comment-13935812
]
Siddharth Seth commented on TEZ-933:
------------------------------------
bq. Siddharth Seth The set parallelism code actually expects a -1 to be set.
Also, I am not sure whether this will be an issue with edges like 1:1 where the
framework is expected to set parallelism depending on upstream vertex.
Oops, comment / commit race. Let me see if anything breaks because of this. I
expect it not to because this is just setting the parallelism which was
configured on the vertex early (instead of potentially using the default of 0).
> Race in getting source / destination numTasks on an Edge
> --------------------------------------------------------
>
> Key: TEZ-933
> URL: https://issues.apache.org/jira/browse/TEZ-933
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Fix For: 0.4.0
>
> Attachments: TEZ-933.1.txt
>
>
> Edges rely on getting properties (specifically numTasks in this case) from
> the source or destination vertex.
> This can end up with an incorrect value being used depending on the state of
> the vertex - whether the vertex has been initialized, whether the parallelism
> has been changed etc.
> As an example
> {code}
> edgeManager.getNumSourceTaskPhysicalOutputs(destinationVertex.getTotalTasks(),
> sourceTaskIndex))
> {code}
> destinationVertex.getTotalTasks() may be incorrect if the destinationVertex
> hasn't yet been initialized. Alternately, this value can change based on
> setParallelism calls.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
