[
https://issues.apache.org/jira/browse/TEZ-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14194766#comment-14194766
]
Bikas Saha commented on TEZ-1547:
---------------------------------
Sorry for any confusion in the code or comments. The dual meaning is not
present. If anything nudged you towards that then let me know and I can add
clarifying comments.
The only purpose of this is to let listeners know that the vertex configuration
(IPO/parallelism/edges etc) will not change. Hence the name has configured in
it. CONFIGURED is shorter than FULLY_CONFIGURED. I can make that change. FULLY
is stressing the same thing again.
This sending of the notification is unrelated to scheduling of tasks on the
sending vertex. That vertex can schedule tasks as it pleases. The event
listener is the one that uses the notification to affect scheduling of tasks on
the listener. The event assures the listener that it is safe to schedule tasks
because the world will not change underneath them and leave them hanging
(literally :)). As you can see, the Shuffle VM first notifies of configuration
complete, and then proceeds to schedule tasks based on the existing min/max
logic. In the future, some VM could start its tasks, do reconfiguration and
then send the done signal.
So this change does not aim to fix the out of order scheduling issues though it
does end up mitigating some of those cases.
Thanks [~daijy] for the test results.
> Make use of state change notifier in VertexManagerPlugins
> ---------------------------------------------------------
>
> Key: TEZ-1547
> URL: https://issues.apache.org/jira/browse/TEZ-1547
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Bikas Saha
> Attachments: TEZ-1547.1.patch, TEZ-1547.3.patch, TEZ-1547.4.patch,
> TEZ-1547.5.patch, TEZ-1547.6.patch, TEZ-1547.7.patch, TEZ-1547.8.patch,
> TEZ-1547.9.patch
>
>
> Instead of the various APIs like onVertexStarted, simple notifications could
> be sent.
> Some existing APIs could end up being deprecated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
