[
https://issues.apache.org/jira/browse/YARN-7018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571962#comment-16571962
]
Manikandan R commented on YARN-7018:
------------------------------------
[~jlowe] and I had a very preliminary offline discussion on approaches. Thank
you [~jlowe] for explaining the requirements in detail.
Summary of the discussion: (Copying [~jlowe]'s suggestion as is)
The point of YARN-7018 is to create a hook that runs before the scheduler and,
as much as possible, is *not* scheduler-dependent. If we have to do something
scheduler-dependent then we might as well just modify the schedulers directly.
I haven't had much time to think about it, but one potential way to implement
this is a very simple plugin API. Something like this:
- Admin configures the plugin via a conf key
- Plugin only has one API call, onNodeHeartbeat or something like that. It's
called when a NODE_UPDATE event is processed and before the main scheduler loop
is called
- Call arguments would include the SchedulerNode and maybe the RMContext.
>From the RMContext the plugin can get the scheduler and do scheduler-specific
>things if it wants to, or can be scheduler-specific if it wants to.
I'd rather not force plugin writers to implement every scheduler supported.
It's a burden to writing a plugin and an obstacle to providing new schedulers
in YARN.
> Interface for adding extra behavior to node heartbeats
> ------------------------------------------------------
>
> Key: YARN-7018
> URL: https://issues.apache.org/jira/browse/YARN-7018
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: resourcemanager
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Priority: Major
>
> This JIRA tracks an interface for plugging in new behavior to node heartbeat
> processing. Adding a formal interface for additional node heartbeat
> processing would allow admins to configure new functionality that is
> scheduler-independent without needing to replace the entire scheduler. For
> example, both YARN-5202 and YARN-5215 had approaches where node heartbeat
> processing was extended to implement new functionality that was essentially
> scheduler-independent and could be implemented as a plugin with this
> interface.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
