[
https://issues.apache.org/jira/browse/YARN-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16676074#comment-16676074
]
Zhankun Tang edited comment on YARN-8851 at 11/6/18 3:30 AM:
-------------------------------------------------------------
{quote}1) Regarding to the
NM_PLUGGABLE_DEVICE_FRAMEWORK_PREFER_CUSTOMIZED_SCHEDULER, should we just use
default scheduler if device plugin doesn't provide their customized scheduler?
We should assume that load device plugin runs "trusted" code, we may not need
to add extra protection here.
{quote}
Zhankun–> Agree.
{quote}2) DeviceSchedulerManager, it sounds like "manages scheduler", however
it handles how to map device to containers, and scheduler is just
implementation details. How about call it DeviceMappingManager?
{quote}
-
{quote}internalAssignDevices should be private, and it is a bit long, might be
better for future maintenance if you can break it down to multiple methods.
{quote}
Zhankun -> Good idea. Will do that.
{quote}I think we could move to make this POC to sub tasks and get them done
piece by piece. It gonna be helpful if you can highlight subtasks required.
{quote}
Zhankun-> The YARN-8880, YARN-8881, YARN-8882, YARN-8883, YARN-8885 are our
Phase 1 highlighted subtasks. They'll be marked "InProgress" to highlight.
Thanks for the review! [~leftnoteasy]
was (Author: tangzhankun):
{quote}1) Regarding to the
NM_PLUGGABLE_DEVICE_FRAMEWORK_PREFER_CUSTOMIZED_SCHEDULER, should we just use
default scheduler if device plugin doesn't provide their customized scheduler?
We should assume that load device plugin runs "trusted" code, we may not need
to add extra protection here.
{quote}
Zhankun–> Agree.
{quote}2) DeviceSchedulerManager, it sounds like "manages scheduler", however
it handles how to map device to containers, and scheduler is just
implementation details. How about call it DeviceMappingManager?
{quote}
-
{quote}internalAssignDevices should be private, and it is a bit long, might be
better for future maintenance if you can break it down to multiple methods.
{quote}
Zhankun -> Good idea. Will do that.
{quote}I think we could move to make this POC to sub tasks and get them done
piece by piece. It gonna be helpful if you can highlight subtasks required.
{quote}
Zhankun-> The YARN-8880, YARN-8881, YARN-8882, YARN-8883, YARN-8885 are our
Phase 1 highlighted subtasks.
Thanks for the review! [~leftnoteasy]
> [Umbrella] A new pluggable device plugin framework to ease vendor plugin
> development
> ------------------------------------------------------------------------------------
>
> Key: YARN-8851
> URL: https://issues.apache.org/jira/browse/YARN-8851
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
> Reporter: Zhankun Tang
> Assignee: Zhankun Tang
> Priority: Major
> Attachments: YARN-8851-WIP2-trunk.001.patch,
> YARN-8851-WIP3-trunk.001.patch, YARN-8851-WIP4-trunk.001.patch,
> YARN-8851-WIP5-trunk.001.patch, YARN-8851-WIP6-trunk.001.patch,
> YARN-8851-WIP7-trunk.001.patch, YARN-8851-WIP8-trunk.001.patch,
> YARN-8851-WIP9-trunk.001.patch, YARN-8851-trunk.001.patch, [YARN-8851]
> YARN_New_Device_Plugin_Framework_Design_Proposal-3.pdf, [YARN-8851]
> YARN_New_Device_Plugin_Framework_Design_Proposal-4.pdf, [YARN-8851]
> YARN_New_Device_Plugin_Framework_Design_Proposal.pdf
>
>
> At present, we support GPU/FPGA device in YARN through a native, coupling
> way. But it's difficult for a vendor to implement such a device plugin
> because the developer needs much knowledge of YARN internals. And this brings
> burden to the community to maintain both YARN core and vendor-specific code.
> Here we propose a new device plugin framework to ease vendor device plugin
> development and provide a more flexible way to integrate with YARN NM.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
