Zhankun Tang created YARN-8989:
----------------------------------
Summary: Move DockerCommandPlugin volume related APIs' invocation
from DockerLinuxContainerRuntime#prepareContainer to #launchContainer
Key: YARN-8989
URL: https://issues.apache.org/jira/browse/YARN-8989
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang
This seems required before we implement isolation in pluggable device framework
for default container and Docker container with LinuxContainerExecutor.
To find a place for plugin "onDevicesAllocated" in current operation flow when
running a container with LCE.
{code:java}
ContainerLaunch#call() ->
1.ContainerLaunch#prepareContainer() - >
LCE#prepareContainer ->
DelegatingLinuxContainerRuntime#prepareContainer ->
DockerLinuxContainerRuntime#prepareContainer ->
DockerCommandPlugin#getCreateDockerVolumeCommand ->
onDeviceAllocated(null,docker); create volume?
2.ContainerLaunch#launchContainer
LCE#launchContainer() ->
resourceHandlerChain#preStart() ->
DeviceResourceHandlerImpl#preStart() ->
onDeviceAllocated(alloc,docker)
allocate device and do isolation for default container with cgroup
{code}
What I want to do here is to move the DockerCommandPlugin APIs invocation from
DockerLinuxContainerRuntime#prepareContainer to #launchContainer. This won't
bring any incompatibility and can benefit the pluggable device framework's
interaction with the device plugin.
The "DeviceRuntimeSpec onDevicesAllocated(Set<Device>allocation, yarnRuntime)"
implemented by device plugin is to let the plugin do some preparation and
return a spec on how to run the container with the allocated device. We
designed a VolumeClaim field in DeviceRuntimeSpec object for the plugin to
declare what volume they need to create.
In current code flow, call this "onDevicesAllocated" in the
DockerCommandPlugin's methods seems weird and can only pass a null value as
allocation. This will complex the vendor device plugin implementation to handle
a null value.
Once we move the DockerCommandPlugin API invocation, it will like this:
{code:java}
ContainerLaunch#call() ->
ContainerLaunch#launchContainer
LCE#launchContainer() ->
resourceHandlerChain#preStart() ->
DeviceResourceHandlerImpl#preStart() ->
onDeviceAllocated(alloc,docker)
allocate device and do isolation for default container with cgroup
DelegatingLinuxContainerRuntime#launchContaienr ->
DockerLinuxContainerRuntime#launchContainer->
DockerCommandPlugin#getCreateDockerVolumeCommand ->
get allocation;onDeviceAllocated(alloc,docker);create volume{code}
After changes, the flow is more smooth and also simplify the plugin
implementation for "onDevicesAllocated"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
