[
https://issues.apache.org/jira/browse/YARN-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387044#comment-15387044
]
Zhankun Tang commented on YARN-3854:
------------------------------------
[~templedf], thanks for pointing out the extra efforts of "docker save" the
images and upload them into HDFS. These steps with extra upload HDFS network
bandwith consuming are indeed unavoidable for this HDFS+save/load approach.
If we use "docker pull" before launching container to prepare image, it becomes
a topic of how to have fine-grained Docker image pull control in YARN when
utilizing private repo (whether its storage backed is HDFS or not).
Explicit pull should have potential benifit if YARN can be aware of the pull
progress and manipulate the process. My concern here is that it's not easy to
control explicit docker pull because there seems no interface for us to check
the progress or pause/cancel the pull process currently. In this situation,
explicit "docker pull" in YARN can only check if the explicit docker pull is
timeout then maybe kill it. These timeout check already exists for launching
container. So it seems no big differences versus just let Docker do it
implicitly. Thoughts?
> Add localization support for docker images
> ------------------------------------------
>
> Key: YARN-3854
> URL: https://issues.apache.org/jira/browse/YARN-3854
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Sidharta Seethana
> Assignee: Zhankun Tang
> Attachments: YARN-3854-branch-2.8.001.patch,
> YARN-3854_Localization_support_for_Docker_image_v1.pdf,
> YARN-3854_Localization_support_for_Docker_image_v2.pdf
>
>
> We need the ability to localize images from HDFS and load them for use when
> launching docker containers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
