[
https://issues.apache.org/jira/browse/YARN-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645737#comment-16645737
]
Chandni Singh commented on YARN-3854:
-------------------------------------
[~ebadger] I thought more about the below issue
{quote}
I'm worried that the image deletion is going to be quite rough and error-prone.
Docker containers sometimes stick around in the Exited or Dead states or keep
running beyond when they should. That will cause the delete to perpetually fail
since the image is still in use, even though it isn't really in use, at least
not in a useful sense. Do we just keep going around and around and trying to
delete the same images even if we keep failing?
{quote}
If we go by NM maintaining an LRU cache of images it has localized and only
deletes the one it localized, then below could be a possible solution
- image meta contains count of running containers. This count is updated when
the container is launched.
- NM already has a service which periodically check the statuses of the
container. If the container is stopped/failed, image meta is updated.
- When the image is evicted and count of running containers = 0, it can be
force deleted.
With this solution, we just rely on NM's view of containers and images.
> Add localization support for docker images
> ------------------------------------------
>
> Key: YARN-3854
> URL: https://issues.apache.org/jira/browse/YARN-3854
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Sidharta Seethana
> Assignee: Chandni Singh
> Priority: Major
> Labels: Docker
> Attachments: Localization Support For Docker Images.pdf,
> YARN-3854-branch-2.8.001.patch,
> YARN-3854_Localization_support_for_Docker_image_v1.pdf,
> YARN-3854_Localization_support_for_Docker_image_v2.pdf,
> YARN-3854_Localization_support_for_Docker_image_v3.pdf
>
>
> We need the ability to localize docker images when those images aren't
> already available locally. There are various approaches that could be used
> here with different trade-offs/issues : image archives on HDFS + docker load
> , docker pull during the localization phase or (automatic) docker pull
> during the run/launch phase.
> We also need the ability to clean-up old/stale, unused images.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
