[jira] [Commented] (YARN-8867) Retrieve the progress of resource localization
[
https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669361#comment-16669361
]
Eric Yang commented on YARN-8867:
-
The JIRA title says progress, instead, what would be reported are status of the
localization. Either change the title of the JIRA to match implementation or
change localization status to progress with numeric percentage for accuracy.
> Retrieve the progress of resource localization
> --
>
> Key: YARN-8867
> URL: https://issues.apache.org/jira/browse/YARN-8867
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Chandni Singh
>Assignee: Chandni Singh
>Priority: Major
> Attachments: YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the progress of localization.
> Unless the client can know when the localization of a resource is complete
> irrespective of the type of the resource, it cannot take any appropriate
> action.
> We need an API in {{ContainerManagementProtocol}} to retrieve the progress on
> the localization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8867) Retrieve the progress of resource localization
[
https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669342#comment-16669342
]
Chandni Singh commented on YARN-8867:
-
[~eyang] Thanks for reviewing. Retrieving the progress is a good point. I just
think it can be done as a next step.
As you stated, this patch adds the basic protocol to query the status of
localized resources. With this, the status would only say if the resource
localization in - IN_PROGRESS, COMPLETED, or FAILED.
Adding more information in {{LocalizationStatus}} like progress could be
handled separately.
Let me know if that is okay?
> Retrieve the progress of resource localization
> --
>
> Key: YARN-8867
> URL: https://issues.apache.org/jira/browse/YARN-8867
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Chandni Singh
>Assignee: Chandni Singh
>Priority: Major
> Attachments: YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the progress of localization.
> Unless the client can know when the localization of a resource is complete
> irrespective of the type of the resource, it cannot take any appropriate
> action.
> We need an API in {{ContainerManagementProtocol}} to retrieve the progress on
> the localization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8867) Retrieve the progress of resource localization
[
https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669325#comment-16669325
]
Eric Yang commented on YARN-8867:
-
[~csingh] The current wip patch looks ok for adding new communication protocol
in yarn, node manager and mapreduce client to retrieve localization status. We
probably want to define what constitutes a localization status. In CLI and UI,
progress field seems to indicate a numeric number of percentage. The current
communication protocol is a list of localization status. If we have container
image to be localized, and it is represented by one of the localization status.
It does not translate well to percentage values. Docker image download is
also hard to be represented as percentage value because docker pull output does
not show percentage output. Given those limitations from backend, it would be
in our best interests to define what user will see as progression. If progress
percentage is still the information that is going to be displayed, we may want
to spend more time on how to calculate status to progress percentage.
Otherwise, localizing a single container without other resource might look odd
for UI to stuck on 0 percent and jump to 100 percent because there is only one
localization status.
> Retrieve the progress of resource localization
> --
>
> Key: YARN-8867
> URL: https://issues.apache.org/jira/browse/YARN-8867
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Chandni Singh
>Assignee: Chandni Singh
>Priority: Major
> Attachments: YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the progress of localization.
> Unless the client can know when the localization of a resource is complete
> irrespective of the type of the resource, it cannot take any appropriate
> action.
> We need an API in {{ContainerManagementProtocol}} to retrieve the progress on
> the localization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8867) Retrieve the progress of resource localization
[
https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16668000#comment-16668000
]
Chandni Singh commented on YARN-8867:
-
The work in progress patch has the new API which retrieves the localization
status of the resource.. [~leftnoteasy] [~jlowe] [~ebadger] [~eyang] please let
me know if the API changes look good to you.
Thanks,
Chandni
> Retrieve the progress of resource localization
> --
>
> Key: YARN-8867
> URL: https://issues.apache.org/jira/browse/YARN-8867
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Chandni Singh
>Assignee: Chandni Singh
>Priority: Major
> Attachments: YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the progress of localization.
> Unless the client can know when the localization of a resource is complete
> irrespective of the type of the resource, it cannot take any appropriate
> action.
> We need an API in {{ContainerManagementProtocol}} to retrieve the progress on
> the localization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
