[jira] [Comment Edited] (YARN-8867) Retrieve the status of resource localization
[
https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750317#comment-16750317
]
Chandni Singh edited comment on YARN-8867 at 1/23/19 6:46 PM:
--
Uploaded patch 7 which address the last review comments. I still have to test
the change.
was (Author: csingh):
Uploaded patch 7 which address the last review comments. I still have to test
change.
> Retrieve the status of resource localization
>
>
> Key: YARN-8867
> URL: https://issues.apache.org/jira/browse/YARN-8867
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Chandni Singh
>Assignee: Chandni Singh
>Priority: Major
> Attachments: YARN-8867.001.patch, YARN-8867.002.patch,
> YARN-8867.003.patch, YARN-8867.004.patch, YARN-8867.005.patch,
> YARN-8867.006.patch, YARN-8867.007.patch, YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the status of localization.
> Unless the client can know when the localization of a resource is complete
> irrespective of the type of the resource, it cannot take any appropriate
> action.
> We need an API in {{ContainerManagementProtocol}} to retrieve the status on
> the localization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8867) Retrieve the status of resource localization
[
https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702501#comment-16702501
]
Eric Yang edited comment on YARN-8867 at 11/28/18 11:24 PM:
[~csingh] Thank you for the update. Patch 006 doesn't show PENDING for
localizationStatus when localization has not happen yet or in progress. It
looks like localizationStatus is null that prevents yarn app -status app1 query
from reporting the transitional status.
Second issue the resource key seems to chop off the first level directory of
dest_file, and use that as the resource key. Example of localizing resources
are:
{code}
"files": [
{
"dest_file": "/tmp/conf/hello",
"properties": {
"content": "Hello world"
},
"type": "TEMPLATE"
},
{
"dest_file": "/hadoop/yarn",
"properties": {},
"src_file":
"hdfs://eyang-1.openstacklocal:9000/user/hbase/.yarn/services/abc/sysfs.tar",
"type": "ARCHIVE"
}
],
{code}
The result looks like:
{code}
"localizationStatuses": [
{
"resourceKey": "conf/hello",
"state": "COMPLETED"
},
{
"resourceKey": "yarn",
"state": "COMPLETED"
}
],
{code}
It is not easy to correlate between resourceKey and dest_file. Would it be
possible to use the full path of dest_file as resourceKey for consistency?
was (Author: eyang):
[~csingh] Thank you for the update. Patch 006 doesn't show PENDING for
localizationStatus when localization has not happen yet or in progress. It
looks like localizationStatus is null that prevents yarn app -status app1 query
from reporting the transitional status.
> Retrieve the status of resource localization
>
>
> Key: YARN-8867
> URL: https://issues.apache.org/jira/browse/YARN-8867
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Chandni Singh
>Assignee: Chandni Singh
>Priority: Major
> Attachments: YARN-8867.001.patch, YARN-8867.002.patch,
> YARN-8867.003.patch, YARN-8867.004.patch, YARN-8867.005.patch,
> YARN-8867.006.patch, YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the status of localization.
> Unless the client can know when the localization of a resource is complete
> irrespective of the type of the resource, it cannot take any appropriate
> action.
> We need an API in {{ContainerManagementProtocol}} to retrieve the status on
> the localization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8867) Retrieve the status of resource localization
[
https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677263#comment-16677263
]
Chandni Singh edited comment on YARN-8867 at 11/6/18 8:17 PM:
--
TestResourceTrackerService failure is unrelated.
[~eyang] [~jlowe] patch 3 is ready for review.
was (Author: csingh):
TestResourceTrackerService failure is unrelated.
[~eyang] [~jlowe] patch 3 is ready to be reviewed.
> Retrieve the status of resource localization
>
>
> Key: YARN-8867
> URL: https://issues.apache.org/jira/browse/YARN-8867
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Chandni Singh
>Assignee: Chandni Singh
>Priority: Major
> Attachments: YARN-8867.001.patch, YARN-8867.002.patch,
> YARN-8867.003.patch, YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the status of localization.
> Unless the client can know when the localization of a resource is complete
> irrespective of the type of the resource, it cannot take any appropriate
> action.
> We need an API in {{ContainerManagementProtocol}} to retrieve the status on
> the localization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
