[jira] [Commented] (YARN-1007) [YARN-321] Enhance History Reader interface for Containers
[
https://issues.apache.org/jira/browse/YARN-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727821#comment-13727821
]
Zhijie Shen commented on YARN-1007:
---
+1 for
{code}
CollectionContainerHistoryData getContainers(ApplicationAttemptId
appAttemptId);
{code}
If the storage is black box,
{code}
CollectionContainerHistoryData getContainers(ApplicationId appId)
{code}
may be not necessary, but can be derived from
{code}
CollectionContainerHistoryData containers = ArrayListContainerHistoryData();
for (ApplicationAttemptHistoryData appAttempt : getApplicationAttempts(appId)) {
containers.addAll(getContainers(appAttempt.getApplicationAttemptId());
}
return containers;
{code}
However, if it the underlying I/O operations can be optimized for direct
getContainers(ApplicationId appId), it will be a different story. Anyway,
currently we'd better focus on making the end-to-end system available, and
leave the performance issues to do later. Thoughts?
[YARN-321] Enhance History Reader interface for Containers
--
Key: YARN-1007
URL: https://issues.apache.org/jira/browse/YARN-1007
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Devaraj K
Assignee: Mayank Bansal
If we want to show the containers used by application/app attempt, We need to
have two more API's which returns collection of ContainerHistoryData for
application id and applcation attempt id something like below.
{code:xml}
CollectionContainerHistoryData getContainers(
ApplicationAttemptId appAttemptId);
CollectionContainerHistoryData getContainers(ApplicationId appId);
{code}
{code:xml}
/**
* This method returns {@link Container} for specified {@link ContainerId}.
*
* @param {@link ContainerId}
* @return {@link Container} for ContainerId
*/
ContainerHistoryData getAMContainer(ContainerId containerId);
{code}
In the above API, we need to change the argument to application attempt id or
we can remove this API because every attempt history data has master
container id field, using master container id, history data can get using
this below API if it takes argument as container id.
{code:xml}
/**
* This method returns {@link ContainerHistoryData} for specified
* {@link ApplicationAttemptId}.
*
* @param {@link ApplicationAttemptId}
* @return {@link ContainerHistoryData} for ApplicationAttemptId
*/
ContainerHistoryData getContainer(ApplicationAttemptId appAttemptId);
{code}
Here application attempt can use numbers of containers but we cannot choose
which container history data to return. This API argument also need to be
changed to take container id instead of app attempt id.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1007) [YARN-321] Enhance History Reader interface for Containers
[
https://issues.apache.org/jira/browse/YARN-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727973#comment-13727973
]
Mayank Bansal commented on YARN-1007:
-
+1 for
{code}
MAPContainerId,ContainerHistoryData getContainers(ApplicationAttemptId
appAttemptId);
{code}
I am adding MAP instead of LIST to get the O(1) if we want to use those objects
later. We can Iterate MAP as a List and still but if we want to store this and
use multiple times then MAP becomes handy.
Thanks,
Mayank
[YARN-321] Enhance History Reader interface for Containers
--
Key: YARN-1007
URL: https://issues.apache.org/jira/browse/YARN-1007
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Devaraj K
Assignee: Mayank Bansal
If we want to show the containers used by application/app attempt, We need to
have two more API's which returns collection of ContainerHistoryData for
application id and applcation attempt id something like below.
{code:xml}
CollectionContainerHistoryData getContainers(
ApplicationAttemptId appAttemptId);
CollectionContainerHistoryData getContainers(ApplicationId appId);
{code}
{code:xml}
/**
* This method returns {@link Container} for specified {@link ContainerId}.
*
* @param {@link ContainerId}
* @return {@link Container} for ContainerId
*/
ContainerHistoryData getAMContainer(ContainerId containerId);
{code}
In the above API, we need to change the argument to application attempt id or
we can remove this API because every attempt history data has master
container id field, using master container id, history data can get using
this below API if it takes argument as container id.
{code:xml}
/**
* This method returns {@link ContainerHistoryData} for specified
* {@link ApplicationAttemptId}.
*
* @param {@link ApplicationAttemptId}
* @return {@link ContainerHistoryData} for ApplicationAttemptId
*/
ContainerHistoryData getContainer(ApplicationAttemptId appAttemptId);
{code}
Here application attempt can use numbers of containers but we cannot choose
which container history data to return. This API argument also need to be
changed to take container id instead of app attempt id.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1007) [YARN-321] Enhance History Reader interface for Containers
[
https://issues.apache.org/jira/browse/YARN-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728127#comment-13728127
]
Hadoop QA commented on YARN-1007:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595668/YARN-1007-1.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1648//console
This message is automatically generated.
[YARN-321] Enhance History Reader interface for Containers
--
Key: YARN-1007
URL: https://issues.apache.org/jira/browse/YARN-1007
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Devaraj K
Assignee: Mayank Bansal
Attachments: YARN-1007-1.patch
If we want to show the containers used by application/app attempt, We need to
have two more API's which returns collection of ContainerHistoryData for
application id and applcation attempt id something like below.
{code:xml}
CollectionContainerHistoryData getContainers(
ApplicationAttemptId appAttemptId);
CollectionContainerHistoryData getContainers(ApplicationId appId);
{code}
{code:xml}
/**
* This method returns {@link Container} for specified {@link ContainerId}.
*
* @param {@link ContainerId}
* @return {@link Container} for ContainerId
*/
ContainerHistoryData getAMContainer(ContainerId containerId);
{code}
In the above API, we need to change the argument to application attempt id or
we can remove this API because every attempt history data has master
container id field, using master container id, history data can get using
this below API if it takes argument as container id.
{code:xml}
/**
* This method returns {@link ContainerHistoryData} for specified
* {@link ApplicationAttemptId}.
*
* @param {@link ApplicationAttemptId}
* @return {@link ContainerHistoryData} for ApplicationAttemptId
*/
ContainerHistoryData getContainer(ApplicationAttemptId appAttemptId);
{code}
Here application attempt can use numbers of containers but we cannot choose
which container history data to return. This API argument also need to be
changed to take container id instead of app attempt id.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1007) [YARN-321] Enhance History Reader interface for Containers
[
https://issues.apache.org/jira/browse/YARN-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728126#comment-13728126
]
Zhijie Shen commented on YARN-1007:
---
The patch is almost fine, but I think it's better to see that getContainers()
fetches more than 1 containers.
[YARN-321] Enhance History Reader interface for Containers
--
Key: YARN-1007
URL: https://issues.apache.org/jira/browse/YARN-1007
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Devaraj K
Assignee: Mayank Bansal
Attachments: YARN-1007-1.patch
If we want to show the containers used by application/app attempt, We need to
have two more API's which returns collection of ContainerHistoryData for
application id and applcation attempt id something like below.
{code:xml}
CollectionContainerHistoryData getContainers(
ApplicationAttemptId appAttemptId);
CollectionContainerHistoryData getContainers(ApplicationId appId);
{code}
{code:xml}
/**
* This method returns {@link Container} for specified {@link ContainerId}.
*
* @param {@link ContainerId}
* @return {@link Container} for ContainerId
*/
ContainerHistoryData getAMContainer(ContainerId containerId);
{code}
In the above API, we need to change the argument to application attempt id or
we can remove this API because every attempt history data has master
container id field, using master container id, history data can get using
this below API if it takes argument as container id.
{code:xml}
/**
* This method returns {@link ContainerHistoryData} for specified
* {@link ApplicationAttemptId}.
*
* @param {@link ApplicationAttemptId}
* @return {@link ContainerHistoryData} for ApplicationAttemptId
*/
ContainerHistoryData getContainer(ApplicationAttemptId appAttemptId);
{code}
Here application attempt can use numbers of containers but we cannot choose
which container history data to return. This API argument also need to be
changed to take container id instead of app attempt id.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1007) [YARN-321] Enhance History Reader interface for Containers
[
https://issues.apache.org/jira/browse/YARN-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728199#comment-13728199
]
Hadoop QA commented on YARN-1007:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595675/YARN-1007-2.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1649//console
This message is automatically generated.
[YARN-321] Enhance History Reader interface for Containers
--
Key: YARN-1007
URL: https://issues.apache.org/jira/browse/YARN-1007
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Devaraj K
Assignee: Mayank Bansal
Attachments: YARN-1007-1.patch, YARN-1007-2.patch
If we want to show the containers used by application/app attempt, We need to
have two more API's which returns collection of ContainerHistoryData for
application id and applcation attempt id something like below.
{code:xml}
CollectionContainerHistoryData getContainers(
ApplicationAttemptId appAttemptId);
CollectionContainerHistoryData getContainers(ApplicationId appId);
{code}
{code:xml}
/**
* This method returns {@link Container} for specified {@link ContainerId}.
*
* @param {@link ContainerId}
* @return {@link Container} for ContainerId
*/
ContainerHistoryData getAMContainer(ContainerId containerId);
{code}
In the above API, we need to change the argument to application attempt id or
we can remove this API because every attempt history data has master
container id field, using master container id, history data can get using
this below API if it takes argument as container id.
{code:xml}
/**
* This method returns {@link ContainerHistoryData} for specified
* {@link ApplicationAttemptId}.
*
* @param {@link ApplicationAttemptId}
* @return {@link ContainerHistoryData} for ApplicationAttemptId
*/
ContainerHistoryData getContainer(ApplicationAttemptId appAttemptId);
{code}
Here application attempt can use numbers of containers but we cannot choose
which container history data to return. This API argument also need to be
changed to take container id instead of app attempt id.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1007) [YARN-321] Enhance History Reader interface for Containers
[
https://issues.apache.org/jira/browse/YARN-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13726697#comment-13726697
]
Mayank Bansal commented on YARN-1007:
-
Taking it over as these are the reader interface changes
[YARN-321] Enhance History Reader interface for Containers
--
Key: YARN-1007
URL: https://issues.apache.org/jira/browse/YARN-1007
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Devaraj K
Assignee: Mayank Bansal
If we want to show the containers used by application/app attempt, We need to
have two more API's which returns collection of ContainerHistoryData for
application id and applcation attempt id something like below.
{code:xml}
CollectionContainerHistoryData getContainers(
ApplicationAttemptId appAttemptId);
CollectionContainerHistoryData getContainers(ApplicationId appId);
{code}
{code:xml}
/**
* This method returns {@link Container} for specified {@link ContainerId}.
*
* @param {@link ContainerId}
* @return {@link Container} for ContainerId
*/
ContainerHistoryData getAMContainer(ContainerId containerId);
{code}
In the above API, we need to change the argument to application attempt id or
we can remove this API because every attempt history data has master
container id field, using master container id, history data can get using
this below API if it takes argument as container id.
{code:xml}
/**
* This method returns {@link ContainerHistoryData} for specified
* {@link ApplicationAttemptId}.
*
* @param {@link ApplicationAttemptId}
* @return {@link ContainerHistoryData} for ApplicationAttemptId
*/
ContainerHistoryData getContainer(ApplicationAttemptId appAttemptId);
{code}
Here application attempt can use numbers of containers but we cannot choose
which container history data to return. This API argument also need to be
changed to take container id instead of app attempt id.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1007) [YARN-321] Enhance History Reader interface for Containers
[
https://issues.apache.org/jira/browse/YARN-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13726709#comment-13726709
]
Mayank Bansal commented on YARN-1007:
-
[~devaraj.k],
Please look into YARN-956, I have done the change for getContainer and
getAMContainer
For this below API,I think we should use getAMContainer(), why you need
collection?
{code}
CollectionContainerHistoryData getContainers(ApplicationAttemptId
appAttemptId);
{code}
For Below API you want all the containers used by this application? means all
the task attempts? Can you please explain little bit?
{code}
CollectionContainerHistoryData getContainers(ApplicationId appId);
{code}
[YARN-321] Enhance History Reader interface for Containers
--
Key: YARN-1007
URL: https://issues.apache.org/jira/browse/YARN-1007
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Devaraj K
Assignee: Mayank Bansal
If we want to show the containers used by application/app attempt, We need to
have two more API's which returns collection of ContainerHistoryData for
application id and applcation attempt id something like below.
{code:xml}
CollectionContainerHistoryData getContainers(
ApplicationAttemptId appAttemptId);
CollectionContainerHistoryData getContainers(ApplicationId appId);
{code}
{code:xml}
/**
* This method returns {@link Container} for specified {@link ContainerId}.
*
* @param {@link ContainerId}
* @return {@link Container} for ContainerId
*/
ContainerHistoryData getAMContainer(ContainerId containerId);
{code}
In the above API, we need to change the argument to application attempt id or
we can remove this API because every attempt history data has master
container id field, using master container id, history data can get using
this below API if it takes argument as container id.
{code:xml}
/**
* This method returns {@link ContainerHistoryData} for specified
* {@link ApplicationAttemptId}.
*
* @param {@link ApplicationAttemptId}
* @return {@link ContainerHistoryData} for ApplicationAttemptId
*/
ContainerHistoryData getContainer(ApplicationAttemptId appAttemptId);
{code}
Here application attempt can use numbers of containers but we cannot choose
which container history data to return. This API argument also need to be
changed to take container id instead of app attempt id.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
