[jira] [Commented] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1344#comment-1344 ] Xuan Gong commented on YARN-1611: - upload a new patch addressed all latest comments > Make admin refresh of capacity scheduler configuration work across RM failover > -- > > Key: YARN-1611 > URL: https://issues.apache.org/jira/browse/YARN-1611 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-1611.1.patch, YARN-1611.2.patch, YARN-1611.2.patch, > YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch, > YARN-1611.6.patch, YARN-1611.7.patch, YARN-1611.8.patch > > > Currently, If we do refresh* for a standby RM, it will failover to the > current active RM, and do the refresh* based on the local configuration file > of the active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover
[
https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1340#comment-1340
]
Hadoop QA commented on YARN-1611:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626507/YARN-1611.8.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2984//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/2984//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2984//console
This message is automatically generated.
> Make admin refresh of capacity scheduler configuration work across RM failover
> --
>
> Key: YARN-1611
> URL: https://issues.apache.org/jira/browse/YARN-1611
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-1611.1.patch, YARN-1611.2.patch, YARN-1611.2.patch,
> YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch,
> YARN-1611.6.patch, YARN-1611.7.patch, YARN-1611.8.patch
>
>
> Currently, If we do refresh* for a standby RM, it will failover to the
> current active RM, and do the refresh* based on the local configuration file
> of the active RM.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1611: Attachment: YARN-1611.8.patch > Make admin refresh of capacity scheduler configuration work across RM failover > -- > > Key: YARN-1611 > URL: https://issues.apache.org/jira/browse/YARN-1611 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-1611.1.patch, YARN-1611.2.patch, YARN-1611.2.patch, > YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch, > YARN-1611.6.patch, YARN-1611.7.patch, YARN-1611.8.patch > > > Currently, If we do refresh* for a standby RM, it will failover to the > current active RM, and do the refresh* based on the local configuration file > of the active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1499) Fair Scheduler changes for moving apps between queues
[
https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1318#comment-1318
]
Hadoop QA commented on YARN-1499:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626504/YARN-1499-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2983//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/2983//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2983//console
This message is automatically generated.
> Fair Scheduler changes for moving apps between queues
> -
>
> Key: YARN-1499
> URL: https://issues.apache.org/jira/browse/YARN-1499
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: YARN-1499-1.patch, YARN-1499.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1499) Fair Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1499: - Attachment: YARN-1499-1.patch > Fair Scheduler changes for moving apps between queues > - > > Key: YARN-1499 > URL: https://issues.apache.org/jira/browse/YARN-1499 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-1499-1.patch, YARN-1499.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1637) Implement a client library for java users to post entities+events
[ https://issues.apache.org/jira/browse/YARN-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1315#comment-1315 ] Zhijie Shen commented on YARN-1637: --- javadoc and test failure seem not to be related. > Implement a client library for java users to post entities+events > - > > Key: YARN-1637 > URL: https://issues.apache.org/jira/browse/YARN-1637 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Zhijie Shen > Attachments: YARN-1637.1.patch, YARN-1637.2.patch > > > This is a wrapper around the web-service to facilitate easy posting of > entity+event data to the time-line server. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1637) Implement a client library for java users to post entities+events
[
https://issues.apache.org/jira/browse/YARN-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1309#comment-1309
]
Hadoop QA commented on YARN-1637:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626503/YARN-1637.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1556 javac
compiler warnings (more than the trunk's current 1545 warnings).
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:
org.apache.hadoop.yarn.client.api.impl.TestNMClient
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2982//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/2982//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2982//console
This message is automatically generated.
> Implement a client library for java users to post entities+events
> -
>
> Key: YARN-1637
> URL: https://issues.apache.org/jira/browse/YARN-1637
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Zhijie Shen
> Attachments: YARN-1637.1.patch, YARN-1637.2.patch
>
>
> This is a wrapper around the web-service to facilitate easy posting of
> entity+event data to the time-line server.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects
[ https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1304#comment-1304 ] Zhijie Shen commented on YARN-1659: --- bq. I don't think we should make this change, as the enum values currently match the names of the optional member variables of the ATSEntity object. IMO, the renaming mentioned by Vinod is not changing to another name. Instead, it's adjusting the format to comply with Java convention. For example, RELATEDENTITIES -> RELATED_ENTITIES, PRIMARYFILTERS -> PRIMARY_FILTERS. bq. I agree removing Entity is a good idea. I've made the change, and will just wait for Zhijie's feedback before I upload another patch. +1 for removing Entity, thought in-memory implementation takes the entity as the key. However, interface shouldn't depend on the implementation. I can create a inner Entity object instead. Please go ahead with the change. > Define ApplicationTimelineStore interface and store-facing entity, > entity-info and event objects > > > Key: YARN-1659 > URL: https://issues.apache.org/jira/browse/YARN-1659 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, > YARN-1659-5.patch, YARN-1659-6.patch, YARN-1659.2.patch > > > These will be used by ApplicationTimelineStore interface. The web services > will convert the store-facing obects to the user-facing objects. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1637) Implement a client library for java users to post entities+events
[ https://issues.apache.org/jira/browse/YARN-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1303#comment-1303 ] Zhijie Shen commented on YARN-1637: --- Upload a new patch. bq. We don't need separate constants like ATS_WEBAPP_ADDRESS etc. We don't need to declare these separate addresses. bq. Similarly the constants for the defaults. Done bq. Rename ATSClient to something like TimelineClient ? bq. Similarly the implementation and all other references. bq. Actually, should we also drop 'ATS' prefix from the user-facing events? Done bq. Remove the references to the REST API in the documentation of ATSClient. It may or may not be REST in the future, it's an implementation detail. bq. Similarly fix the exception messages in ATSClientImpl to not refer to rest API. Done bq. Document YarnJacksonJaxbJsonProvider - javadoc etc. Is it needed in this patch? If not let's push it out to the patches where it is needed. Add a javadoc. Yes, it's required by TimelineClientImpl, which will help serialize the pojo objects into the json format that the server side can deserialize back to the corresponding pojo objects. > Implement a client library for java users to post entities+events > - > > Key: YARN-1637 > URL: https://issues.apache.org/jira/browse/YARN-1637 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Zhijie Shen > Attachments: YARN-1637.1.patch, YARN-1637.2.patch > > > This is a wrapper around the web-service to facilitate easy posting of > entity+event data to the time-line server. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1637) Implement a client library for java users to post entities+events
[ https://issues.apache.org/jira/browse/YARN-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1637: -- Attachment: YARN-1637.2.patch > Implement a client library for java users to post entities+events > - > > Key: YARN-1637 > URL: https://issues.apache.org/jira/browse/YARN-1637 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Zhijie Shen > Attachments: YARN-1637.1.patch, YARN-1637.2.patch > > > This is a wrapper around the web-service to facilitate easy posting of > entity+event data to the time-line server. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1667) Make admin refreshSuperUserGroupsConfiguration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888758#comment-13888758 ] Vinod Kumar Vavilapalli commented on YARN-1667: --- Similar comment as I put on YARN-1676. The point of "refreshSuperUserGroupsConf" call is that the proxy-user-groups mappings as returned by the configuration have changed (locally or remotely). So the test should also simulate a change of the configuration and validate the same after refresh. > Make admin refreshSuperUserGroupsConfiguration work across RM failover > -- > > Key: YARN-1667 > URL: https://issues.apache.org/jira/browse/YARN-1667 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-1667.1.patch, YARN-1667.2.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1669) Make admin refreshServiceAcls work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888754#comment-13888754 ] Vinod Kumar Vavilapalli commented on YARN-1669: --- ServiceAuthorizationManager shouldn't even know about things like useRemoteConfiguration. I think you just need a new API which takes a refresheConfiguration and doesn't load any files from the system property hadoop.policy.file. > Make admin refreshServiceAcls work across RM failover > - > > Key: YARN-1669 > URL: https://issues.apache.org/jira/browse/YARN-1669 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-1669.1.patch, YARN-1669.2.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects
[
https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888752#comment-13888752
]
Billie Rinaldi commented on YARN-1659:
--
I agree removing Entity is a good idea. I've made the change, and will just
wait for Zhijie's feedback before I upload another patch.
Regarding the field names, the API retrieves objects that look like the
following, where entity and entitytype are mandatory and the remaining fields
are optional. I think it makes sense to have the enum values for Field match
these field names, as opposed to having to do some translation between the two.
But it's not a big deal, I can add the underscores if you would prefer.
{noformat}
{
"entity": ...,
"entitytype": ...
"relatedentities": ...,
"primaryfilters": ...,
"secondaryfilters": ...,
"otherinfo": ...,
}
{noformat}
> Define ApplicationTimelineStore interface and store-facing entity,
> entity-info and event objects
>
>
> Key: YARN-1659
> URL: https://issues.apache.org/jira/browse/YARN-1659
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Billie Rinaldi
>Assignee: Billie Rinaldi
> Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch,
> YARN-1659-5.patch, YARN-1659-6.patch, YARN-1659.2.patch
>
>
> These will be used by ApplicationTimelineStore interface. The web services
> will convert the store-facing obects to the user-facing objects.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888750#comment-13888750 ] Vinod Kumar Vavilapalli commented on YARN-1611: --- Also, once you make these changes, the configuration stuff is completely decoupled from HA. The tests in the patch can go to a different class-file than TestRMFailOver. > Make admin refresh of capacity scheduler configuration work across RM failover > -- > > Key: YARN-1611 > URL: https://issues.apache.org/jira/browse/YARN-1611 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-1611.1.patch, YARN-1611.2.patch, YARN-1611.2.patch, > YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch, > YARN-1611.6.patch, YARN-1611.7.patch > > > Currently, If we do refresh* for a standby RM, it will failover to the > current active RM, and do the refresh* based on the local configuration file > of the active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1668) Make admin refreshAdminAcls work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888749#comment-13888749 ] Vinod Kumar Vavilapalli commented on YARN-1668: --- Once you do the suggestions I made in YARN-1611, the configuration stuff is decoupled from HA. The tests in the patch can go to a different class-file than TestRMFailOver. > Make admin refreshAdminAcls work across RM failover > --- > > Key: YARN-1668 > URL: https://issues.apache.org/jira/browse/YARN-1668 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-1668.1.patch, YARN-1668.2.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1676) Make admin refreshUserToGroupsMappings of configuration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888747#comment-13888747 ] Vinod Kumar Vavilapalli commented on YARN-1676: --- The point of "refreshUserToGroupsMappings" call is that the user-to-group mappings as returned by the GroupMappingProvider have changed (locally or remotely). So the test should also simulate an addition/removal of a user's group and validate the same after refresh. > Make admin refreshUserToGroupsMappings of configuration work across RM > failover > --- > > Key: YARN-1676 > URL: https://issues.apache.org/jira/browse/YARN-1676 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-1676.1.patch, YARN-1676.2.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1530) [Umbrella] Store, manage and serve per-framework application-timeline data
[ https://issues.apache.org/jira/browse/YARN-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888738#comment-13888738 ] Vinod Kumar Vavilapalli commented on YARN-1530: --- Thanks for the comments everyone. Some responses follow: Lohit bq. Yes, proxy server inside library, but only in AM not containers. Yes, that's a good idea and inline with what I might have hinted in the doc. When containers also start writing out events, we need to have these intermediate aggregators for scalability. I was thinking NodeManagers but your idea of AMs is better. If we make everyone only depend on a library, the underlying protocol can be either RPC or REST together with a RPC server or a proxy like you mentioned. If REST is itself the first-class API, yes, proxy-server is the way. Bobby, bq. I am also a bit nervous about using the history service for recovery or as a backend for the current MR APIs if we have a pub/sub system as a link between the applications and the history service. Agreed. This is a problem that exists even today with MR apps in a way with HDFS acting as a pubsub system. We've seen the corner cases you are hinting at even with the HDFS based history files. I think we now understand enough about these edge cases. I haven't yet made the jump to making these events be the base for recovery, but points noted for when we wish to. Chris, bq. 1. Is the expectation that people will be able to use this for INFO, WARN, ERROR type application logging? Application logging is not supposed to be through this channel. This feature is fundamentally for meta-event information - frankly, the kind we log in MR JobHistory files. We already have the ability for containers to write logs to a local log-directory and the corresponding features for aggregation, rendering. This is a high throughput path that IMO cannot be sustained by the solution of this JIRA. I'll make it clear in the doc. bq. 2. Regarding app-specific UIs, is it going to be possible to embed app-specific UIs with YARN's UI, instead of having to run an app-specific web-ui? There is some mention of JS UIs, but it's a little unclear whether this would be embedded in YARN, or served from "somewhere" (to quote the docs. If it's served from the RM (or some other web-ui in YARN), will it be up to ops to decide which libraries are embedded, or up to I thought I wrote it clearly, will have to reread and edit if necessary. The idea is to either let users host their UI's elsewhere or give an ability for admins to install UIs (that they have vetted for stability/security etc) in the HIstoryServer itself. bq. 3. What are the planned "out of the box implementations" for both the storage and transport layers? REST+LevelDB+HBase? Are Flume and Kafka implementations expected to happen outside of the YARN project? We are planning Library(wrapping REST)+LevelDB+LocalFS for smaller deployments and out of the box. We will then move to working on Library(wrapping REST)+LevelDB+HBase for larger deployments. The discussion of Flume/Kafka is related to the event-aggregation which we currently are doing via a simple REST put to a web-service. Whether/when we move to those implementations will depend on the urgency with which we run into issues mentioned in the doc with REST APIs. But it's open at this point of time. > [Umbrella] Store, manage and serve per-framework application-timeline data > -- > > Key: YARN-1530 > URL: https://issues.apache.org/jira/browse/YARN-1530 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli > Attachments: application timeline design-20140108.pdf, application > timeline design-20140116.pdf, application timeline design-20140130.pdf > > > This is a sibling JIRA for YARN-321. > Today, each application/framework has to do store, and serve per-framework > data all by itself as YARN doesn't have a common solution. This JIRA attempts > to solve the storage, management and serving of per-framework data from > various applications, both running and finished. The aim is to change YARN to > collect and store data in a generic manner with plugin points for frameworks > to do their own thing w.r.t interpretation and serving. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1634) Define an in-memory implementation of ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888735#comment-13888735 ] Vinod Kumar Vavilapalli commented on YARN-1634: --- Looks fine enough. Will look at the test more carefully once YARN-1659 is in. > Define an in-memory implementation of ApplicationTimelineStore > -- > > Key: YARN-1634 > URL: https://issues.apache.org/jira/browse/YARN-1634 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Zhijie Shen > Attachments: YARN-1634.1.patch > > > As per the design doc, the store needs to pluggable. We need a base > interface, and an in-memory implementation for testing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects
[ https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888733#comment-13888733 ] Vinod Kumar Vavilapalli commented on YARN-1659: --- Looks better. Few more questions: - you still have a o.a.h.y.server.applicationhistoryservice.apptimeline.Entity. Seems like it isn't needed anymore if you change the Reader's getEntity() API to take in both the ID and Type? bq. I don't think we should make this change, as the enum values currently match the names of the optional member variables of the ATSEntity object. Note sure I follow. Zhijie, you want to look at the patch too? > Define ApplicationTimelineStore interface and store-facing entity, > entity-info and event objects > > > Key: YARN-1659 > URL: https://issues.apache.org/jira/browse/YARN-1659 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, > YARN-1659-5.patch, YARN-1659-6.patch, YARN-1659.2.patch > > > These will be used by ApplicationTimelineStore interface. The web services > will convert the store-facing obects to the user-facing objects. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-1635: - Attachment: YARN-1635.3.patch Rebased patch on new YARN-1659 patch. > Implement a Leveldb based ApplicationTimelineStore > -- > > Key: YARN-1635 > URL: https://issues.apache.org/jira/browse/YARN-1635 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Billie Rinaldi > Attachments: YARN-1635.1.patch, YARN-1635.2.patch, YARN-1635.3.patch > > > As per the design doc, we need a levelDB + local-filesystem based > implementation to start with and for small deployments. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects
[ https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-1659: - Attachment: YARN-1659-6.patch Attaching a new patch including the following changes: * converted the interface to use the user-facing objects where possible (this involved some modifications and additions to the user-facing objects, which are included in this patch) * added javadocs * set interface audience appropriately * addressed the comment for the GenericObjectMapper and removed it from this patch, will include it in a patch for YARN-1635 * documented default limit * changed fields to fieldsToRetrieve * checked initial line of license blocks I think the only comment I haven't addressed is adding underscores into the Field enum values. I don't think we should make this change, as the enum values currently match the names of the optional member variables of the ATSEntity object. Please let me know if I missed anything or if you have any additional comments for the new patch. One thing I am unsure of is the best method to return error codes in the new ATSPutErrors class. Currently an error code is just an integer, but could easily be an enum instead. > Define ApplicationTimelineStore interface and store-facing entity, > entity-info and event objects > > > Key: YARN-1659 > URL: https://issues.apache.org/jira/browse/YARN-1659 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, > YARN-1659-5.patch, YARN-1659-6.patch, YARN-1659.2.patch > > > These will be used by ApplicationTimelineStore interface. The web services > will convert the store-facing obects to the user-facing objects. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1633) Define user-faced entity, entity-info and event objects
[ https://issues.apache.org/jira/browse/YARN-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888594#comment-13888594 ] Hudson commented on YARN-1633: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1660 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1660/]) YARN-1633. Defined user-facing entity, entity-info and event objects related to Application Timeline feature. Contributed by Zhijie Shen. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563356) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntities.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntity.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvents.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/package-info.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java > Define user-faced entity, entity-info and event objects > --- > > Key: YARN-1633 > URL: https://issues.apache.org/jira/browse/YARN-1633 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Zhijie Shen > Fix For: 2.4.0 > > Attachments: YARN-1633.1.patch, YARN-1633.2.patch > > > Define the core objects of the application-timeline effort. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1632) TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package
[ https://issues.apache.org/jira/browse/YARN-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888595#comment-13888595 ] Hudson commented on YARN-1632: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1660 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1660/]) YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563160) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563157) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice/TestApplicationMasterService.java > TestApplicationMasterServices should be under > org.apache.hadoop.yarn.server.resourcemanager package > --- > > Key: YARN-1632 > URL: https://issues.apache.org/jira/browse/YARN-1632 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.9, 2.2.0 >Reporter: Chen He >Assignee: Chen He >Priority: Minor > Fix For: 3.0.0, 2.4.0 > > Attachments: yarn-1632v2.patch > > > ApplicationMasterService is under > org.apache.hadoop.yarn.server.resourcemanager package. However, its unit test > file TestApplicationMasterService is placed under > org.apache.hadoop.yarn.server.resourcemanager.applicationmasterservice > package which only contains one file (TestApplicationMasterService). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1504) RM changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888590#comment-13888590 ] Hudson commented on YARN-1504: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1660 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1660/]) YARN-1504. RM changes for moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563371) * /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppMoveEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestMoveApplication.java > RM changes for moving apps between queues > - > > Key: YARN-1504 > URL: https://issues.apache.org/jira/browse/YARN-1504 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 3.0.0 > > Attachments: YARN-1504-1.patch, YARN-1504.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1632) TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package
[ https://issues.apache.org/jira/browse/YARN-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888581#comment-13888581 ] Hudson commented on YARN-1632: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1685 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1685/]) YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563160) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563157) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice/TestApplicationMasterService.java > TestApplicationMasterServices should be under > org.apache.hadoop.yarn.server.resourcemanager package > --- > > Key: YARN-1632 > URL: https://issues.apache.org/jira/browse/YARN-1632 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.9, 2.2.0 >Reporter: Chen He >Assignee: Chen He >Priority: Minor > Fix For: 3.0.0, 2.4.0 > > Attachments: yarn-1632v2.patch > > > ApplicationMasterService is under > org.apache.hadoop.yarn.server.resourcemanager package. However, its unit test > file TestApplicationMasterService is placed under > org.apache.hadoop.yarn.server.resourcemanager.applicationmasterservice > package which only contains one file (TestApplicationMasterService). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1504) RM changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888576#comment-13888576 ] Hudson commented on YARN-1504: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1685 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1685/]) YARN-1504. RM changes for moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563371) * /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppMoveEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestMoveApplication.java > RM changes for moving apps between queues > - > > Key: YARN-1504 > URL: https://issues.apache.org/jira/browse/YARN-1504 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 3.0.0 > > Attachments: YARN-1504-1.patch, YARN-1504.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1633) Define user-faced entity, entity-info and event objects
[ https://issues.apache.org/jira/browse/YARN-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888580#comment-13888580 ] Hudson commented on YARN-1633: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1685 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1685/]) YARN-1633. Defined user-facing entity, entity-info and event objects related to Application Timeline feature. Contributed by Zhijie Shen. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563356) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntities.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntity.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvents.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/package-info.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java > Define user-faced entity, entity-info and event objects > --- > > Key: YARN-1633 > URL: https://issues.apache.org/jira/browse/YARN-1633 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Zhijie Shen > Fix For: 2.4.0 > > Attachments: YARN-1633.1.patch, YARN-1633.2.patch > > > Define the core objects of the application-timeline effort. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1632) TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package
[ https://issues.apache.org/jira/browse/YARN-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888531#comment-13888531 ] Hudson commented on YARN-1632: -- FAILURE: Integrated in Hadoop-Yarn-trunk #468 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/468/]) YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563160) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563157) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice/TestApplicationMasterService.java > TestApplicationMasterServices should be under > org.apache.hadoop.yarn.server.resourcemanager package > --- > > Key: YARN-1632 > URL: https://issues.apache.org/jira/browse/YARN-1632 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.9, 2.2.0 >Reporter: Chen He >Assignee: Chen He >Priority: Minor > Fix For: 3.0.0, 2.4.0 > > Attachments: yarn-1632v2.patch > > > ApplicationMasterService is under > org.apache.hadoop.yarn.server.resourcemanager package. However, its unit test > file TestApplicationMasterService is placed under > org.apache.hadoop.yarn.server.resourcemanager.applicationmasterservice > package which only contains one file (TestApplicationMasterService). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1633) Define user-faced entity, entity-info and event objects
[ https://issues.apache.org/jira/browse/YARN-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888530#comment-13888530 ] Hudson commented on YARN-1633: -- FAILURE: Integrated in Hadoop-Yarn-trunk #468 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/468/]) YARN-1633. Defined user-facing entity, entity-info and event objects related to Application Timeline feature. Contributed by Zhijie Shen. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563356) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntities.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntity.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvents.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/package-info.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java > Define user-faced entity, entity-info and event objects > --- > > Key: YARN-1633 > URL: https://issues.apache.org/jira/browse/YARN-1633 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Zhijie Shen > Fix For: 2.4.0 > > Attachments: YARN-1633.1.patch, YARN-1633.2.patch > > > Define the core objects of the application-timeline effort. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1504) RM changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888526#comment-13888526 ] Hudson commented on YARN-1504: -- FAILURE: Integrated in Hadoop-Yarn-trunk #468 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/468/]) YARN-1504. RM changes for moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563371) * /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppMoveEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestMoveApplication.java > RM changes for moving apps between queues > - > > Key: YARN-1504 > URL: https://issues.apache.org/jira/browse/YARN-1504 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 3.0.0 > > Attachments: YARN-1504-1.patch, YARN-1504.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
