[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: YARN-6413.v7.patch Fix Checkstyle and whitespace > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch, YARN-6413.v2.patch, > YARN-6413.v3.patch, YARN-6413.v4.patch, YARN-6413.v5.patch, > YARN-6413.v6.patch, YARN-6413.v7.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: YARN-6413.v6.patch Addressed [~jianhe]'s comments. Hi Jian, a quick question: will the yarn-native-services branch add the option to configure which implementing class to use for registry? > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch, YARN-6413.v2.patch, > YARN-6413.v3.patch, YARN-6413.v4.patch, YARN-6413.v5.patch, YARN-6413.v6.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: YARN-6413.v5.patch Fix Checkstyle. The asflicense failures are caused by https://issues.apache.org/jira/browse/HADOOP-14990 > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch, YARN-6413.v2.patch, > YARN-6413.v3.patch, YARN-6413.v4.patch, YARN-6413.v5.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: YARN-6413.v4.patch > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch, YARN-6413.v2.patch, > YARN-6413.v3.patch, YARN-6413.v4.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: (was: YARN-6413.v3.patch) > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch, YARN-6413.v2.patch, > YARN-6413.v3.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: YARN-6413.v3.patch Reupload to rerun jenkins > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch, YARN-6413.v2.patch, > YARN-6413.v3.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: YARN-6413.v3.patch Remove configuration (should be added in a different patch, not being used here), add licenses > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch, YARN-6413.v2.patch, > YARN-6413.v3.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: YARN-6413.v2.patch Fix findbugs error. > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch, YARN-6413.v2.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Description: Add a RegistryOperations implementation that writes records to the file system. This does not include any changes to the API, to avoid compatibility issues. (was: Add a RegistryOperations implementation that writes records to the file system.) > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch > > > Add a RegistryOperations implementation that writes records to the file > system. This does not include any changes to the API, to avoid compatibility > issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: YARN-6413.v1.patch We've decided to stick with the existing API, in order to avoid the compatibility issues raised by [~jianhe]. This patch now only contains a FS-based implementation and no API changes. > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch, YARN-6413.v1.patch > > > Add a RegistryOperations implementation that writes records to the file > system. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch First patch with no API changes, only store implementation. > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-YARN-6413-Yarn-Registry-FS-Implementation.patch, > 0002-Registry-API-v2.patch, 0003-Registry-API-api-only.patch, > 0004-Registry-API-api-stubbed.patch > > > Add a RegistryOperations implementation that writes records to the file > system. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Description: Add a RegistryOperations implementation that writes records to the file system. (was: Right now the Yarn Registry API (defined in the RegistryOperations interface) is a very thin layer over Zookeeper. This jira proposes changing the interface to abstract away the implementation details so that we can write a FS-based implementation of the registry service, which will be used to support AMRMProxy HA. The new interface will use register/delete/resolve APIs instead of Zookeeper-specific operations like mknode. ) > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch, > 0003-Registry-API-api-only.patch, 0004-Registry-API-api-stubbed.patch > > > Add a RegistryOperations implementation that writes records to the file > system. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Yarn Registry FS implementation
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Summary: Yarn Registry FS implementation (was: Decouple Yarn Registry API from ZK) > Yarn Registry FS implementation > --- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch, > 0003-Registry-API-api-only.patch, 0004-Registry-API-api-stubbed.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5603) Metrics for Federation StateStore
[ https://issues.apache.org/jira/browse/YARN-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5603: Attachment: YARN-5603.003.patch One more checkstyle fix. > Metrics for Federation StateStore > - > > Key: YARN-5603 > URL: https://issues.apache.org/jira/browse/YARN-5603 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: 0001-Add-Federation-Client-metrics.patch, > YARN-5603.001.patch, YARN-5603.002.patch, YARN-5603.003.patch > > > This JIRA proposes addition of metrics for Federation StateStore -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5603) Metrics for Federation StateStore
[ https://issues.apache.org/jira/browse/YARN-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5603: Attachment: YARN-5603.002.patch Checkstyle fixes > Metrics for Federation StateStore > - > > Key: YARN-5603 > URL: https://issues.apache.org/jira/browse/YARN-5603 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: 0001-Add-Federation-Client-metrics.patch, > YARN-5603.001.patch, YARN-5603.002.patch > > > This JIRA proposes addition of metrics for Federation StateStore -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5603) Metrics for Federation StateStore
[ https://issues.apache.org/jira/browse/YARN-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5603: Attachment: YARN-5603.001.patch Renamed the file and capitalized all the static final fields. > Metrics for Federation StateStore > - > > Key: YARN-5603 > URL: https://issues.apache.org/jira/browse/YARN-5603 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: 0001-Add-Federation-Client-metrics.patch, > YARN-5603.001.patch > > > This JIRA proposes addition of metrics for Federation StateStore -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5603) Metrics for Federation StateStore
[ https://issues.apache.org/jira/browse/YARN-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131592#comment-16131592 ] Ellen Hui commented on YARN-5603: - [~asuresh], could you please take a look? > Metrics for Federation StateStore > - > > Key: YARN-5603 > URL: https://issues.apache.org/jira/browse/YARN-5603 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: 0001-Add-Federation-Client-metrics.patch > > > This JIRA proposes addition of metrics for Federation StateStore -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5603) Metrics for Federation StateStore
[ https://issues.apache.org/jira/browse/YARN-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5603: Attachment: 0001-Add-Federation-Client-metrics.patch Add Federation StateStore Client metrics, and instrumentation for SQLFederationStateStore. > Metrics for Federation StateStore > - > > Key: YARN-5603 > URL: https://issues.apache.org/jira/browse/YARN-5603 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: 0001-Add-Federation-Client-metrics.patch > > > This JIRA proposes addition of metrics for Federation StateStore -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: 0004-Registry-API-api-stubbed.patch Hi [~jianhe], I've added some stubs, and a ServiceRecordKey implementation that just wraps the ZK path for the ZK implementation to use. The client can use it however they want, though I think it should be standardized one way or another. My thought is this: the ZK implementation will implement RegistryOperations, RegistryStoreProtocol, and RegistryListenerProtocol. The RegistryOperations methods will be stubbed out, so anything that relies on them will still compile, and yarn-native-services will compile. After yarn-native-services moves over to to actually use the new APIs, RegistryOperations can then be fully deprecated. I want to confirm with you that the methods in RegistryStoreProtocol and RegistryListenerProtocol, if I implement them and remove RegistryOperations entirely, will be ok for yarn-native-services. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch, > 0003-Registry-API-api-only.patch, 0004-Registry-API-api-stubbed.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: 0003-Registry-API-api-only.patch This patch only adds the API, record, types, and exceptions, the implementation has not been touched. It compiles on top of yarn-native-services f1a358e178e. [~jianhe], can you please take a look at the latest patch, and let me know if this way of splitting the interface will work for you? > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch, > 0003-Registry-API-api-only.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099352#comment-16099352 ] Ellen Hui edited comment on YARN-6413 at 7/25/17 1:10 AM: -- bq. looks like RegistryUtils#extractServiceRecords is removed. Do you mind change RegistryDNSServer to use the new methods ? (i.e. make this patch compile with yarn-native-services branch) Ah, I see. RegistryUtils#extractServiceRecords is one of the methods that depends on having the hierarchical namespace, which is why it was removed. I can add a select-multiple method with a filter of some sort, would that work? The point of this from our end was to abstract away the path. Right now yarn-native-services doesn't compile for me with a pom.xml error (pulled just now), is the branch healthy? bq. The DNS today depends on the ZK path layout to reconstruct the service record. So changing the zk path will break DNS. Ok, I will put the ZK path layout back to the way it was before. was (Author: ellenfkh): bq:looks like RegistryUtils#extractServiceRecords is removed. Do you mind change RegistryDNSServer to use the new methods ? (i.e. make this patch compile with yarn-native-services branch) Ah, I see. RegistryUtils#extractServiceRecords is one of the methods that depends on having the hierarchical namespace, which is why it was removed. I can add a select-multiple method with a filter of some sort, would that work? The point of this from our end was to abstract away the path. Right now yarn-native-services doesn't compile for me with a pom.xml error (pulled just now), is the branch healthy? bq:The DNS today depends on the ZK path layout to reconstruct the service record. So changing the zk path will break DNS. Ok, I will put the ZK path layout back to the way it was before. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099352#comment-16099352 ] Ellen Hui commented on YARN-6413: - bq: looks like RegistryUtils#extractServiceRecords is removed. Do you mind change RegistryDNSServer to use the new methods ? (i.e. make this patch compile with yarn-native-services branch) Ah, I see. RegistryUtils#extractServiceRecords is one of the methods that depends on having the hierarchical namespace, which is why it was removed. I can add a select-multiple method with a filter of some sort, would that work? The point of this from our end was to abstract away the path. Right now yarn-native-services doesn't compile for me with a pom.xml error (pulled just now), is the branch healthy? bq: The DNS today depends on the ZK path layout to reconstruct the service record. So changing the zk path will break DNS. Ok, I will put the ZK path layout back to the way it was before. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099352#comment-16099352 ] Ellen Hui edited comment on YARN-6413 at 7/25/17 1:09 AM: -- bq:looks like RegistryUtils#extractServiceRecords is removed. Do you mind change RegistryDNSServer to use the new methods ? (i.e. make this patch compile with yarn-native-services branch) Ah, I see. RegistryUtils#extractServiceRecords is one of the methods that depends on having the hierarchical namespace, which is why it was removed. I can add a select-multiple method with a filter of some sort, would that work? The point of this from our end was to abstract away the path. Right now yarn-native-services doesn't compile for me with a pom.xml error (pulled just now), is the branch healthy? bq:The DNS today depends on the ZK path layout to reconstruct the service record. So changing the zk path will break DNS. Ok, I will put the ZK path layout back to the way it was before. was (Author: ellenfkh): bq: looks like RegistryUtils#extractServiceRecords is removed. Do you mind change RegistryDNSServer to use the new methods ? (i.e. make this patch compile with yarn-native-services branch) Ah, I see. RegistryUtils#extractServiceRecords is one of the methods that depends on having the hierarchical namespace, which is why it was removed. I can add a select-multiple method with a filter of some sort, would that work? The point of this from our end was to abstract away the path. Right now yarn-native-services doesn't compile for me with a pom.xml error (pulled just now), is the branch healthy? bq: The DNS today depends on the ZK path layout to reconstruct the service record. So changing the zk path will break DNS. Ok, I will put the ZK path layout back to the way it was before. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098902#comment-16098902 ] Ellen Hui commented on YARN-6413: - bq. In RegistryDNSServer, all the methods removed are essential to registryDNS - the implementation of RegistryDNS is depending on the zookeeper’s listener functionality. This is one big difference from the state store implementation. we can not remove those methods. I did not remove any methods, I just didn't add the ones that exist in yarn-native-services but not trunk. I thought doing so would probably cause more conflicts than not. bq. The path is used by RegistryDNSServer for reconstructing the DNS record(e.g. BaseServiceRecordProcessor#getContainerName). If we change the path, everything there will break. Also, the registry documentation needs to change. My understanding was that DNS went through ZK directly without going through the interface, so it wouldn't be affected by the service records setting up the path differently. I can change the path construction back for the ZK impl if it needs that. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: 0002-Registry-API-v2.patch Addressed comments from [~jianhe]: * The V2 has every method used by RegistryDNS except for monitorRegistryEntries and registrPathListener, which were added in a yarn-native-services commit. * Removed extra imports from RegistryOperations * The path was rearranged in order to flatten the namespace * removed getSubpathsForUserServiceRecordKey * Removed catch stub and replaced with a proper log > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, 0002-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: (was: 0001-WIP-Registry-API-v2.patch) > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050803#comment-16050803 ] Ellen Hui commented on YARN-6413: - I think you were looking at the old patch, I have removed it now. The current patch should not have formatting changes. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049646#comment-16049646 ] Ellen Hui commented on YARN-6413: - [~jianhe] I think I had cleaned up all of the formatting changes in the latest patch, could you point me to the files that have lots of format changes? There are a couple files that were renamed, so they will show up as large diffs, but the actual changes were small there. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-WIP-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: 0001-Registry-API-v2.patch Completed patch. [~jianhe], could you please especially check the security parts of this? I tried to preserve the behaviour of the original registry, as best as I understood it, but I'm not sure I got it right. This should yarn-native-sevices should compile against this with minimal effort, just fixing some class references on the other end. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-Registry-API-v2.patch, > 0001-WIP-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-6413: Attachment: 0001-WIP-Registry-API-v2.patch WIP. [~jianhe], could you please take a look at the API and let me know if it looks reasonable to you? If so, I will clean it up and finish the implementation and tests. > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > Attachments: 0001-WIP-Registry-API-v2.patch > > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6413) Decouple Yarn Registry API from ZK
[ https://issues.apache.org/jira/browse/YARN-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui reassigned YARN-6413: --- Assignee: Ellen Hui (was: Jian He) > Decouple Yarn Registry API from ZK > -- > > Key: YARN-6413 > URL: https://issues.apache.org/jira/browse/YARN-6413 > Project: Hadoop YARN > Issue Type: Improvement > Components: amrmproxy, api, resourcemanager >Reporter: Ellen Hui >Assignee: Ellen Hui > > Right now the Yarn Registry API (defined in the RegistryOperations interface) > is a very thin layer over Zookeeper. This jira proposes changing the > interface to abstract away the implementation details so that we can write a > FS-based implementation of the registry service, which will be used to > support AMRMProxy HA. > The new interface will use register/delete/resolve APIs instead of > Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6413) Decouple Yarn Registry API from ZK
Ellen Hui created YARN-6413: --- Summary: Decouple Yarn Registry API from ZK Key: YARN-6413 URL: https://issues.apache.org/jira/browse/YARN-6413 Project: Hadoop YARN Issue Type: Improvement Components: amrmproxy, api, resourcemanager Reporter: Ellen Hui Assignee: Jian He Right now the Yarn Registry API (defined in the RegistryOperations interface) is a very thin layer over Zookeeper. This jira proposes changing the interface to abstract away the implementation details so that we can write a FS-based implementation of the registry service, which will be used to support AMRMProxy HA. The new interface will use register/delete/resolve APIs instead of Zookeeper-specific operations like mknode. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5519) Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover
[
https://issues.apache.org/jira/browse/YARN-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5519:
Attachment: YARN-5519-YARN-2915.v4.patch
* Add [~subru]'s javadoc suggestions
* Fix typos in {{FederationStateStoreBaseTest}}
* Remove the line in {{MemoryFederationStore}} that modifies a
{{SubClusterInfo}}'s lastStartTime timestamp during registration
> Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover
>
>
> Key: YARN-5519
> URL: https://issues.apache.org/jira/browse/YARN-5519
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Giovanni Matteo Fumarola
>Assignee: Ellen Hui
> Attachments: YARN-5519-YARN-2915.v1.patch,
> YARN-5519-YARN-2915.v2.patch, YARN-5519-YARN-2915.v3.patch,
> YARN-5519-YARN-2915.v4.patch
>
>
> This JIRA tracks the addition of SubClusterId into
> AddApplicationHomeSubClusterResponse.
> in the design of [YARN-3659|https://issues.apache.org/jira/browse/YARN-3659],
> to handle better fail-over scenario the response needs SubclusterId as field.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5519) Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover
[
https://issues.apache.org/jira/browse/YARN-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421674#comment-15421674
]
Ellen Hui commented on YARN-5519:
-
The test failed on a timing issue, since the {{MemoryFederationStateStore}}
updates the lastHeartbeat field of a {{SubClusterInfo}} during registration,
causing the stored value and the expected one to differ slightly. I will remove
the line in {{MemoryFederationStateStore}} that modifies the timestamp, so that
the timestamp passed in through the {{SubClusterRegisterRequest}} is stored
unchanged.
> Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover
>
>
> Key: YARN-5519
> URL: https://issues.apache.org/jira/browse/YARN-5519
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Giovanni Matteo Fumarola
>Assignee: Ellen Hui
> Attachments: YARN-5519-YARN-2915.v1.patch,
> YARN-5519-YARN-2915.v2.patch, YARN-5519-YARN-2915.v3.patch
>
>
> This JIRA tracks the addition of SubClusterId into
> AddApplicationHomeSubClusterResponse.
> in the design of [YARN-3659|https://issues.apache.org/jira/browse/YARN-3659],
> to handle better fail-over scenario the response needs SubclusterId as field.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5519) Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover
[ https://issues.apache.org/jira/browse/YARN-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5519: Attachment: YARN-5519-YARN-2915.v3.patch Pick the fix for FederationStateStoreBaseTest from YARN-3672 > Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover > > > Key: YARN-5519 > URL: https://issues.apache.org/jira/browse/YARN-5519 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Giovanni Matteo Fumarola >Assignee: Ellen Hui > Attachments: YARN-5519-YARN-2915.v1.patch, > YARN-5519-YARN-2915.v2.patch, YARN-5519-YARN-2915.v3.patch > > > This JIRA tracks the addition of SubClusterId into > AddApplicationHomeSubClusterResponse. > in the design of [YARN-3659|https://issues.apache.org/jira/browse/YARN-3659], > to handle better fail-over scenario the response needs SubclusterId as field. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5519) Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover
[
https://issues.apache.org/jira/browse/YARN-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5519:
Attachment: YARN-5519-YARN-2915.v2.patch
Address feedback from [~subru]
* add Javadoc to new get/set in {{AddApplicationHomeSubClusterResponse}}
* update Javadoc for {{AddApplicationHomeSubCluster}} in
{{FederationApplicationHomeSubClusterStore}} to reflect the updated behavior
* removed unnecessary changes to {{testUpdateApplicationHomeSubCluster}}
> Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover
>
>
> Key: YARN-5519
> URL: https://issues.apache.org/jira/browse/YARN-5519
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Giovanni Matteo Fumarola
>Assignee: Ellen Hui
> Attachments: YARN-5519-YARN-2915.v1.patch,
> YARN-5519-YARN-2915.v2.patch
>
>
> This JIRA tracks the addition of SubClusterId into
> AddApplicationHomeSubClusterResponse.
> in the design of [YARN-3659|https://issues.apache.org/jira/browse/YARN-3659],
> to handle better fail-over scenario the response needs SubclusterId as field.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5519) Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover
[ https://issues.apache.org/jira/browse/YARN-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5519: Attachment: YARN-5519-YARN-2915.v1.patch > Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover > > > Key: YARN-5519 > URL: https://issues.apache.org/jira/browse/YARN-5519 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Giovanni Matteo Fumarola >Assignee: Ellen Hui > Attachments: YARN-5519-YARN-2915.v1.patch > > > This JIRA tracks the addition of SubClusterId into > AddApplicationHomeSubClusterResponse. > in the design of [YARN-3659|https://issues.apache.org/jira/browse/YARN-3659], > to handle better fail-over scenario the response needs SubclusterId as field. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5407) In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore
[ https://issues.apache.org/jira/browse/YARN-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5407: Attachment: YARN-5407-YARN-2915.v2.patch Wrong diff uploaded previously > In-memory based implementation of the FederationApplicationStateStore, > FederationPolicyStateStore > - > > Key: YARN-5407 > URL: https://issues.apache.org/jira/browse/YARN-5407 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5407-YARN-2915.v0.patch, > YARN-5407-YARN-2915.v1.patch, YARN-5407-YARN-2915.v2.patch > > > YARN-5307 defines the FederationApplicationStateStore API. YARN-3664 defines > the FederationPolicyStateStore API. This JIRA tracks an in-memory based > implementation which is useful for both single-box testing and for future > unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5407) In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore
[ https://issues.apache.org/jira/browse/YARN-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5407: Attachment: (was: YARN-5408-YARN-2915.v2.patch) > In-memory based implementation of the FederationApplicationStateStore, > FederationPolicyStateStore > - > > Key: YARN-5407 > URL: https://issues.apache.org/jira/browse/YARN-5407 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5407-YARN-2915.v0.patch, > YARN-5407-YARN-2915.v1.patch, YARN-5407-YARN-2915.v2.patch > > > YARN-5307 defines the FederationApplicationStateStore API. YARN-3664 defines > the FederationPolicyStateStore API. This JIRA tracks an in-memory based > implementation which is useful for both single-box testing and for future > unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5407) In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore
[
https://issues.apache.org/jira/browse/YARN-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5407:
Attachment: YARN-5408-YARN-2915.v2.patch
Address feedback from [~subru]
* Move queue from {{SetSubClusterPolicyConfigurationRequest}} to
{{SubClusterPolicyConfiguration}}
* Add helper methods for registerSubCluster/addApplication/setPolicy in
{{FederationStateStoreBaseTest}}
> In-memory based implementation of the FederationApplicationStateStore,
> FederationPolicyStateStore
> -
>
> Key: YARN-5407
> URL: https://issues.apache.org/jira/browse/YARN-5407
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5407-YARN-2915.v0.patch,
> YARN-5407-YARN-2915.v1.patch, YARN-5408-YARN-2915.v2.patch
>
>
> YARN-5307 defines the FederationApplicationStateStore API. YARN-3664 defines
> the FederationPolicyStateStore API. This JIRA tracks an in-memory based
> implementation which is useful for both single-box testing and for future
> unit tests that depend on the state store.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5407) In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore
[ https://issues.apache.org/jira/browse/YARN-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5407: Attachment: YARN-5407-YARN-2915.v1.patch > In-memory based implementation of the FederationApplicationStateStore, > FederationPolicyStateStore > - > > Key: YARN-5407 > URL: https://issues.apache.org/jira/browse/YARN-5407 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5407-YARN-2915.v0.patch, > YARN-5407-YARN-2915.v1.patch > > > YARN-5307 defines the FederationApplicationStateStore API. YARN-3664 defines > the FederationPolicyStateStore API. This JIRA tracks an in-memory based > implementation which is useful for both single-box testing and for future > unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5408) Compose Federation membership/application/policy APIs into an uber FederationStateStore API
[ https://issues.apache.org/jira/browse/YARN-5408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5408: Attachment: YARN-5408-YARN-2915.v2.patch Address comments from [~subru]. > Compose Federation membership/application/policy APIs into an uber > FederationStateStore API > --- > > Key: YARN-5408 > URL: https://issues.apache.org/jira/browse/YARN-5408 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5408-YARN-2915.v1.patch, > YARN-5408-YARN-2915.v2.patch > > > This is a simple composition of the three APIs defined in YARN-3664, > YARN-5307, YARN-3662. This is for convenience so that we provide a single API > for both implementations and for consumers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5408) Compose Federation membership/application/policy APIs into an uber FederationStateStore API
[ https://issues.apache.org/jira/browse/YARN-5408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5408: Attachment: YARN-5408-YARN-2915.v1.patch > Compose Federation membership/application/policy APIs into an uber > FederationStateStore API > --- > > Key: YARN-5408 > URL: https://issues.apache.org/jira/browse/YARN-5408 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5408-YARN-2915.v1.patch > > > This is a simple composition of the three APIs defined in YARN-3664, > YARN-5307, YARN-3662. This is for convenience so that we provide a single API > for both implementations and for consumers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5407) In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore
[ https://issues.apache.org/jira/browse/YARN-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5407: Description: YARN-5307 defines the FederationApplicationStateStore API. YARN-3664 defines the FederationPolicyStateStore API. This JIRA tracks an in-memory based implementation which is useful for both single-box testing and for future unit tests that depend on the state store. (was: YARN-5307 defines the FederationApplicationStateStore API. This JIRA tracks an in-memory based implementation which is useful for both single-box testing and for future unit tests that depend on the state store.) > In-memory based implementation of the FederationApplicationStateStore, > FederationPolicyStateStore > - > > Key: YARN-5407 > URL: https://issues.apache.org/jira/browse/YARN-5407 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5407-YARN-2915.v0.patch > > > YARN-5307 defines the FederationApplicationStateStore API. YARN-3664 defines > the FederationPolicyStateStore API. This JIRA tracks an in-memory based > implementation which is useful for both single-box testing and for future > unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5408) Define overall API for FederationStateStore
[ https://issues.apache.org/jira/browse/YARN-5408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5408: Description: This is a simple composition of the three APIs defined in YARN-3664, YARN-5307, YARN-3662 (was: This is a simple composition of the three APIs defined in YARN-3664, YARN-5307, ) > Define overall API for FederationStateStore > --- > > Key: YARN-5408 > URL: https://issues.apache.org/jira/browse/YARN-5408 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > > This is a simple composition of the three APIs defined in YARN-3664, > YARN-5307, YARN-3662 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5408) Define overall API for FederationStateStore
[ https://issues.apache.org/jira/browse/YARN-5408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5408: Description: This is a simple composition of the three APIs defined in YARN-3664, YARN-5307, (was: YARN-3664 defines the FederationPolicyStore API. This JIRA tracks an in-memory based implementation which is useful for both single-box testing and for future unit tests that depend on the state store.) > Define overall API for FederationStateStore > --- > > Key: YARN-5408 > URL: https://issues.apache.org/jira/browse/YARN-5408 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > > This is a simple composition of the three APIs defined in YARN-3664, > YARN-5307, -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5407) In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore
[ https://issues.apache.org/jira/browse/YARN-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5407: Summary: In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore (was: In-memory based implementation of the FederationApplicationStateStore) > In-memory based implementation of the FederationApplicationStateStore, > FederationPolicyStateStore > - > > Key: YARN-5407 > URL: https://issues.apache.org/jira/browse/YARN-5407 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5407-YARN-2915.v0.patch > > > YARN-5307 defines the FederationApplicationStateStore API. This JIRA tracks > an in-memory based implementation which is useful for both single-box testing > and for future unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5408) Define overall API for FederationStateStore
[ https://issues.apache.org/jira/browse/YARN-5408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5408: Summary: Define overall API for FederationStateStore (was: In-memory based implementation of the FederationPolicyStore) > Define overall API for FederationStateStore > --- > > Key: YARN-5408 > URL: https://issues.apache.org/jira/browse/YARN-5408 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > > YARN-3664 defines the FederationPolicyStore API. This JIRA tracks an > in-memory based implementation which is useful for both single-box testing > and for future unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5407) In-memory based implementation of the FederationApplicationStateStore
[ https://issues.apache.org/jira/browse/YARN-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5407: Attachment: YARN-5407-YARN-2915.v0.patch > In-memory based implementation of the FederationApplicationStateStore > - > > Key: YARN-5407 > URL: https://issues.apache.org/jira/browse/YARN-5407 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5407-YARN-2915.v0.patch > > > YARN-5307 defines the FederationApplicationStateStore API. This JIRA tracks > an in-memory based implementation which is useful for both single-box testing > and for future unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5390) Federation Subcluster Resolver
[ https://issues.apache.org/jira/browse/YARN-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5390: Attachment: YARN-5390-YARN-2915.v4.patch Fix indentation in pom.xml > Federation Subcluster Resolver > -- > > Key: YARN-5390 > URL: https://issues.apache.org/jira/browse/YARN-5390 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Carlo Curino >Assignee: Ellen Hui > Attachments: YARN-5390-YARN-2915.v0.patch, > YARN-5390-YARN-2915.v1.patch, YARN-5390-YARN-2915.v2.patch, > YARN-5390-YARN-2915.v3.patch, YARN-5390-YARN-2915.v4.patch > > > This JIRA tracks effort to create a mechanism to resolve nodes/racks resource > names to sub-cluster identifiers. This is needed by the federation policies > in YARN-5323, YARN-5324, YARN-5325 to operate correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5390) Federation Subcluster Resolver
[ https://issues.apache.org/jira/browse/YARN-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5390: Attachment: YARN-5390-YARN-2915.v3.patch Address feedback from [~subru]. > Federation Subcluster Resolver > -- > > Key: YARN-5390 > URL: https://issues.apache.org/jira/browse/YARN-5390 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Carlo Curino >Assignee: Ellen Hui > Attachments: YARN-5390-YARN-2915.v0.patch, > YARN-5390-YARN-2915.v1.patch, YARN-5390-YARN-2915.v2.patch, > YARN-5390-YARN-2915.v3.patch > > > This JIRA tracks effort to create a mechanism to resolve nodes/racks resource > names to sub-cluster identifiers. This is needed by the federation policies > in YARN-5323, YARN-5324, YARN-5325 to operate correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5406) In-memory based implementation of the FederationMembershipStateStore
[ https://issues.apache.org/jira/browse/YARN-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5406: Attachment: YARN-5406-YARN-2915.v1.patch Addressed feedback from [~subru] and [~jianhe]. > In-memory based implementation of the FederationMembershipStateStore > > > Key: YARN-5406 > URL: https://issues.apache.org/jira/browse/YARN-5406 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5406-YARN-2915.v0.patch, > YARN-5406-YARN-2915.v1.patch > > > YARN-3662 defines the FederationMembershipStateStore API. This JIRA tracks an > in-memory based implementation which is useful for both single-box testing > and for future unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5390) Federation Subcluster Resolver
[
https://issues.apache.org/jira/browse/YARN-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403189#comment-15403189
]
Ellen Hui edited comment on YARN-5390 at 8/2/16 10:38 PM:
--
Hi [~leftnoteasy], thanks for the quick feedback!
* This interface will be used in the three patches you looked at, although you
are correct that they have not been updated yet. For instance, the
LocalityMulticastAMRMProxyFederationPolicy prototype in YARN-5325 uses the
FederationSubClusterResolver interface to split resource requests. There are
some examples of the resolver being used in the splitResourceRequests method of
that class, although some of the classnames are out of date. From the javadoc:
{panel}
host localized ResourceRequest are always forwarded to the RM
that owns the node, based on the feedback of a FederationSubClusterResolver
rack localized ResourceRequest are forwarded to the RM that owns
the rack (if the FederationSubClusterResolver provides this info) or
they are forwarded as if they were ANY (this is important for deployment that
stripe racks across sub-clusters) as there is not a single resolution.
ANY request corresponding to node/rack local requests are only forwarded
to the set of RMs that owns the node-local requests. The number of containers
listed in each ANY is proportional to the number of node-local container
requests (associated to this ANY via the same allocateRequestId)
{panel}
* The FederationInterceptor from YARN-5325 will be responsible for managing the
lifecyle of the SubClusterResolver.
* I think it's better to leave the SubClusterResolver methods non-static, since
we want to allow the implementation to be pluggable and I can't think of a
particular reason it should be static. Please let me know if you disagree, I
may be missing something.
Thanks!
was (Author: ellenfkh):
Hi [~wangda], thanks for the quick feedback!
This interface will be used in the three patches you looked at, although you
are correct that they have not been updated yet. For instance, the
LocalityMulticastAMRMProxyFederationPolicy prototype in YARN-5325 uses the
FederationSubClusterResolver interface to split resource requests. From the
javadoc:
host localized ResourceRequest are always forwarded to the RM
that owns the node, based on the feedback of a FederationSubClusterResolver
rack localized ResourceRequest are forwarded to the RM that owns
the rack (if the FederationSubClusterResolver provides this info) or
they are forwarded as if they were ANY (this is important for deployment that
stripe racks across sub-clusters) as there is not a single resolution.
ANY request corresponding to node/rack local requests are only forwarded
to the set of RMs that owns the node-local requests. The number of containers
listed in each ANY is proportional to the number of node-local container
requests (associated to this ANY via the same allocateRequestId)
There are some examples of the resolver being used in the splitResourceRequests
method of the same class, although some of the classnames are out of date.
The FederationInterceptor from YARN-5325 will be responsible for managing the
lifecyle of the SubClusterResolver.
I think it's better to leave the SubClusterResolver methods non-static, since
we want to allow the implementation to be pluggable and I can't think of a
particular reason it should be static. Please let me know if you disagree, I
may be missing something.
Thanks!
> Federation Subcluster Resolver
> --
>
> Key: YARN-5390
> URL: https://issues.apache.org/jira/browse/YARN-5390
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Carlo Curino
>Assignee: Ellen Hui
> Attachments: YARN-5390-YARN-2915.v0.patch,
> YARN-5390-YARN-2915.v1.patch, YARN-5390-YARN-2915.v2.patch
>
>
> This JIRA tracks effort to create a mechanism to resolve nodes/racks resource
> names to sub-cluster identifiers. This is needed by the federation policies
> in YARN-5323, YARN-5324, YARN-5325 to operate correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5390) Federation Subcluster Resolver
[ https://issues.apache.org/jira/browse/YARN-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403189#comment-15403189 ] Ellen Hui commented on YARN-5390: - Hi [~wangda], thanks for the quick feedback! This interface will be used in the three patches you looked at, although you are correct that they have not been updated yet. For instance, the LocalityMulticastAMRMProxyFederationPolicy prototype in YARN-5325 uses the FederationSubClusterResolver interface to split resource requests. From the javadoc: host localized ResourceRequest are always forwarded to the RM that owns the node, based on the feedback of a FederationSubClusterResolver rack localized ResourceRequest are forwarded to the RM that owns the rack (if the FederationSubClusterResolver provides this info) or they are forwarded as if they were ANY (this is important for deployment that stripe racks across sub-clusters) as there is not a single resolution. ANY request corresponding to node/rack local requests are only forwarded to the set of RMs that owns the node-local requests. The number of containers listed in each ANY is proportional to the number of node-local container requests (associated to this ANY via the same allocateRequestId) There are some examples of the resolver being used in the splitResourceRequests method of the same class, although some of the classnames are out of date. The FederationInterceptor from YARN-5325 will be responsible for managing the lifecyle of the SubClusterResolver. I think it's better to leave the SubClusterResolver methods non-static, since we want to allow the implementation to be pluggable and I can't think of a particular reason it should be static. Please let me know if you disagree, I may be missing something. Thanks! > Federation Subcluster Resolver > -- > > Key: YARN-5390 > URL: https://issues.apache.org/jira/browse/YARN-5390 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Carlo Curino >Assignee: Ellen Hui > Attachments: YARN-5390-YARN-2915.v0.patch, > YARN-5390-YARN-2915.v1.patch, YARN-5390-YARN-2915.v2.patch > > > This JIRA tracks effort to create a mechanism to resolve nodes/racks resource > names to sub-cluster identifiers. This is needed by the federation policies > in YARN-5323, YARN-5324, YARN-5325 to operate correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5406) In-memory based implementation of the FederationMembershipStateStore
[ https://issues.apache.org/jira/browse/YARN-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5406: Attachment: YARN-5406-YARN-2915.v0.patch > In-memory based implementation of the FederationMembershipStateStore > > > Key: YARN-5406 > URL: https://issues.apache.org/jira/browse/YARN-5406 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Ellen Hui > Attachments: YARN-5406-YARN-2915.v0.patch > > > YARN-3662 defines the FederationMembershipStateStore API. This JIRA tracks an > in-memory based implementation which is useful for both single-box testing > and for future unit tests that depend on the state store. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5390) Federation Subcluster Resolver
[ https://issues.apache.org/jira/browse/YARN-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5390: Attachment: YARN-5390-YARN-2915.v2.patch > Federation Subcluster Resolver > -- > > Key: YARN-5390 > URL: https://issues.apache.org/jira/browse/YARN-5390 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Carlo Curino >Assignee: Ellen Hui > Attachments: YARN-5390-YARN-2915.v0.patch, > YARN-5390-YARN-2915.v1.patch, YARN-5390-YARN-2915.v2.patch > > > This JIRA tracks effort to create a mechanism to resolve nodes/racks resource > names to sub-cluster identifiers. This is needed by the federation policies > in YARN-5323, YARN-5324, YARN-5325 to operate correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5390) Federation Subcluster Resolver
[ https://issues.apache.org/jira/browse/YARN-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5390: Attachment: YARN-5390-YARN-2915.v1.patch > Federation Subcluster Resolver > -- > > Key: YARN-5390 > URL: https://issues.apache.org/jira/browse/YARN-5390 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Carlo Curino >Assignee: Ellen Hui > Attachments: YARN-5390-YARN-2915.v0.patch, > YARN-5390-YARN-2915.v1.patch > > > This JIRA tracks effort to create a mechanism to resolve nodes/racks resource > names to sub-cluster identifiers. This is needed by the federation policies > in YARN-5323, YARN-5324, YARN-5325 to operate correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398258#comment-15398258
]
Ellen Hui commented on YARN-5203:
-
All of these unit tests pass locally.
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Labels: api-breaking, bug, incompatible
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch,
> YARN-5203.v2.patch, YARN-5203.v3.patch, YARN-5203.v4.patch, YARN-5203.v5.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: YARN-5203.v5.patch
Add missing license to ExecutionTypeRequestInfo
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Labels: api-breaking, bug, incompatible
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch,
> YARN-5203.v2.patch, YARN-5203.v3.patch, YARN-5203.v4.patch, YARN-5203.v5.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: YARN-5203.v4.patch
Thanks [~subru] for the feedback.
- Add ExecutionTypeRequestInfo object
- Change list of ResourceRequestInfo in AppInfo to private
- Rename testUnmarshalApp to testUnmarshalAppInfo
There is still one checkstyle error from too many arguments to
verifyResourceRequestsGeneric in TestRMWebServicesApps, but I am inclined to
ignore it since verifyAppInfoGeneric does the same thing.
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Labels: api-breaking, bug, incompatible
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch,
> YARN-5203.v2.patch, YARN-5203.v3.patch, YARN-5203.v4.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393063#comment-15393063
]
Ellen Hui commented on YARN-5203:
-
Sure, added in TestRMWebServicesApps in v3 of the patch. As expected, the test
fails with an UnmarshalException when tested against trunk, and passes when run
with the full patch.
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Labels: api-breaking, bug
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch,
> YARN-5203.v2.patch, YARN-5203.v3.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: YARN-5203.v3.patch
Feedback from [~subru]. Rename ResourceRequestsInfo field and getter back to
ResourceRequests, and add a test case to verify that the XML element can be
parsed directly into an AppInfo object.
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Labels: api-breaking, bug
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch,
> YARN-5203.v2.patch, YARN-5203.v3.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390334#comment-15390334
]
Ellen Hui commented on YARN-5203:
-
I think all of the errors here are false positives. The unread fields findbugs
is seeing are XML elements, and so is one of the protected fields that
checkstyle is seeing. The second checkstyle error is from 10 arguments in a
test utility method, which I think is fine.
testUAMRecoveryOnRMWorkPreservingRestart passes locally, I will check it again.
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch,
> YARN-5203.v2.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: YARN-5203.v2.patch
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch,
> YARN-5203.v2.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5390) Federation Subcluster Resolver
[ https://issues.apache.org/jira/browse/YARN-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ellen Hui updated YARN-5390: Attachment: YARN-5390-YARN-2915.v0.patch > Federation Subcluster Resolver > -- > > Key: YARN-5390 > URL: https://issues.apache.org/jira/browse/YARN-5390 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Carlo Curino >Assignee: Ellen Hui > Attachments: YARN-5390-YARN-2915.v0.patch > > > This JIRA tracks effort to create a mechanism to resolve nodes/racks resource > names to sub-cluster identifiers. This is needed by the federation policies > in YARN-5323, YARN-5324, YARN-5325 to operate correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385137#comment-15385137
]
Ellen Hui commented on YARN-5203:
-
Hi [~sunilg], thanks for the comment. I tested this more thoroughly and
[~subru] is right; keeping the old resourceRequests element causes an
UnmarshalException in the Federation Router, which was the original symptom of
this bug.
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: YARN-5203.v1.patch
Add ExecutionType, raw ResourceRequests for backwards compatibility.
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch, YARN-5203.v1.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: YARN-5203.v0.patch
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: (was: YARN-5203.v0.patch)
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: (was: YARN-5203.v0.patch)
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: YARN-5203.v0.patch
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: (was:
0001-YARN-5203.-Wrap-ResourceRequest-in-JAXB-object-in-RM.patch)
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: YARN-5203.v0.patch
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments: YARN-5203.v0.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5203) Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API
[
https://issues.apache.org/jira/browse/YARN-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ellen Hui updated YARN-5203:
Attachment: 0001-YARN-5203.-Wrap-ResourceRequest-in-JAXB-object-in-RM.patch
> Return ResourceRequest JAXB object in ResourceManager Cluster Applications
> REST API
> ---
>
> Key: YARN-5203
> URL: https://issues.apache.org/jira/browse/YARN-5203
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Subru Krishnan
>Assignee: Ellen Hui
> Attachments:
> 0001-YARN-5203.-Wrap-ResourceRequest-in-JAXB-object-in-RM.patch
>
>
> The ResourceManager Cluster Applications REST API returns {{ResourceRequest}}
> as String rather than a JAXB object. This prevents downstream tools like
> Federation Router (YARN-3659) that depend on the REST API to unmarshall the
> {{AppInfo}}. This JIRA proposes updating {{AppInfo}} to return a JAXB version
> of the {{ResourceRequest}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
